EP1222608A1 - Verfahren und gerät zur bildregistrierung, mit verwendung diffeomorphismen mit grosserer deformation auf einer kugel - Google Patents
Verfahren und gerät zur bildregistrierung, mit verwendung diffeomorphismen mit grosserer deformation auf einer kugelInfo
- Publication number
- EP1222608A1 EP1222608A1 EP00965280A EP00965280A EP1222608A1 EP 1222608 A1 EP1222608 A1 EP 1222608A1 EP 00965280 A EP00965280 A EP 00965280A EP 00965280 A EP00965280 A EP 00965280A EP 1222608 A1 EP1222608 A1 EP 1222608A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- image
- template image
- target
- large deformation
- points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 75
- 230000009466 transformation Effects 0.000 claims description 61
- 238000012545 processing Methods 0.000 claims description 9
- 230000000737 periodic effect Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims 2
- 230000006870 function Effects 0.000 description 46
- 230000000875 corresponding effect Effects 0.000 description 26
- 238000013507 mapping Methods 0.000 description 23
- 238000004422 calculation algorithm Methods 0.000 description 21
- 238000005457 optimization Methods 0.000 description 19
- 230000015572 biosynthetic process Effects 0.000 description 17
- 238000003786 synthesis reaction Methods 0.000 description 17
- 238000013459 approach Methods 0.000 description 15
- 210000004556 brain Anatomy 0.000 description 13
- 238000000844 transformation Methods 0.000 description 10
- 239000013598 vector Substances 0.000 description 10
- 238000006073 displacement reaction Methods 0.000 description 7
- 239000011159 matrix material Substances 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 5
- 230000004927 fusion Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 4
- 210000003484 anatomy Anatomy 0.000 description 4
- 230000008901 benefit Effects 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 4
- 230000001054 cortical effect Effects 0.000 description 4
- 239000007787 solid Substances 0.000 description 4
- 239000002131 composite material Substances 0.000 description 3
- 238000002939 conjugate gradient method Methods 0.000 description 3
- 238000013500 data storage Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 230000033001 locomotion Effects 0.000 description 3
- 238000002595 magnetic resonance imaging Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 125000004122 cyclic group Chemical group 0.000 description 2
- 238000013501 data transformation Methods 0.000 description 2
- 238000000537 electroencephalography Methods 0.000 description 2
- 238000002599 functional magnetic resonance imaging Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000002582 magnetoencephalography Methods 0.000 description 2
- 239000012528 membrane Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012831 peritoneal equilibrium test Methods 0.000 description 2
- 238000012636 positron electron tomography Methods 0.000 description 2
- 238000012877 positron emission topography Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000002603 single-photon emission computed tomography Methods 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 238000002604 ultrasonography Methods 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- 241000510032 Ellipsaria lineolata Species 0.000 description 1
- 241000255777 Lepidoptera Species 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 210000001638 cerebellum Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
- G06T3/153—Transformations for image registration, e.g. adjusting or mapping for alignment of images using elastic snapping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/754—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries involving a deformation of the sample pattern or of the reference pattern; Elastic matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30016—Brain
Definitions
- the present invention relates to image processing systems and methods, and more particularly to image registration systems that combine two or more images into a composite image; in particular the fusion of anatomical manifold-based knowledge with volume imagery via large deformation mapping which supports both kinds of information simultaneously, as well as individually, and which can be implemented on a rapid convolution FFT based computer system.
- Image registration involves combining two or more images, or selected points from the images, to produce a composite image containing data from each of the registered images. During registration, a transformation is computed that maps related points among the combined images so that points defining the same structure in each of the combined images are correlated in the composite image.
- Each of the coefficients, k t is assumed
- mapping relationship u(x) is extended from the set of N landmark points to the continuum using a linear quadratic form regularization optimization of the equation:
- Timoshenko Theory of Elasticity, McGraw-Hill, 1934 (hereinafter referred to as Timoshenko) and R.L. Bisplinghoff, J.W. Marr, and T.H.H. Pian, Statistics ofDeformable Solids, Dover Publications, Inc., 1965 (hereinafter referred to as Bisplinghoff).
- Others have used this operator in their work, see e.g., Amit, U. Grenander, and M. Piccioni, "Structural image restoration through deformable templates," J. American Statistical Association.
- a distance measure represented by the expression D(u)
- D(u) represents the distance between a template T(x) and a target image S(x).
- the distance measure D(u) measuring the disparity between imagery has various forms, e.g., the Gaussian squared error distance J a correlation distance, or a
- fusion approaches involve small deformation mapping coordinates x ⁇ ⁇ of one set of imagery to a second set of imagery.
- Other techniques include the mapping of predefined
- the distance measure changes depending upon whether landmarks or imagery are being matched.
- the field u(x) specifying the mapping h is extended from the 5 set of points ⁇ .t, ⁇ identified in the target to the points ⁇ y,- ⁇ measured with Gaussian error co- variances ⁇ ,- :
- the second approach is purely volume image data driven, in which the volume
- h(-) - - «( ⁇ )
- u arg min (5)
- the data function D(u) measures the disparity between imagery and has various forms. Other distances are used besides the Gaussian squared error distance, including correlation distance, Kullback Liebler distance, and others.
- small deformation methods provide geometrically meaningful deformations under conditions where the imagery being matched are small, linear, or affine changes from one image to the other.
- Small deformation mapping does not allow the automatic calculation of tangents, curvature, surface areas, and geometric properties of the imagery.
- Fig. 9 shows an oval template image with several landmarks highlighted.
- Fig. 10 shows a target image that is greatly deformed from the template image. The target image is a largely deformed oval that has been twisted.
- Fig. 1 1 shows the results of image
- the diffeomorphic transformations constructed are of high dimensions having, for example a dimension greater than 12 of the Affine transform up-to the order of the number of voxels in the volume.
- a transformation is diffeomorphic if the transformation from the template to the target is one-to-one, onto,
- a transformation is said to be one-to-one if no two distinct points in the template are mapped to the same point in the target.
- a transformation is said to be onto if every point in the target is mapped from a point in the template. The importance of generating diffeomorphisms is that tangents,
- Fig. 12 illustrates the image mapping illustrated in Fig. 1 1 using diffeomorphic transformation.
- the present invention overcomes the limitations of the conventional techniques of
- image registration by providing a methodology which combines, or fuses, some aspects of techniques where an individual with expertise in the structure of the object represented in the images labels a set of landmarks in each image that are to be registered and techniques that use mathematics of small deformation multi-target registration, which is purely image data driven.
- a large deformation transform is computed using the selected coordinate frame, a manifold landmark transformation operator, and at least one manifold landmark transformation boundary value.
- the large deformation transform relates
- Fig. 1 is a target and template image of an axial section of a human head with 0- dimensional manifolds
- Fig. 2 is schematic diagram illustrating an apparatus for registering images in accordance with the present invention
- Fig. 3 is a flow diagram illustrating the method of image registration according to the
- Fig. 4 is a target and a template image with 1-dimensional manifolds
- Fig. 5 is a target and a template image with 2-dimensional manifolds
- Fig. 6 is a target and a template image with 3-dimensional manifolds
- Fig. 7 is sequence of images illustrating registration of a template and target image.
- Fig. 8 is a flow diagram illustrating the computation of a fusing transform
- Fig. 9 is an oval template image which has landmark points selected and highlighted
- Fig. 10 is a deformed and distorted oval target image with corresponding landmark points highlighted and selected;
- Fig. 1 1 is an image matching of the oval target and template images; and Fig. 12 is an image matching using diffeomorphism.
- Fig. 1 shows two axial views of a human head.
- template image 100 contains points 102, 104, and 1 14 identifying
- Target image 120 contains points 108, 110, 116, corresponding respectively to template image points 102, 104, 114, via vectors 106, 1 12, 1 18, respectively.
- Fig. 2 shows apparatus to carry out the preferred embodiment of this invention.
- medical imaging scanner 214 obtains the images show in Fig. 1 and stores them on a computer
- CPU central processing unit
- a parallel computer platform having multiple CPUs is also a suitable hardware platform for the present invention, including, but not limited to, massively parallel machines and workstations with multiple processors.
- Computer memory 206 which is connected to a computer central processing unit (CPU) 204.
- CPU central processing unit
- One of ordinary skill in the art will recognize that a parallel computer platform having multiple CPUs is also a suitable hardware platform for the present invention, including, but not limited to, massively parallel machines and workstations with multiple processors.
- Computer memory 206 which is connected to a computer central processing unit (CPU) 204.
- CPU central processing unit
- CPU 204 can be directly connected to CPU 204, or this memory can be remotely connected through
- Registering images 100, 120 unifies registration based on landmark deformations and image data transformation using a coarse-to-fine approach.
- the highest dimensional transformation required during registration is computed from the solution of a sequence of lower dimensional problems driven by successive refinements.
- the method is based on information either provided by an operator,
- an operator using pointing device 208, moves cursor 210 to select points 102, 104, 1 14 in Fig. 1, which are then displayed on a computer monitor 202 along with images 100, 120.
- Selected image points 102, 104, and 1 14 are 0- dimensional manifold landmarks.
- CPU 204 computes a first transform relating the manifold landmark points in template image 100 to their corresponding image points in target image 120.
- a second CPU 204 transform is computed by fusing the first
- the operator can select an equation for the distance measure several ways including, but not limited to, selecting an
- Registration is completed by CPU 204 applying the second computed transform to all points in the template image 100.
- the transforms, boundary values, region of interest, and distance measure can be defaults read from memory or determined automatically.
- Fig. 3 illustrates the method of this invention in operation.
- First an operator defines a set of N manifold landmark points x t where /,..., N, represented by the variable M, in the template image (step 300). These points should correspond to points that are easy to identify in the target image.
- each landmark point, x t , in the template image is a corresponding point y i in the target image.
- the operator therefore next identifies the corresponding points, y ( , in the target image are identified (step 310).
- the nature of this process means that the corresponding points can only be identified within some degree of accuracy.
- This mapping between the template and target points can be specified with a resolution having a Gaussian error of
- K.(x,x) is the Green's function of a volume landmark transformation operator L 2 (assuming L is self-adjoint):
- the operator may select a region of interest in the target image. Restricting the computation to a relatively small region of interest reduces both computation and storage requirements because transformation is computed only over a subregion of interest. It is also possible to select a region of interest in the target image. Restricting the computation to a relatively small region of interest reduces both computation and storage requirements because transformation is computed only over a subregion of interest. It is also possible to select a region of interest in the target image.
- the entire image is the desired region of interest. In other applications, there may be default regions of interest that are automatically identified.
- the number of computations required is proportional to the number of points in the region of interest, so the computational savings equals the ratio of the total number of points in
- N points with a region of interest having M points is a factor o ⁇ N/M.
- the computation time and the data storage are reduced by a factor of eight.
- performing the computation only over the region of interest makes it necessary only to store a subregion, providing a data storage savings for the template image, the target image, and the transform values.
- CPU 204 computes a transform that embodies the mapping relationship between these two sets of points (step 350).
- This transform can be estimated using Bayesian optimization, using the following equation:
- A is a 3 x 3 matrix
- b [b b 2 , b ⁇ ] is a 3 x 1 vector
- the foregoing steps of the image registration method provide a coarse matching of a template and a target image.
- Fine matching of the images requires using the full image data and the landmark information and involves selecting a distance measure by solving a synthesis equation that simultaneously maps selected image landmarks in the template and target images and matches all image points within a region of interest.
- An example of this synthesis equation is: arg min ⁇ f ⁇ T(x - u(x)) - S(x) ⁇ 2 dx + f ⁇ Lu ⁇ 2 ⁇ ⁇ ⁇ y ' ⁇ ⁇ ⁇ '° ' " ( ⁇ )
- the operator L in equation (1 1) may be the same operator used in equation (9), or alternatively, another operator may be used with a different set of boundary conditions.
- the distance measure in the first term measures the relative position of points in the target image with respect to points in the template image.
- this synthesis equation uses a quadratic distance measure, one of ordinary skill in the art will recognize that there are other suitable distance measures.
- CPU 204 then computes a second or fusing transformation (Step 370) using the synthesis equation relating all points within a region of interest in the target image to all corresponding points in the template image.
- the synthesis equation is defined so that the resulting transform incorporates, or fuses, the mapping of manifold landmarks to corresponding target image points determined when calculating the first transform.
- the computation using the synthesis equation is accomplished by solving a sequence of optimization problems from coarse to fine scale via estimation of the basis coefficients ⁇ .
- This is analogous to multi-grid methods, but here the notion of refinement from coarse to fine is accomplished by increasing the number of basis components d. As the number of basis functions increases, smaller and smaller variabilities between the template and target are accommodated.
- the basis coefficients are determined by gradient descent, i.e.,
- ⁇ is a fixed step size and ⁇ are the eigenvalues of the eigenvectors ⁇ k .
- Equation (13) is then used to estimate the new values of the basis coefficients ⁇ k n'l) given the current estimate of the displacement field i "'(x) (step 804).
- Equation (15) is then used to compute the new estimate of the displacement field ⁇ n> (x) given the current estimate of the basis coefficients ⁇ k (n> (step 806).
- the next part of the computation is to decide whether or not to increase the number d of basis functions ⁇ used to represent the transformation (step 80S). Increasing the number of basis functions allows more deformation. Normally, the algorithm is started with a small number of basis functions corresponding to low frequency eigen functions and then on defined iterations the number of frequencies is increased by one (step 810). This coarse-to-fine strategy matches larger structures before smaller structures. The preceding computations (steps 804-810) are repeated until the computation has converged or the maximum number of iterations is reached (step 812). The final displacement field is then used to transform the template image (step 814).
- CPU 204 uses this transform to register the template image with the target image (step 380).
- the spectrum of the second transformation, h is highly concentrated around zero. This means that the spectrum mostly contains low frequency components.
- the transformation can be represented by a subsampled version provided that the sampling frequency is greater than the Nyquist frequency of the transformation.
- the computation may be accelerated by computing the transformation on a coarse grid and extending it to the full voxel lattice e.g., in the case of 3D images, by interpolation.
- the computational complexity of the algorithm is proportional to the dimension of the lattice on which the transformation is computed. Therefore, the computation acceleration equals the ratio of the full voxel lattice to the coarse computational lattice.
- Fig. 4 shows a template image 400 of a section of a brain with 1- dimensional manifolds 402 and 404 corresponding to target image 406 1-dimensional manifolds 408 and 410 respectively.
- Fig. 5 shows a template image 500 of a section of a brain with 2-dimensional manifold 502 corresponding to target image 504 2-dimensional manifold 506.
- Fig. 6 shows a template image 600 of a section of a brain with 3-dimensional manifold 602 corresponding to target image 604 3-dimensional manifold 606.
- M(3),dS is the Lebesgue measure on P
- dS is the surface measure on M(2)
- dS is the line measure on Mfl
- M(0), dS is the atomic measure.
- the Fredholm integral equation degenerates into a summation given by equation (10).
- step 370 It is also possible to compute the transform (step 370) with rapid convergence by solving a series of linear minimization problems where the solution to the series of linear problems converges to the solution of the nonlinear problem. This avoids needing to solve the nonlinear minimization problem directly.
- the computation converges faster than a direct solution of the synthesis equation because the basis coefficients ⁇ k are updated with optimal step sizes.
- step 370 of Fig. 3 computing the registration transform fusing landmark and image data, is implemented using the conjugate gradient method, the computation will involve a series of inner products.
- the FFT exploits the structure of the eigen functions and the computational efficiency of the FFT to compute these inner-products.
- one form of a synthesis equation for executing Step 370 of Fig. 3 will include the following three terms:
- a distance function used to measure the disparity between images is the Gaussian square error distance j
- distance functions such as the correlation distance, or the Kullback Liebler distance, can be written in the form ⁇ D(T(x-u(x)) , S(x))dx.
- D(.,.) is a distance function relating points in the template and target images.
- the displacement field is assumed to have the form:
- the basis coefficients ⁇ ⁇ . are determined by gradient descent, i.e.,
- D ' (.,.) is the derivative with respect to the first argument.
- D ' (.,.) is the derivative with respect to the first argument.
- each of the inner-products in the algorithm if computed directly, would have a computational complexity of the order ( ⁇ 3 ) 2 .
- the overall complexity of image registration is also ( ⁇ 3 ) 2 .
- each of the FFTs proposed has a computational complexity on the order of ⁇ 3 log, N 3 .
- boundary conditions such as the Dirichlet, Neumann, or mixed Dirichlet and Neumann boundary conditions are also suitable.
- the following equation is used in an embodiment of the present invention using one set of mixed Dirichlet and Neumann boundary conditions:
- Modifying boundary conditions requires modifying the butterflies of the FFT from complex exponentials to appropriate sines and cosines.
- template image 700 In Fig. 7, four images, template image 700, image 704, image 706, and target image 708, illustrate the sequence of registering a template image and a target image.
- Template image 700 has 0-dimensional landmark manifolds 702. Applying the landmark manifold transform computed at step 350 in Fig. 3 to image 700 produces image 704. Applying a second transform computed using the synthesis equation combining landmark manifolds and image data to image 700 produces image 706.
- Image 706 is the final result of registering template image 700 with target image 708.
- Landmark manifold 710 in image 708 corresponds to landmark manifold 702 in template image 700.
- the large deformation maps h: ⁇ ⁇ ⁇ are constructed by introducing the time variable,
- the distance between the target and template imagery landmarks is preferably defined as .
- a preferable diffeomo hism is the minimizer of Eqns. 40, 41 with D(u(T)) the landmark distance:
- ⁇ (x,T) f T (I -Vu (x,t))v(x,t)dt ( 43)
- a diffeomo ⁇ hic map is computed using an image transformation operator and image transformation boundary values relating the template image to the target image. Subsequently the template image is registered with the target image using the diffromo ⁇ hic map.
- ⁇ (x,T) f T (I - j ⁇ (x,t))v(x,t)dt and (48)
- a method for registering images consistent with the present invention preferably includes the following steps:
- STEP 2 Solve optimization via sequence of optimization problems from coarse to fine scale via re-estimation of the basis coefficients ⁇ v , analogous to multi-grid methods with the notion of refinement from coarse to fine accomplished by increasing the number of basis components. For each v k ,
- the differential operator L can be chosen to be any in a class of linear differential operators; we have used operators of the form (-a ⁇ - bvv +ciy, p ⁇ 1. The operators are 3 3 matrices
- transformations may be chosen, including the affine motions, rigid motions generated from subgroups of the generalized linear group, large deformation landmark transformations which are diffeomo ⁇ hisms, or the high dimensional large deformation image matching transformation (the dimension of the transformations of the vector fields are listed in increasing order). Since these are all diffeomo ⁇ hisms, they can be composed.
- the particle flows ⁇ (t) are defined by the velocities v(-) according to the fundamental O.D.E.
- N N v(x,t) ⁇ Kt ⁇ (ts ) ⁇ (K( ⁇ (t)y ⁇ (x t) (62)
- the method of the present embodiment utilizes the Largrangian positions.
- STEP 4 After stopping, then compute the optimal velocity field using equation 62 and transform using equation 64.
- h(-, t) ⁇ - ⁇
- h(x, t) x - u(x, t)
- the transformation and velocity fields are related via the O.D.E d duu((xx.,tt)) v(x,t) + Vu(x,t)v(x,t),t ⁇ [0,7J
- the deformation fields are di generated from the velocity fields assumed to be piecewise constant over quantized time
- the body force b(x-u(x,t)) is given by the variation of the distance D(u) with respect to the field at time t.
- the PDE is solved numerically (G. E. Christensen, R. D. Rabbitt, and M. I. Miller, "Deformable templates using large deformation kinematics," 7E££ Transactions on Image Processing, 5(10): 1435- 1447, October 1996 for details (hereinafter referred to as Christensen).
- ⁇ ( ', ) J i f ⁇ a ⁇ d j. r ⁇ ( - " )( v,- )) - ⁇ t ⁇ v/ oi * - «fc ' • e - V U ( ⁇ ) ⁇ W d t
- Section 5 The implementation presented in Section 5 is modified for different boundary conditions by modifying the butterflys of the FFT from complex exponentials to appropriate sines and cosines.
- a method for registering images using large deformation diffeomo ⁇ hisms on a sphere comprises selecting a coordinate frame suitable for spherical geometries and registering the target and template image using a large deformation diffeomo ⁇ hic transform in the selected coordinate frame. While there are many applications that benefit from an image registration technique adapted to spherical geometries, one such example, registering brain images, is discussed herein to illustrate the technique. One skilled in the art will recognize that other imaging applications are equally suited for this technique, such as registering images of other anatomical regions and registering non-anatomical imagery containing spherical regions of interest.
- An application of brain image registration is the visualization of conical studies.
- Current methods of visualizing cortical brain studies use flat maps. Although flat maps bring the buried cortex into full view and provide compact representations, limitations are introduced by the artificial cuts needed to preserve topological relationships across the cortical surface. Mapping the cortical hemisurface to a sphere, however, allows points on the surface to be represented by a two-dimensional coordinate system that preserves the topology. Spherical maps allow visualization of the full extent of sulci and the buried cortex within the folds.
- An embodiment consistent with the present invention generates large deformation diffeomo ⁇ hisms on the sphere S 2 which has a one-to-one correspondence with a reconstructed cortical surface.
- the final transformed coordinate map is defined as ⁇ ( ,1) ⁇
- the template image and target image spheres are characterized by the set of landmarks ⁇ xicide,y ⁇ , n - 1,2, ..N ⁇ S 2 .
- Diffeomo ⁇ hic matches are constructed by forcing the velocity fields to be associated with quadratic energetics on S 2 x [0, 1].
- the diffeomo ⁇ hic landmark matching is constructed to minimize a running smoothness energy on the velocity field as well as the end point distance between the template and target landmarks.
- I is a differential operator
- ⁇ (x,y) is the solid angle between points x,y on the sphere
- d ⁇ (x) sin ⁇ d ⁇ d ⁇ is the surface measure.
- the coordinate frames are given by — and according to :
- each hemisphere of the brain is mapped to a sphere individually, and the area they are attached to each other is transformation invariant. For some applications, this conforms to an anatomical constraint, e.g., the point at which the brain attached to the spine.
- the location producing zero-valued coordinate frames can be user selected by computing a rigid alignment to move this location in the image.
- One example of such an alignment involves aligning the sets of landmarks by computing a rigid transform (which, in a spherical coordinate frame, is a rotation) and then applying another rigid transform to ensure that the fixed points are aligned correctly.
- An embodiment of the present invention uses stereographic projection to parameterize the unit sphere corresponding to the right hemisphere of the brain, with the center shifted to (-1,0,0) and take the shadow (u,v) of each point in the yz plane while shining a light from point (-2,0,0). Note that the shadow of the point from where the light source is located is a infinity, and this becomes the point where the coordinate frames vanish. The location of the light source and the projection plane can be adjusted to place the transformation invariant point as needed.
- P denote the stereographic projection from S 2 ⁇ (-2,0,0) to (u,v) ⁇ JF 2 :
- I6 «(9,y) (4v( ⁇ ,uQ 2 -4a( ⁇ , ⁇ ) 2 ⁇ 16) -8 «( ⁇ , ⁇ )v(8, ⁇ ) , ⁇ ⁇ V( ⁇ , ⁇ )+v 2 ( ⁇ , ⁇ ) + 4) 2 ' ( u ( ⁇ , ⁇ ) 2 ⁇ v( ⁇ >V ) 2 ⁇ 4) 2 ( ⁇ , ⁇ ) 2 + v( ⁇ , ⁇ ) 2 + 4) 2 l8
- a similar parameterization can be obtained for the left hemisphere by shifting the center to (1,0,0) and shining the light from (2,0,0) which becomes the transformation invariant point. Accordingly, an appropriate coordinate frame for images containing objects having spherical geometry is generated for subsequent image registration.
- the covariance operator is then computed using spherical harmonics.
- the covariance operator (which is the Green's function squared of the Laplacian operator) will determine the solution to the diffeomo ⁇ hic matching (see equation 93 below).
- There are (2n ⁇ 1) spherical harmonics of order n for each n and they are of the even and odd harmonic form with 0 ⁇ m ⁇ n: ( ⁇ , ⁇ ) k ⁇ P ⁇ cosy )cosm ⁇ (88)
- An embodiment of the present invention uses a gradient algorithm to register the images o
- ⁇ [t ; . - 1] 1 for t k - 1, and 0 otherwise, ⁇ is the gradient step.
- v, v (m ⁇ , and for all x ⁇ S 2
- FIG. 2 shows an apparatus to carry out an embodiment of this invention.
- a medial imaging scanner 214 obtains image 100 and 120 and stores them in computer memory 206 which is connected to computer control processing unit (CPU) 204.
- CPU computer control processing unit
- CPU 204 One of the ordinary skill in the art will recognize that a parallel computer platform having multiple CPUs is also a suitable hardware platform for the present invention, including, but not limited to, massively parallel machines and workstations with multiple processors.
- Computer memory 206 can be directly connected to CPU 204, or this memory can be remotely connected through a communications network.
- the methods described herein use information either provided by an operator, stored as defaults, or determined automatically about the various substructures of the template and the target, and varying degrees of knowledge about these substructures derived from anatomical imagery, acquired from modalities like CT, MRI, functional MRI, PET, ultrasound, SPECT, MEG, EEG, or cryosection.
- an operator can guide cursor 210 using pointing device 208 to select in image 100.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15514199P | 1999-09-22 | 1999-09-22 | |
US155141P | 1999-09-22 | ||
PCT/US2000/025971 WO2001022352A1 (en) | 1999-09-22 | 2000-09-22 | Method and apparatus for image registration using large deformation diffeomorphisms on a sphere |
Publications (1)
Publication Number | Publication Date |
---|---|
EP1222608A1 true EP1222608A1 (de) | 2002-07-17 |
Family
ID=22554253
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP00965280A Withdrawn EP1222608A1 (de) | 1999-09-22 | 2000-09-22 | Verfahren und gerät zur bildregistrierung, mit verwendung diffeomorphismen mit grosserer deformation auf einer kugel |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP1222608A1 (de) |
AU (1) | AU7601900A (de) |
WO (1) | WO2001022352A1 (de) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10176570B2 (en) * | 2016-11-16 | 2019-01-08 | Sony Corporation | Inter-patient brain registration |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4906940A (en) * | 1987-08-24 | 1990-03-06 | Science Applications International Corporation | Process and apparatus for the automatic detection and extraction of features in images and displays |
-
2000
- 2000-09-22 EP EP00965280A patent/EP1222608A1/de not_active Withdrawn
- 2000-09-22 WO PCT/US2000/025971 patent/WO2001022352A1/en not_active Application Discontinuation
- 2000-09-22 AU AU76019/00A patent/AU7601900A/en not_active Abandoned
Non-Patent Citations (1)
Title |
---|
See references of WO0122352A1 * |
Also Published As
Publication number | Publication date |
---|---|
WO2001022352A1 (en) | 2001-03-29 |
AU7601900A (en) | 2001-04-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6633686B1 (en) | Method and apparatus for image registration using large deformation diffeomorphisms on a sphere | |
EP1057137B1 (de) | Erkennung verzerrter bilder durch schnelle faltung und mittels bilderfassung von markanten punkten und volumeninformation | |
EP0910832B1 (de) | Verfahren und gerät zum ausrichten von bildern | |
Zhou et al. | Cocosnet v2: Full-resolution correspondence learning for image translation | |
US6408107B1 (en) | Rapid convolution based large deformation image matching via landmark and volume imagery | |
Botsch et al. | Primo: coupled prisms for intuitive surface modeling | |
Liwicki et al. | Euler principal component analysis | |
US7561757B2 (en) | Image registration using minimum entropic graphs | |
CN108027878A (zh) | 用于面部对齐的方法 | |
WO2001043070A2 (en) | Method and apparatus for cross modality image registration | |
Agudo et al. | Online Dense Non-Rigid 3D Shape and Camera Motion Recovery. | |
Steedly et al. | Spectral Partitioning for Structure from Motion. | |
Gu et al. | Matching 3d shapes using 2d conformal representations | |
Koehl et al. | Automatic alignment of genus-zero surfaces | |
Grossmann et al. | Computational surface flattening: a voxel-based approach | |
Hermann et al. | A visual analytics perspective on shape analysis: State of the art and future prospects | |
Cootes | Statistical shape models | |
WO2001022352A1 (en) | Method and apparatus for image registration using large deformation diffeomorphisms on a sphere | |
Tristán et al. | A fast B-spline pseudo-inversion algorithm for consistent image registration | |
von Klinski et al. | Model-based image processing using snakes and mutual information | |
Filip et al. | Regularized multi-structural shape modeling of the knee complex based on deep functional maps | |
Younes | Combining geodesic interpolating splines and affine transformations | |
Heldmann et al. | A variational approach for volume-to-slice registration | |
Patane et al. | Surface-and volume-based techniques for shape modeling and analysis | |
Shen et al. | Fourier methods for 3D surface modeling and analysis |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20020415 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT |
|
AX | Request for extension of the european patent |
Free format text: AL;LT;LV;MK;RO;SI |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 20030401 |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AT BE DE FR |