WO2000036565A9 - Method and apparatus for processing images with regions representing target objects - Google Patents

Method and apparatus for processing images with regions representing target objects

Info

Publication number
WO2000036565A9
WO2000036565A9 PCT/US1999/029798 US9929798W WO0036565A9 WO 2000036565 A9 WO2000036565 A9 WO 2000036565A9 US 9929798 W US9929798 W US 9929798W WO 0036565 A9 WO0036565 A9 WO 0036565A9
Authority
WO
WIPO (PCT)
Prior art keywords
image
data elements
region
target object
elements representing
Prior art date
Application number
PCT/US1999/029798
Other languages
French (fr)
Other versions
WO2000036565A1 (en
Inventor
Michael I Miller
Ulf Grenander
Sarang C Joshi
Original Assignee
Michael I Miller
Ulf Grenander
Sarang C Joshi
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Michael I Miller, Ulf Grenander, Sarang C Joshi filed Critical Michael I Miller
Priority to AU23632/00A priority Critical patent/AU2363200A/en
Priority to EP99967332A priority patent/EP1141894B1/en
Priority to DE69943026T priority patent/DE69943026D1/en
Publication of WO2000036565A1 publication Critical patent/WO2000036565A1/en
Publication of WO2000036565A9 publication Critical patent/WO2000036565A9/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
    • G06T3/153—Transformations for image registration, e.g. adjusting or mapping for alignment of images using elastic snapping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/155—Segmentation; Edge detection involving morphological operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/32—Determination of transform parameters for the alignment of images, i.e. image registration using correlation-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/754—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries involving a deformation of the sample pattern or of the reference pattern; Elastic matching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20156—Automatic seed setting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30016—Brain
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion

Definitions

  • the present invention relates to image processing systems and methods, and more particularly to image processing systems that process images with regions representing target objects.
  • Image registration is an image processing technique for combining two or more images, or selected points from the images, to produce a composite image containing data from each of the registered images. During registration, a transformation is computed that maps related points among the combined images so that points defining the corresponding structures in each of the combined images are correlated in the composite image.
  • Image processing techniques suitable for image registration follow two different approaches.
  • the first requires that an individual with expertise in the structure of the object represented in the images label a set of landmarks in each of the images that are to be registered. For example, when registering two MRI images of different axial slices of a human head, a physician may label points, or a contour adjacent to these points, corresponding to the cerebellum in two images. The two images are then registered by relying on a known relationship among the landmarks in the two brain images.
  • the second currently-practiced technique for image registration uses the mathematics of small deformation multi-target registration and is image data driven. Here, volume based imagery is generated of the two targets from which a coordinate system transformation is constructed.
  • the present invention provides a methodology for image processing that can compensate for the presence of target objects in images.
  • a method and apparatus registers a first and a second image, where the second image contains a target object.
  • the method comprises the steps of identifying a region in the second image containing one or more image data elements representing the target object, transforming the second image to reduce the size of the region containing the target object, and registering the first image and the second image.
  • a method comprises the steps of identifying a region in the second image containing one or more image data elements representing a target object. After identifying this region, the first and second images are registered using a first transform component that
  • Fig. 1 is a flow diagram of a method for growing a seed region in accordance with the present invention
  • Fig. 2 is schematic diagram illustrating an apparatus for processing images in accordance with the present invention
  • Fig. 3 is a computer display of coronal, sagittal, and axial views of a head image containing a tumor with a defined seed region;
  • Fig. 4 is a computer display of coronal, sagittal, and axial views of a head image containing a tumor with an identified tumor region;
  • Fig. 5 is a flow diagram of an image processing method for automatically removing image data elements corresponding to an object embedded in an image in accordance with the present invention
  • Fig. 6 is a computer display of coronal, sagittal, and axial views of a head image after a tumor has been removed from the image using image processing in accordance with the present invention.
  • Fig. 7 is a flow diagram of a method for registering a target image and an atlas image in accordance with the present invention.
  • a method and system which locates objects in image data.
  • FIG. 1 illustrates a presently preferred embodiment of the invention and together with the general description given above and detailed description of the preferred embodiment given below, serve to explain the principles of the invention.
  • region includes one or more image data elements.
  • An image data element includes, for example, a pixel or a voxel.
  • a method consistent with the present invention helps an operator to locate regions in an image representing objects such as structural anomalies. The method begins with an image containing one or more target objects of interest.
  • a physician or technician can use the method to determine the extent of a mass effect tumor present in the image.
  • a mass effect tumor is a tumor that displaces the tissue of surrounding structures.
  • this type of tumor is an example of the type of object that can be localized in an image for subsequent analysis.
  • the first step of the method shown in Fig. 1 identifies a seed region in the image containing at least one data element representing diseased tissue located in the tumor (step 100).
  • the seed region can be identified in any of a number of ways. For example, a computer system can identify a data element in the MRI that represents the tumor. One suitable technique would use histogram or thresholding algorithms known in the art to identify a candidate seed region representing a portion of the tumor. Alternatively, an operator can identify the seed region by selecting a data element or group of data elements in the image representing a portion of the tumor. A mathematical texture or intensity model (or a model that embodies both texture and intensity) is then defined for data elements representing diseased tissue in the tumor and normal tissue in the image (steps 102 and 104, respectively).
  • Texture and intensity models define the distribution of the gray scale values of data elements in an image. Moreover, texture can be modeled as a structured variation of intensity. There are a number of possible texture and intensity models known in the art that are suitable for the present method, including, but not limited to Gaussian and mixtures of Gaussian distributions. Gaussian- based distributions, like other types of parametric statistical distributions, are appropriate models to use because the models can be adapted to the image by modifying only a few parameters, such as the mean and variance of the distribution. The models can be selected by software that inspects the tumor region and normal region of the image and selects appropriate models by, for example, estimating a mean and variance of the data elements in the regions of interest. Alternatively, an operator can specify appropriate models.
  • the seed region is grown to increase the number of data elements representing the tumor contained in the seed region, while decreasing the number of data elements representing normal tissue contained in the seed region (step 106).
  • the method seeks to maximize the number of data elements representing the tumor contained in the seed region, while minimizing the number of data elements representing normal tissue contained in the seed region.
  • the result of the operation of this method is to define a boundary in the image that encloses the tumor, but does not include significant amounts of normal tissue not affected by the tumor.
  • Fig. 2 is an apparatus for implementing embodiments consistent with the present invention.
  • Each of the described techniques involves computations that are preferably executed in hardware and or software stored in a computer system.
  • the images that are processed are acquired from imaging devices during processing and/or are retrieved from storage devices.
  • medical imaging scanner 214 obtains images and stores them on a computer memory 206 which is connected to a computer central processing unit (CPU) 204.
  • CPU central processing unit
  • CPU central processing unit
  • Computer memory 206 can be directly connected to CPU 204, or this memory can be remotely connected through a communications network.
  • the computational operations described herein can be allocated among several devices in a distributed network, including, but not limited to, networks linked by the Internet.
  • the computer system of Fig. 2 also can include user interfaces for operator input.
  • the interfaces include keyboard 212 and mouse 208, which controls cursor 210.
  • Display 202 has windows 216 and 218.
  • An embodiment of a process consistent with the present invention for growing a surface from a seed sphere to define the bounding surface of the tumor estimates a closed surface M which reduces an elastic regularization energy while increasing the data log- likelihood associated with the texture characteristics of the diseases and normal anatomy.
  • a target anatomical imagery I(x) in a region of interest ⁇ Begin by first defining a texture model for the interior and the exterior of diseased region of interest.
  • An example of such a model is an independent Gaussian texture model with different mean and variances for the interior and the exterior.
  • ⁇ int , o int be the mean and variance of the Gaussian texture of the tumor.
  • ⁇ i Ext , o E 2 l ⁇ t be the mean and variance of the Gaussian texture of the exterior or the normal anatomy surrounding the tumor. This defines the data likelihood model.
  • M be a ciosed triangulated surface with interior M ml . Then the energy defined by the texture model and the surface M becomes:
  • a regularization energy is selected for the prior that constrains the estimate of the surface to be smooth.
  • the surface lattice is preferably modeled as an elastic sheet. The energy is then locally given by the distance between the sites in the lattice.
  • N(x,) the set of all neighbors of vertex x, in the triangulated graph.
  • n the outward normal at each vertex.
  • a preferred iterative gradient algorithm for generating a surface minimizing combined energy is:
  • Fig.3 illustrates an application of this method to medical images.
  • Fig.3 includes four windows 300, 302, 304, and 306.
  • Windows 300, 304, and 306 contain sagittal, axial, and coronal views of a three-dimensional MRI of a patient's brain containing mass effect tumor 308.
  • An operator draws an initial seed region 310 in any one of the views, using for example a computer pointing device to select one or more image data elements that correspond to the tumor.
  • software can determine the boimdary of a seed region, by for example, selecting a group of image data elements falling in a selected bin of a histogram for the image.
  • Window 302 is a wireframe representation of the selected seed region in its initial condition prior to growing it to encompass the tumor.
  • the seed region can be, for example, a single point, a two-dimensional area, or a three-dimensional volume.
  • Fig. 4 shows the windows of Fig. 3 after growing the seed region to encompass tumor B08. As seen in Fig. 4, what was an initial region 310, has grown to region 410. In accordance with the method, region 410 was grown to maximize the volume of the tumor contained in the region, while avoiding extending 410 to encroach on normal tissue in the image. In this example, all of the tumor tissue has not been completely contained in region 410, however the results can be improved by allowing additional iterations of the method and/or refining the definitions of the intensity or texture models used for the tumor or normal tissue.
  • Fig. 4 shows coronal, sagittal, and axial views of an image where the tumor image boimdary 412 has been automatically identified using the method of Fig. 1.
  • Window 402 is a wireframe of the three-dimensional region after growing it to cover the tumor. Accordingly, by applying this method, the region of the image occupied by the tumor is automatically located and segmented from the remainder of the image.
  • Fig. 5 is a flow diagram of an image processing method for automatically removing an object embedded in a deformable substance.
  • the method is applicable to processing many types of images including, but not limited to, medical images.
  • This method is especially suited for removing tumors from medical images because as certain tumors grow in the brain, the tumors deform the structures in the brain.
  • the method begins with an image with an identified object boundary separating the object from the substance into which the object is embedded.
  • the object boundary can be automatically identified using, for example, the method of Fig. 1.
  • the region in the image representing the object to be removed can be identified by an operator using, for example, a mouse or other pointing device to trace the object boimdary on a digital representation of the image (step 500).
  • a transform is automatically computed such that when the transform is applied to the image the tumor region shrinks, preferably to a single point.
  • the transform that is computed not only shrinks the tumor region, but also predicts the shape of the normal anatomy after removal of the tumor.
  • the computed transform is applied to the image with the tumor so that the size of the region of interest is reduced and the normal shape of the image without the object in the region of interest is restored.
  • the method reverses the deformation caused by the introduction of the object in the region of interest. Accordingly, reducing the size of the image area occupied by the tumor (or reducing the proportionate size of the tumor region by enlarging or emphasizing other areas of the image), can reduce the amount of data that can skew the registration process.
  • Fig. 6 shows coronal (606), sagittal (600), and axial (604) views of the image after the operation of step 504 described above restores the image to remove the tumor.
  • the tumor has been removed from the image. 612 corresponds to the location of the tumor boundary prior to removal.
  • the representation of the anatomical structures have been restored to their normal characteristics, i.e., pre-tumor shape and density. Compare, for example, ventricle region 414 in the axial image in Fig. 4 with the corresponding representation of the ventricle region 614 in image 606 where the tumor has been electronically removed by applying the computed transform.
  • Fig. 6 shows that the result of the image processing described herein restores the representation in the image of the ventricle region and surrounding tissue to their normal appearance.
  • libraries containing reference images are used to restore, modify, or classify target images. These applications include generating a composite image for a physician to use in planning radiation therapy to remove a tumor. These libraries are also used to build image models that inco ⁇ orate reference image and target image data. Images in these libraries are known in the art as “atlases” or “atlas images.” Atlases are used, for example, to compare image data in the target image with corresponding image data in an atlas. During registration, a correlation or mapping is computed of corresponding image data elements among images. Many computational techniques for automatically comparing atlas and target images fail because of anomalies in the target image that distort the target image to such a degree that correlations between the atlas and target images are difficult to determine.
  • the target image is transformed in accordance with, for example, the method of Fig. 5, described in greater detail above, to remove the appearance of the anomaly in the image prior to relating the target image to a corresponding atlas image.
  • the target image and the atlas image are compared at points where the target image has not been distorted by an anomaly, compared after filtering target image data elements that have been distorted by an anomaly.
  • mass-effect tumors appear as discrete regions of dark pixels in an image. See, for example, tumor region 410 in Fig. 4 is consistent with a mass- effect tumor.
  • An embodiment consistent with the present invention maps an atlas image to a target image containing representations of discrete anomalies by mapping the atlas image data to the target image at locations corresponding to anatomy that has not been affected by the tumor. Using this technique, the mapping process will not be skewed or confused by regions in the target image that have been corrupted by the presence of the anomaly.
  • I(x) a pathological anatomical image referred to as I(x), where x is a data element in the image in a region of interest ⁇ .
  • the image contains a mass-effect tumor occupying a sub-region rin region ⁇ .
  • a reference library contains at least one corresponding normal anatomic atlas image T(x) that does not contain a tumor.
  • the objective is to generate a composite image that combines the atlas T(x) and the target image I(x) by mapping corresponding data in both images.
  • the method begins with the step of identifying the sub-region in the image representing a tumor (Fig. 7, step 702). This step can be accomplished in a number of ways. An operator can outline the sub-region of the image displayed on a screen using a pointing device. Automated methods can also be used to identify the tumor sub-region. These automated methods include image edge detection algorithms known in the art and region growing techniques such as the procedure described in greater detail above.
  • a transform h,(x) is computed that shrinks the tumor sub-region of the target image to a single data element.
  • the transform preferably restores the image to a state where structures have a normal appearance while reducing the size of the tumor.
  • the computed transform h,(x) is then applied to the target image to generate a normal image (step 704).
  • the foregoing steps are preferably executed in accordance with the procedure described in detail above for removing an object embedded in a deformable substance.
  • a second transform h 2 (x) is computed for mapping the corresponding atlas image and the generated normal image (step 706).
  • the second transform used for mapping is preferably generated using image registration techniques such as, for example, the techniques described in U.S. Patent Application Nos. 08/678,628 and 09/186,359, inco ⁇ orated by reference herein in their entirety.
  • a third transform h 3 (x) is generated from the first and second transforms (step 708).
  • the third transform is applied to map the atlas image to the original target image, the image that contains a tumor region (step 710). According to this approach, a mapping between the atlas image and the target image containing a tumor is generated in a way that reduces the potential errors that are often introduced when transformations are generated using image data representing the tumor.
  • Some anomalies that appear in images are infused with the image and are not easy to isolate, shrink, and reverse displacement caused by the anomaly.
  • One type of such anomalies are infiltrating tumors. Examples of infiltrating tumors include tumors that change the tissue characteristics but do not displace the normal tissue.
  • a method consistent with the present invention filters the tumor region when computing a map relating the atlas and the target.
  • An embodiment of such a filter consistent with the present invention excludes image data elements corresponding to the tumor region.
  • One skilled in the art will recognize that there are other suitable filters with varying weights that can be applied to image data elements.
  • x is a data element in the image in a region of interest ⁇ .
  • the image contains an infiltrating tumor occupying a sub-region rin region ⁇ .
  • the distance function used in atlas mapping methods such as, for example, those disclosed in U.S. Patent Application Serial Nos. 08/678,628 and 09/186,359, is modified to accommodate this pathology.
  • U.S. Patent Application Serial Nos. 08/678,628 and 09/186,359 disclose an image mapping process that uses a distance measure D, defining the disparity between a template image transformed to target image I(x), T(h(x)), and image I(x). An equation for expressing this disparity is:
  • ⁇ is a region of interest in the target image I(x)
  • h(x) is a transform for transforming data elements in target image I(x) to corresponding data elements in atlas image T(x).
  • This form of disparity measure is effective at mapping normal target images, i.e., images without anomalies, to atlas images.
  • Pathological anatomical target images e.g., target images containing an infiltrating tumor
  • the above disparity equation is preferably modified so that the measure of disparity excludes the sub- region ⁇ when comparing the target image and the transformed atlas image.
  • the preferred disparity measure is:
  • F is a function that inco ⁇ orates a mathematical model that maps data element values in the sub-region containing an infiltrating tumor corresponding to the data element values in a normal image.
  • the function F will exploit any information available about the relationship between data elements in the tumor region and what value those elements would have if the image were normal. Information such as the mean intensity value and the variance of the intensity values can be used to generate the function F, assuming for example, image element intensity values adhere to a Gaussian model.
  • CT computed tomography
  • MRI magnetic resonance imaging
  • this invention can also be used on images acquired from other imaging modalities.
  • application of the present invention is not limited to anatomical images.
  • This invention also applies to non-anatomical images, including, but not limited to, satellite imagery, photographs, radar images, and images acquired from multiple sources.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Image Processing (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)

Abstract

An apparatus and method for processing images with regions representing target objects involves registering a first and a second image, where the second image contains a target object. The method comprises the steps of identifying a region in the second image containing one or more image data elements representing the target object, transforming the second image to reduce the size of the region containing the target object, and registering the first image and the second image. Another embodiment comprises the steps of identifying a region in the second image containing one or more image data elements representing a target object. After identifying this region, the first and second images are registered using a first transform component that filters one or more data elements in the second image that are contained in the identified region, and a second transform component that includes one or more data elements in the second image that are contained in the identified region.

Description

METHOD AND APPARATUS FOR PROCESSING IMAGES WITH REGIONS REPRESENTING
TARGET OBJECTS
RELATED APPLICATION
This patent application claims priority to U.S. provisional patent application 60/114,716 filed December 16, 1998, which is herein incorporated by reference in its entirety.
BACKGROUND OF THE INVENTION
The present invention relates to image processing systems and methods, and more particularly to image processing systems that process images with regions representing target objects.
Image registration is an image processing technique for combining two or more images, or selected points from the images, to produce a composite image containing data from each of the registered images. During registration, a transformation is computed that maps related points among the combined images so that points defining the corresponding structures in each of the combined images are correlated in the composite image.
Image processing techniques suitable for image registration follow two different approaches. The first requires that an individual with expertise in the structure of the object represented in the images label a set of landmarks in each of the images that are to be registered. For example, when registering two MRI images of different axial slices of a human head, a physician may label points, or a contour adjacent to these points, corresponding to the cerebellum in two images. The two images are then registered by relying on a known relationship among the landmarks in the two brain images. The second currently-practiced technique for image registration uses the mathematics of small deformation multi-target registration and is image data driven. Here, volume based imagery is generated of the two targets from which a coordinate system transformation is constructed. Conventional image processing techniques may not adequately take into account the existence of anomalies in an image that can limit performance when registering target images with corresponding atlas images that do not contain representation of anomalies. There is, therefore, a need for an image processing system that overcomes the limitations of the conventional techniques.
SUMMARY OF THE INVENTION
The present invention provides a methodology for image processing that can compensate for the presence of target objects in images.
Additional features and advantages of the invention will be set forth in the description which follows, and in part, will be apparent from the description, or may be learned by practicing the invention. The objectives and other advantages of the invention will be realized and obtained by the method and apparatus particularly pointed out in the written description and the claims hereof as well as in the appended drawings.
To achieve these and other advantages and in accordance with the purpose of the invention, as embodied and broadly described, a method and apparatus according to the invention registers a first and a second image, where the second image contains a target object. The method comprises the steps of identifying a region in the second image containing one or more image data elements representing the target object, transforming the second image to reduce the size of the region containing the target object, and registering the first image and the second image.
Another embodiment consistent with the present invention registers a first image and a second image. A method comprises the steps of identifying a region in the second image containing one or more image data elements representing a target object. After identifying this region, the first and second images are registered using a first transform component that
filters one or more data elements in the second image that are contained in the identified region, and a second transform component that includes one or more data elements in the second image that are contained in the identified region.
Both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.
DESCRIPTION OF THE FIGURES
The accompanying drawings, which are incorporated in and constitute part of the specification, illustrate a presently preferred embodiment of the invention and together with the general description given above and detailed description of the preferred embodiment given below, serve to explain the principles of the invention.
Fig. 1 is a flow diagram of a method for growing a seed region in accordance with the present invention; Fig. 2 is schematic diagram illustrating an apparatus for processing images in accordance with the present invention;
Fig. 3 is a computer display of coronal, sagittal, and axial views of a head image containing a tumor with a defined seed region;
Fig. 4 is a computer display of coronal, sagittal, and axial views of a head image containing a tumor with an identified tumor region;
Fig. 5 is a flow diagram of an image processing method for automatically removing image data elements corresponding to an object embedded in an image in accordance with the present invention;
Fig. 6 is a computer display of coronal, sagittal, and axial views of a head image after a tumor has been removed from the image using image processing in accordance with the present invention; and
Fig. 7 is a flow diagram of a method for registering a target image and an atlas image in accordance with the present invention.
DETAILED DESCRIPTION OF THE INVENTION
A method and system is disclosed which locates objects in image data. The
accompanying drawings, which are incoφorated in and constimte part of the specification, illustrate a presently preferred embodiment of the invention and together with the general description given above and detailed description of the preferred embodiment given below, serve to explain the principles of the invention. In many applications requiring image analysis, a technician must visually inspect several images and engage in a labor intensive process of manually labeling regions of interest in the images. The term "region" as used herein includes one or more image data elements. An image data element includes, for example, a pixel or a voxel. A method consistent with the present invention helps an operator to locate regions in an image representing objects such as structural anomalies. The method begins with an image containing one or more target objects of interest. For example, consider an image of a human brain acquired using a magnetic resonance imager (MRI). A physician or technician can use the method to determine the extent of a mass effect tumor present in the image. A mass effect tumor is a tumor that displaces the tissue of surrounding structures. Thus, this type of tumor is an example of the type of object that can be localized in an image for subsequent analysis.
The first step of the method shown in Fig. 1 identifies a seed region in the image containing at least one data element representing diseased tissue located in the tumor (step 100). The seed region can be identified in any of a number of ways. For example, a computer system can identify a data element in the MRI that represents the tumor. One suitable technique would use histogram or thresholding algorithms known in the art to identify a candidate seed region representing a portion of the tumor. Alternatively, an operator can identify the seed region by selecting a data element or group of data elements in the image representing a portion of the tumor. A mathematical texture or intensity model (or a model that embodies both texture and intensity) is then defined for data elements representing diseased tissue in the tumor and normal tissue in the image (steps 102 and 104, respectively). Texture and intensity models define the distribution of the gray scale values of data elements in an image. Moreover, texture can be modeled as a structured variation of intensity. There are a number of possible texture and intensity models known in the art that are suitable for the present method, including, but not limited to Gaussian and mixtures of Gaussian distributions. Gaussian- based distributions, like other types of parametric statistical distributions, are appropriate models to use because the models can be adapted to the image by modifying only a few parameters, such as the mean and variance of the distribution. The models can be selected by software that inspects the tumor region and normal region of the image and selects appropriate models by, for example, estimating a mean and variance of the data elements in the regions of interest. Alternatively, an operator can specify appropriate models.
After the appropriate texture or intensity models are defined, the seed region is grown to increase the number of data elements representing the tumor contained in the seed region, while decreasing the number of data elements representing normal tissue contained in the seed region (step 106). In an embodiment consistent with the present invention, the method seeks to maximize the number of data elements representing the tumor contained in the seed region, while minimizing the number of data elements representing normal tissue contained in the seed region. The result of the operation of this method is to define a boundary in the image that encloses the tumor, but does not include significant amounts of normal tissue not affected by the tumor.
Fig. 2 is an apparatus for implementing embodiments consistent with the present invention. Each of the described techniques involves computations that are preferably executed in hardware and or software stored in a computer system. The images that are processed are acquired from imaging devices during processing and/or are retrieved from storage devices. For example, medical imaging scanner 214 obtains images and stores them on a computer memory 206 which is connected to a computer central processing unit (CPU) 204. One of ordinary skill in the art will recognize that a parallel computer platform having multiple CPUs is also a suitable hardware platform for the present invention, including, but not limited to, parallel machines and workstations with multiple processors. Computer memory 206 can be directly connected to CPU 204, or this memory can be remotely connected through a communications network. Moreover, the computational operations described herein can be allocated among several devices in a distributed network, including, but not limited to, networks linked by the Internet. The computer system of Fig. 2 also can include user interfaces for operator input. The interfaces include keyboard 212 and mouse 208, which controls cursor 210. Display 202 has windows 216 and 218.
An embodiment of a process consistent with the present invention for growing a surface from a seed sphere to define the bounding surface of the tumor estimates a closed surface M which reduces an elastic regularization energy while increasing the data log- likelihood associated with the texture characteristics of the diseases and normal anatomy. Consider, for example, a target anatomical imagery I(x) in a region of interest Ω. Begin by first defining a texture model for the interior and the exterior of diseased region of interest. An example of such a model is an independent Gaussian texture model with different mean and variances for the interior and the exterior. Let μint, oint be the mean and variance of the Gaussian texture of the tumor. Let \iExt, oE 2 lζtbe the mean and variance of the Gaussian texture of the exterior or the normal anatomy surrounding the tumor. This defines the data likelihood model. Let M be a ciosed triangulated surface with interior Mml. Then the energy defined by the texture model and the surface M becomes:
( I(x) -μ ) 2 ( I (x) -μ ) 2 L (M) = J XM. _. _ 2 ιnt dx + J «icnt Ext dx
2σ mt 2σ' Ext
A regularization energy is selected for the prior that constrains the estimate of the surface to be smooth. For defining the regularization energy the surface lattice is preferably modeled as an elastic sheet. The energy is then locally given by the distance between the sites in the lattice.
Let {x, eMi = 1, ..., M} be the vertices of a triangulated graph. Preferably the vertices {x, eMi = 1, ..., M\ of the triangulation minimize an energy which is based on the distance of the vertex to it's neighbors. Define the neighborhood N(x,) as the set of all neighbors of vertex x, in the triangulated graph. Let n„ be the outward normal at each vertex. The elastic energy associated with the surface Mis defined as
Figure imgf000011_0001
The estimate of the bounding surface M is then given by
M=argm±nL(M) + αi (N) , M
where α is the Language multiplier. A preferred iterative gradient algorithm for generating a surface minimizing combined energy is:
Initialize (M) to a seed sphere in the tumor.; Input (uιnt,σ2 ιπt,μeχt,σ2 ext) ; for i :=1 to M do
Compute (N(x,)); end
Initialize^,); Set step-size e; while (supJ-JJ) > threshold ) do for i :=1 to M do
Compute (nxι);
C' := 2 ∑ 7eW<xJ, < ;- ;> ;
Figure imgf000011_0002
end; for i := 1 to M do
Figure imgf000011_0003
end; end;
As previously discussed, this procedure is suited for image processing associated with image registration. Fig.3 illustrates an application of this method to medical images. Fig.3 includes four windows 300, 302, 304, and 306. Windows 300, 304, and 306 contain sagittal, axial, and coronal views of a three-dimensional MRI of a patient's brain containing mass effect tumor 308. An operator draws an initial seed region 310 in any one of the views, using for example a computer pointing device to select one or more image data elements that correspond to the tumor. Alternatively, software can determine the boimdary of a seed region, by for example, selecting a group of image data elements falling in a selected bin of a histogram for the image. The location of this seed region is also shown in each of the other views. Window 302 is a wireframe representation of the selected seed region in its initial condition prior to growing it to encompass the tumor. As mentioned above, the seed region can be, for example, a single point, a two-dimensional area, or a three-dimensional volume.
Fig. 4 shows the windows of Fig. 3 after growing the seed region to encompass tumor B08. As seen in Fig. 4, what was an initial region 310, has grown to region 410. In accordance with the method, region 410 was grown to maximize the volume of the tumor contained in the region, while avoiding extending 410 to encroach on normal tissue in the image. In this example, all of the tumor tissue has not been completely contained in region 410, however the results can be improved by allowing additional iterations of the method and/or refining the definitions of the intensity or texture models used for the tumor or normal tissue. Fig. 4 shows coronal, sagittal, and axial views of an image where the tumor image boimdary 412 has been automatically identified using the method of Fig. 1. Window 402 is a wireframe of the three-dimensional region after growing it to cover the tumor. Accordingly, by applying this method, the region of the image occupied by the tumor is automatically located and segmented from the remainder of the image.
Fig. 5 is a flow diagram of an image processing method for automatically removing an object embedded in a deformable substance. Here again, the method is applicable to processing many types of images including, but not limited to, medical images. This method is especially suited for removing tumors from medical images because as certain tumors grow in the brain, the tumors deform the structures in the brain. The method begins with an image with an identified object boundary separating the object from the substance into which the object is embedded. The object boundary can be automatically identified using, for example, the method of Fig. 1. In another embodiment consistent with the present invention, the region in the image representing the object to be removed can be identified by an operator using, for example, a mouse or other pointing device to trace the object boimdary on a digital representation of the image (step 500).
At step 502, a transform is automatically computed such that when the transform is applied to the image the tumor region shrinks, preferably to a single point. The transform that is computed not only shrinks the tumor region, but also predicts the shape of the normal anatomy after removal of the tumor. At step 504 the computed transform is applied to the image with the tumor so that the size of the region of interest is reduced and the normal shape of the image without the object in the region of interest is restored. In other words, the method reverses the deformation caused by the introduction of the object in the region of interest. Accordingly, reducing the size of the image area occupied by the tumor (or reducing the proportionate size of the tumor region by enlarging or emphasizing other areas of the image), can reduce the amount of data that can skew the registration process.
Fig. 6 shows coronal (606), sagittal (600), and axial (604) views of the image after the operation of step 504 described above restores the image to remove the tumor. Note in the restored image, two objectives have been accomplished. First, the tumor has been removed from the image. 612 corresponds to the location of the tumor boundary prior to removal. Second, the representation of the anatomical structures have been restored to their normal characteristics, i.e., pre-tumor shape and density. Compare, for example, ventricle region 414 in the axial image in Fig. 4 with the corresponding representation of the ventricle region 614 in image 606 where the tumor has been electronically removed by applying the computed transform. Fig. 6 shows that the result of the image processing described herein restores the representation in the image of the ventricle region and surrounding tissue to their normal appearance.
In many image processing applications, libraries containing reference images are used to restore, modify, or classify target images. These applications include generating a composite image for a physician to use in planning radiation therapy to remove a tumor. These libraries are also used to build image models that incoφorate reference image and target image data. Images in these libraries are known in the art as "atlases" or "atlas images." Atlases are used, for example, to compare image data in the target image with corresponding image data in an atlas. During registration, a correlation or mapping is computed of corresponding image data elements among images. Many computational techniques for automatically comparing atlas and target images fail because of anomalies in the target image that distort the target image to such a degree that correlations between the atlas and target images are difficult to determine.
To overcome these limitations, two approaches consistent with the present invention compensate for anomalies in the target imagery to improve the ability to exploit atlas images. In the first approach, the target image is transformed in accordance with, for example, the method of Fig. 5, described in greater detail above, to remove the appearance of the anomaly in the image prior to relating the target image to a corresponding atlas image. In the second approach, the target image and the atlas image are compared at points where the target image has not been distorted by an anomaly, compared after filtering target image data elements that have been distorted by an anomaly. These two techniques will be described in greater detail below.
In some imaging modalities, mass-effect tumors appear as discrete regions of dark pixels in an image. See, for example, tumor region 410 in Fig. 4 is consistent with a mass- effect tumor. An embodiment consistent with the present invention maps an atlas image to a target image containing representations of discrete anomalies by mapping the atlas image data to the target image at locations corresponding to anatomy that has not been affected by the tumor. Using this technique, the mapping process will not be skewed or confused by regions in the target image that have been corrupted by the presence of the anomaly.
Consider a pathological anatomical image referred to as I(x), where x is a data element in the image in a region of interest Ω . The image contains a mass-effect tumor occupying a sub-region rin region Ω . A reference library contains at least one corresponding normal anatomic atlas image T(x) that does not contain a tumor. The objective is to generate a composite image that combines the atlas T(x) and the target image I(x) by mapping corresponding data in both images.
The method begins with the step of identifying the sub-region in the image representing a tumor (Fig. 7, step 702). This step can be accomplished in a number of ways. An operator can outline the sub-region of the image displayed on a screen using a pointing device. Automated methods can also be used to identify the tumor sub-region. These automated methods include image edge detection algorithms known in the art and region growing techniques such as the procedure described in greater detail above.
Once the tumor sub-region is identified, a transform h,(x) is computed that shrinks the tumor sub-region of the target image to a single data element. The transform preferably restores the image to a state where structures have a normal appearance while reducing the size of the tumor. The computed transform h,(x) is then applied to the target image to generate a normal image (step 704). The foregoing steps are preferably executed in accordance with the procedure described in detail above for removing an object embedded in a deformable substance.
Once a normal appearing image has been generated, a second transform h2(x) is computed for mapping the corresponding atlas image and the generated normal image (step 706). The second transform used for mapping is preferably generated using image registration techniques such as, for example, the techniques described in U.S. Patent Application Nos. 08/678,628 and 09/186,359, incoφorated by reference herein in their entirety.
Next, a third transform h3(x) is generated from the first and second transforms (step 708). The third transform is preferably composed from the inverse of the first transform and the second transform; i.e., h3(x) = hfx)'1 ° Λ2(x). The third transform is applied to map the atlas image to the original target image, the image that contains a tumor region (step 710). According to this approach, a mapping between the atlas image and the target image containing a tumor is generated in a way that reduces the potential errors that are often introduced when transformations are generated using image data representing the tumor.
Some anomalies that appear in images are infused with the image and are not easy to isolate, shrink, and reverse displacement caused by the anomaly. One type of such anomalies are infiltrating tumors. Examples of infiltrating tumors include tumors that change the tissue characteristics but do not displace the normal tissue. Rather than trying to shrink a region in an fmage corresponding to an infiltrating tumor prior to mapping the image with an atlas image, a method consistent with the present invention filters the tumor region when computing a map relating the atlas and the target. An embodiment of such a filter consistent with the present invention excludes image data elements corresponding to the tumor region. One skilled in the art will recognize that there are other suitable filters with varying weights that can be applied to image data elements.
Consider the pathological anatomical image I(x). Here again, x is a data element in the image in a region of interest Ω. The image contains an infiltrating tumor occupying a sub-region rin region Ω. In this process the distance function used in atlas mapping methods such as, for example, those disclosed in U.S. Patent Application Serial Nos. 08/678,628 and 09/186,359, is modified to accommodate this pathology. U.S. Patent Application Serial Nos. 08/678,628 and 09/186,359 disclose an image mapping process that uses a distance measure D, defining the disparity between a template image transformed to target image I(x), T(h(x)), and image I(x). An equation for expressing this disparity is:
[D(I(x), T(h(x)))dx,
wherein Ω is a region of interest in the target image I(x), h(x) is a transform for transforming data elements in target image I(x) to corresponding data elements in atlas image T(x). This form of disparity measure is effective at mapping normal target images, i.e., images without anomalies, to atlas images.
Pathological anatomical target images, e.g., target images containing an infiltrating tumor, have had the normal tissue characteristics changed in the region of the infiltrating tumor in a sub-region τ in region Ω . To accommodate these kinds of anomalies, the above disparity equation is preferably modified so that the measure of disparity excludes the sub- region τ when comparing the target image and the transformed atlas image. Thus, the preferred disparity measure is:
'D(I(x), T(h(x)))dx,
•O/τ Note that the range of integration in this disparity measure is computed over the region of the anatomy that has not changed.
If information is available to generate a mathematical model for predicting how the anomaly in the target image changes data element values, a further modification of the disparity measure is preferably used:
D(I(x), 7Th(x)))dx + (D(F(I(x)), T(h(x)))dx,
O/τ
where F is a function that incoφorates a mathematical model that maps data element values in the sub-region containing an infiltrating tumor corresponding to the data element values in a normal image. The function F will exploit any information available about the relationship between data elements in the tumor region and what value those elements would have if the image were normal. Information such as the mean intensity value and the variance of the intensity values can be used to generate the function F, assuming for example, image element intensity values adhere to a Gaussian model.
To represent shape abnormalities assume that the anatomy is enclosed in a bounded rectangular background space Ω c R3 and preferably use the variability operator
L = Δ + a; where a is a constant having a value less than zero, and with Dirichlet boundary conditions on 8Ω. Making a negative ensures non-singularity and existence of a solution to the problem below. For simplicity, assume the three components of the deformation field u(x); x e Ω are independent; for the coupled case replace L by the Navier operator. Let c, the location of an image data element in the region of interest Ω, be the center of an expanding abnormality that expands into a sub-region τ in Ω. For example, τ could be a ball with the center at c. More general shapes are also suitable for this approach. The "normal" remainder will be denoted N= Ω / r. The observed image is represented as follows: nx) = I,cπψ[h(x)] = Ilemp[x + u(x)] so that we are led to the following field equation for the pathology induced displacement
(Lu)(x) = (Au)(x) + au(x) = 0; x e N with the boundary condition u(x) = uJX) = c - u; x dv u(x) = 0; x e dΩ Note that «(•) preferably should not be defined in the abnormal region τ. One skilled in the art will recognize that this is in the form of a Freedholm equation that can be computed directly using, for example, an over-relaxation scheme as described in the text Numerical Recipes in C. section 17.5.
While the disclosed system and method is useful for medical imaging systems used
for noninvasive exploration of human anatomy, for example, computed tomography (CT) and magnetic resonance imaging (MRI), this invention can also be used on images acquired from other imaging modalities. Furthermore, application of the present invention is not limited to anatomical images. This invention also applies to non-anatomical images, including, but not limited to, satellite imagery, photographs, radar images, and images acquired from multiple sources. It will be apparent to those skilled in the art that various modifications and variations can be made to the disclosed embodiments of the present invention without departing from the spirit or scope of the invention. Thus it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.

Claims

20We claim:
1. A method for identifying a target object in an image, comprising the steps of: identifying a seed region containing one or more image data elements representing the target object; defining a model of image data elements representing the target object; defining a model of image data elements representing structure in the image other than the target object; and growing the seed region to increase the number of image data elements representing the target object contained in the seed region while reducing the number of image data elements representing structure other that the target object contained in the seed region.
2. The method of claim 1 , wherein the step of defining a model of image data elements representing the target object further comprises the substep of: defining a texture or intensity model.
3. The method of claim 1 , wherein the step of defining a model of image data elements representing structure in the image other than the target object further comprises the substep of: defining a texture or intensity model.
4. The method of claim 1 , wherein the step of growing the seed region further comprises the substep of: maximizing the number of image data elements representing the target object contained in the seed region while minimizing the number of image data elements representing structure other that the target object contained in the seed region.
5. The method of claim 1, wherein the step of growing the seed region further comprises the substep of: growing the seed region elastically.
6. A method for removing a target object from an image, comprising the steps of: identifying a region in the image containing one or more image data elements representing the target object; transforming the image so that the identified region has been reduced in size; and restoring the representation of structure in the image to a shape that corresponds to removal of the target object.
7. The method of claim 6, wherein the step of transforming the image so that the identified region has been reduced in size further comprises the substep of: reducing the proportionate size of the identified region in relation to the image.
8. A method for registering a first and a second image, wherein the second image contains a target object, comprising the steps of: identifying a region in the second image containing one or more image data elements representing the target object; transforming the second image to reduce the size of the identified region; and registering the first image and the second image.
9. The method of claim 8, wherein the step of transforming the second image to reduce the size of the identified region further comprises the substep of: reducing the proportionate size of the identified region in relation to the second image.
10. A method for registering a first and a second image, wherein the second image contains a target object, comprising the steps of: identifying a region in the second image containing one or more image data elements representing the target object; and registering the first image and the second image using a transform that filters data elements in the second image that are contained in the identified region.
11. The method of claim 10, wherein the step of registering the first image and the second image using a transform that filters data elements in the second image that are contained in the identified region further comprises the substep of: excluding data elements in the second image that are contained in the identified region.
12. A method for registering a first and a second image, wherein the second image contains a target object, comprising the steps of: identifying a region in the second image containing one or more image data elements representing the target object; and registering the first image and the second image using a first transform component that filters one or more data elements in the second image that are contained in the identified region, and a second transform component that includes one or more data elements in the second image that are contained in the identified region.
13. The method of claim 12, wherein the step of registering the first image and the second image further comprises the substep of: excluding one or more data elements in the second image that are contained in the identified region.
14. A method for registering a first and a second image, wherein the second image contains a target object, comprising the steps of: identifying a seed region in the second image containing one or more image data elements representing the target object; defining a model of image data elements representing the target object; defining a model of image data elements representing structure in the second image other than the target object; growing the seed region in the second image to increase the number of image data elements representing the target object contained in the seed region while reducing the number of image data elements representing structure other that the target object contained in the seed region; transforming the second image to reduce the size of the seed region; and registering the first image and the second image.
15. A method for identifying a tumor in an image, comprising the steps of: identifying a seed region containing one or more image data elements representing the tumor; defining a model of image data elements representing the tumor; defining a model of image data elements representing structure in the image other than the tumor; and growing the seed region to increase the number of image data elements representing the tumor contained in the seed region while reducing the number of image data elements representing structure other that the tumor contained in the seed region.
16. A method for removing a tumor from an image, comprising the steps of: identifying a region in the image containing one or more image data elements representing the tumor; transforming the image so that the identified region has been reduced in size; and restoring the representation of structure in the image to a shape that corresponds to removal of the tumor.
17. A method for registering a first and a second image, wherein the second image contains a tumor, comprising the steps of: identifying a region in the second image containing one or more image data elements representing the tumor; transforming the second image to reduce the size of the identified region; and registering the first image and the second image.
18. A method for registering a first and a second image, wherein the second image contains a tumor, comprising the steps of: identifying a region in the second image containing one or more image data elements representing the tumor; and registering the first image and the second image using a transform that filters data elements in the second image that are contained in the identified region.
19. A method for registering a first and a second image, wherein the second image contains a tumor, comprising the steps of: identifying a region in the second image containing one or more image data elements representing the tumor; and registering the first image and the second image using a first transform component that filters one or more data elements in the second image that are contained in the identified region, and a second transform component that includes one or more data elements in the second image that are contained in the identified region.
20. A method for registering a first and a second image, wherein the second image contains a tumor, comprising the steps of: identifying a seed region in the second image containing one or more image data elements representing the tumor; defining a model of image data elements representing the tumor; defining a model of image data elements representing structure in the second image other than the tumor; growing the seed region in the second image to increase the number of image data elements representing the tumor contained in the seed region while reducing the number of image data elements representing structure other that the tumor contained in the seed region; transforming the second image to reduce the size of the seed region; and registering the first image and the second image.
21. A method for generating a radiation treatment plan using a first image and a second image comprising the steps of: identifying a seed region in said second image containing one or more image data elements representing a target object; and registering the first image and the second image using a transform that filters one or more data elements in the second image that are contained in said identified region to produce a composite image for radiation treatment.
22. A method for generating a radiation treatment plan using a first image and a second image comprising the steps of: identifying a seed region in said second image containing one or more image data elements representing a target object; growing the seed region to increase the number of image data elements representing the target object contained in said seed region while reducing the number of image data elements representing structure other that the target object contained in said seed region; transforming the second image to reduce the size of the region containing the target object; and registering the first image and the second image to produce a composite image for radiation treatment.
23. An apparatus for identifying a tumor in an image comprising: means for identifying a seed region containing one or more image data elements representing the tumor; means for defining a model of image data elements representing the tumor; means for defining a model of image data elements representing structure in the image other than the tumor; and means for growing the seed region to increase the number of image data elements representing the tumor contained in the seed region while reducing the number of image data elements representing structure other that the tumor contained in the seed region.
24. An apparatus for removing a tumor from an image comprising: means for identifying a region in the image containing one or more image data elements representing the tumor; means for transforming the image so that the identified region has been reduced in size; and means for restoring the representation of structure in the image to a shape that corresponds to removal of the tumor.
25. An apparatus for registering a first and a second image, wherein the second image contains a tumor, comprising: means for identifying a region in the second image containing one or more image data elements representing the tumor, means for transforming the second image to reduce the size of the identified region; and means for registering the first image and the second image.
! 26. An apparatus for registering a first and a second image, wherein the second image contains a tumor, comprising: means for identifying a region in the second image containing one or more image data elements representing the tumor; and means for registering the first image and the second image using a transform that filters data elements in the second image that are contained in the identified region.
27. An apparatus for registering a first and a second image, wherein the second image contains a tumor, comprising: means for identifying a region in the second image containing one or more image data elements representing the tumor; and means for registering the first image and the second image using a first transform component that filters one or more data elements in the second image that are contained in the identified region, and a second transform component that includes one or more data elements in the second image that are contained in the identified region.
28. An apparatus for registering a first and a second image, wherein the second image contains a tumor, comprising: means for identifying a seed region in the second image containing one or more image data elements representing the tumor; means for defining a model of image data elements representing the tumor; means for defining a model of image data elements representing structure in the second image other than the tumor; means for growing the seed region in the second image to increase the number of image data elements representing the tumor contained in the seed region while reducing the number of image data elements representing structure other that the tumor contained in the seed region; means for transforming the second image to reduce the size of the seed region; and means for registering the first image and the second image.
29. An apparatus for generating a radiation treatment plan using a first image and a second image comprising: means for identifying a seed region in said second image containing one or more image data elements representing a target object; and means for registering the first image and the second image using a transform that filters one or more data elements in the second image that are contained in said identified region to produce a composite image for radiation treatment.
30. An apparatus for generating a radiation treatment plan using a first image and a second image comprising: means for identifying a seed region in said second image containing one or more image data elements representing a target object; means for growing the seed region to increase the number of image data elements representing the target object contained in said seed region while reducing the number of image data elements representing structure other that the target object contained in said seed region; means for transforming the second image to reduce the size of the region containing the target object; and means for registering the first image and the second image to produce a composite image for radiation treatment.
31. An article of manufacture for identifying a tumor in an image comprising:
a module configured to cause a computer to identify a seed region containing one or more image data elements representing the tumor; a module configured to cause the computer to define a model of image data elements representing the tumor; a module configured to cause the computer to define a model of image data elements representing structure in the image other than the tumor; and a module configured to cause the computer to grow the seed region to increase the number of image data elements representing the tumor contained in the seed region while reducing the number of image data elements representing structure other that the tumor contained in the seed region.
I 32. An article of manufacture for removing a tumor from an image, comprising: a module configured to cause a computer to identify a region in the image containing one or more image data elements representing the tumor; a module configured to cause the computer to transform the image so that the identified region has been reduced in size; and a module configured to cause the computer to restore the representation of structure in the image to a shape that corresponds to removal of the tumor.
33. An article of manufacture for registering a first and a second image, wherein the second image contains a tumor, comprising: a module configured to cause a computer to identify a region in the second image containing one or more image data elements representing the tumor; a module configured to cause the computer to transform the second image to reduce the size of the identified region; and a module configured to cause the computer to register the first image and the second image.
34. An article of manufacture for registering a first and a second image, wherein the second image contains a tumor, comprising: a module configured to cause a computer to identify a region in the second image containing one or more image data elements representing the tumor; and a module configured to cause the computer to register the first image and the second image using a transform that filters data elements in the second image that are contained in the identified region.
35. An article of manufacture for registering a first and a second image, wherein the second image contains a tumor, comprising: a module configured to cause a computer to identify a region in the second image containing one or more image data elements representing the tumor; and a module configured to cause the computer to register the first image and the second image using a first transform component that filters one or more data elements in the second image that are contained in the identified region, and a second transform component that includes one or more data elements in the second image that are contained in the identified region.
36. An article of manufacture for registering a first and a second image, wherein the second image contains a tumor, comprising: a module configured to cause a computer to identify a seed region in the second image containing one or more image data elements representing the tumor; a module configured to cause the computer to define a model of image data elements representing the tumor; a module configured to cause the computer to define a model of image data elements representing structure in the second image other than the tumor; a module configured to cause the computer to grow the seed region in the second image to increase the number of image data elements representing the tumor contained in the seed region while reducing the number of image data elements representing structure other that the tumor contained in the seed region; a module configured to cause the computer to transform the second image to reduce the size of the seed region; and a module configured to cause the computer to register the first image and the second image.
37. An article of manufacture for generating a radiation treatment plan using a first image and a second image comprising: a module configured to cause a computer to identify a seed region in said second image containing one or more image data elements representing a target object; and a module configured to cause the computer to register the first image and the second image using a transform that filters one or more data elements in the second image that are contained in said identified region to produce a composite image for radiation treatment.
38. An article of manufacture for generating a radiation treatment plan using a first image and a second image comprising: a module configured to cause a computer to identify a seed region in said second image containing one or more image data elements representing a target object; a module configured to cause the computer to grow the seed region to increase the number of image data elements representing the target object contained in said seed region while reducing the number of image data elements representing structure other that the target object contained in said seed region; a module configured to cause the computer to transform the second image to reduce the size of the region containing the target object; and a module configured to cause the computer to register the first image and the second image to produce a composite image for radiation treatment.
39. An apparatus for identifying a target object in an image comprising: means for identifying a seed region containing one or more image data elements representing the target object; means for defining a model of image data elements representing the target object; means for defining a model of image data elements representing structure in the image other than the target object; and means for growing the seed region to increase the number of image data elements representing the target object contained in the seed region while reducing the number of image data elements representing structure other that the target object contained in the seed region.
40. The apparatus of claim 39, wherein the means for defining a model of image data elements representing the target object further comprises: means for defining a texture or intensity model.
41. The apparatus of claim 39, wherein the means for defining a model of image data elements representing structure in the image other than the target object further comprises: means for defining a texture or intensity model.
42. The apparatus of claim 39, wherein the means for growing the seed region further comprises: means for maximizing the number of image data elements representing the target object contained in the seed region while minimizing the number of image data elements representing structure other that the target object contained in the seed region.
43. The apparatus of claim 39, wherein the means for growing the seed region further comprises: means for growing the seed region elastically.
44. An apparatus for removing a target object from an image comprising: means for identifying a region in the image containing one or more image data elements representing the target object; means for transforming the image so that the identified region has been reduced in size; and means for restoring the representation of structure in the image to a shape that corresponds to removal of the target object.
45. The apparatus of claim 44, wherein the means for transforming the image so that the identified region has been reduced in size further comprises: means for reducing the proportionate size of the identified region in relation to the image.
46. An apparatus for registering a first and a second image, wherein the second image contains a target object, comprising: means for identifying a region in the second image containing one or more image data elements representing the target object; means for transforming the second image to reduce the size of the identified region; and means for registering the first image and the second image.
47. The apparatus of clam 46, wherein the means for transforming the second image to reduce the size of the identified region further comprises: means for reducing the proportionate size of the identified region in relation to the second image.
48. An apparatus for registering a first and a second image, wherein the second image contains a target object, comprising: means for identifying a region in the second image containing one or more image data elements representing the target object; and means for registering the first image and the second image using a transform that filters data elements in the second image that are contained in the identified region.
49. The apparatus of claim 48, wherein the means for registering the first image and the second image using a transform that filters data elements in the second image that are contained in the identified region further comprises: means for excluding data elements in the second image that are contained in the identified region.
50. An apparatus for registering a first and a second image, wherein the second image contains a target object, comprising: means for identifying a region in the second image containing one or more image data elements representing the target object; and means for registering the first image and the second image using a first transform component that filters one or more data elements in the second image that are contained in the identified region, and a second transform component that includes one or more data elements in the second image that are contained in the identified region.
51. The apparatus of claim 50, wherein the means for registering the first image and the second image further comprises: means for excluding one or more data elements in the second image that are contained in the identified region.
52. An apparatus for registering a first and a second image, wherein the second image contains a target object, comprising: means for identifying a seed region in the second image containing one or more image data elements representing the target object; means for defining a model of image data elements representing the target object; means for defining a model of image data elements representing structure in the second image other than the target object; means for growing the seed region in the second image to increase the number of image data elements representing the target object contained in the seed region while reducing the number of image data elements representing structure other that the target object contained in the seed region; means for transforming the second image to reduce the size of the seed region; and means for registering the first image and the second image.
53. An article of manufacture for identifying a target object in an image comprising:
a module configured to cause a computer to identify a seed region containing one or more image data elements representing the target object; a module configured to cause the computer to define a model of image data elements representing the target object; a module configured to cause the computer to define a model of image data elements representing structure in the image other than the target object; and a module configured to cause the computer to grow the seed region to increase the number of image data elements representing the target object contained in the seed region while reducing the number of image data elements representing structure other that the target object contained in the seed region.
54. The article of manufacture of claim 53, wherein the module configured to cause the computer to define a model of image data elements representing the target object further comprises: a module configured to cause the computer to define a texture or intensity model.
55. The article of manufacture of claim 53, wherein the module configured to cause the
' computer to define a model of image data elements representing structure in the image other than the target object further comprises: a module configured to cause the computer to define a texture or intensity model.
56. The article of manufacture of claim 53, wherein module configured to cause the computer to grow the seed region further comprises: a module configured to cause the computer to maximize the number of image data elements representing the target object contained in the seed region while minimizing the number of image data elements representing strucmre other that the target object contained in the seed region.
57. The article of manufacture of claim 53, wherein the module configured to cause the computer to grow the seed region further comprises: - a module configured to cause the computer to grow the seed region elastically.
58. An article of manufacture for removing a target object from an image, comprising: a module configured to cause a computer to identify a region in the image containing one or more image data elements representing the target object; a module configured to cause the computer to transform the image so that the identified region has been reduced in size; and a module configured to cause the computer to restore the representation of structure in the image to a shape that corresponds to removal of the target object.
59. The article of manufacture of claim 58, wherein the module configured to cause the computer to transform the image so that the identified region has been reduced in size further comprises: a module configured to cause the computer to reduce the proportionate size of the identified region in relation to the image.
60. An article of manufacture for registering a first and a second image, wherein the second image contains a target object, comprising: a module configured to cause a computer to identify a region in the second image containing one or more image data elements representing the target object; a module configured to cause the computer to transform the second image to reduce the size of the identified region; and a module configured to cause the computer to register the first image and the second image.
61. The article of manufacture of claim 60, wherein the module configured to cause the computer to transform the second image to reduce the size of the identified region further comprises: a module configured to cause the computer to reduce the proportionate size of the identified region in relation to the second image.
62. An article of manufacture for registering a first and a second image, wherein the second image contains a target object, comprising: a module configured to cause a computer to identify a region in the second image containing one or more image data elements representing the target object; and a module configured to cause the computer to register the first image and the second image using a transform that filters data elements in the second image that are contained in the identified region.
63. The article of manufacture of claim 62, wherein the module configured to cause the computer to register the first image and the second image using a transform that filters data elements in the second image that are contained in the identified region further comprises: a module configured to cause the computer to exclude data elements in the second image that are contained in the identified region.
64. An article of manufacture for registering a first and a second image, wherein the second image contains a target object, comprising: a module configured to cause a computer to identify a region in the second image containing one or more image data elements representing the target object; and a module configured to cause the computer to register the first image and the second image using a first transform component that filters one or more data elements in the second image that are contained in the identified region, and a second transform component that includes one or more data elements in the second image that are contained in the identified region.
65. The article of manufacture of claim 64, wherein the module configured to cause the computer to register the first image and the second image further comprises: a module configured to cause the computer to exclude one or more data elements in the second image that are contained in the identified region.
66. An article of manufacture for registering a first and a second image, wherein the second image contains a target object, comprising: a module configured to cause a computer to identify a seed region in the second image containing one or more image data elements representing the target object; a module configured to cause the computer to define a model of image data elements representing the target object; a module configured to cause the computer to define a model of image data elements representing structure in the second image other than the target object; a module configured to cause the computer to grow the seed region in the second image to increase the number of image data elements representing the target object contained in the seed region while reducing the number of image data elements representing structure other that the target object contained in the seed region; a module configured to cause the computer to transform the second image to reduce the size of the seed region; and a module configured to cause the computer to register the first image and the second image.
67. A method for comparing a first image and a second image, wherein the second image contains a target object, comprising the steps of: identifying one or more image data elements in the second image corresponding to the target object; developing a map using the identified one or more image data elements that compensates for the target object; and correlating the first image and the second image using the map.
68. The method of claim 67, wherein the target object is a tumor.
PCT/US1999/029798 1998-12-16 1999-12-16 Method and apparatus for processing images with regions representing target objects WO2000036565A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
AU23632/00A AU2363200A (en) 1998-12-16 1999-12-16 Method and apparatus for processing images with regions representing target objects
EP99967332A EP1141894B1 (en) 1998-12-16 1999-12-16 Method and apparatus for processing images with regions representing target objects
DE69943026T DE69943026D1 (en) 1998-12-16 1999-12-16 METHOD AND DEVICE FOR PROCESSING IMAGES WITH REGIONS REPRESENTING TARGET OBJECTS

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US11471698P 1998-12-16 1998-12-16
US60/114,716 1998-12-16

Publications (2)

Publication Number Publication Date
WO2000036565A1 WO2000036565A1 (en) 2000-06-22
WO2000036565A9 true WO2000036565A9 (en) 2001-11-01

Family

ID=22357002

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US1999/029798 WO2000036565A1 (en) 1998-12-16 1999-12-16 Method and apparatus for processing images with regions representing target objects

Country Status (4)

Country Link
EP (1) EP1141894B1 (en)
AU (1) AU2363200A (en)
DE (1) DE69943026D1 (en)
WO (1) WO2000036565A1 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002043003A1 (en) * 2000-11-24 2002-05-30 Kent Ridge Digital Labs Methods and apparatus for processing medical images
SE522437C2 (en) * 2001-06-07 2004-02-10 C Technologies Ab Method and apparatus for extracting information from a target area within a two-dimensional graphic object in an image
US7292037B2 (en) 2004-09-30 2007-11-06 Brainlab Ag Method and device for generating a CT data set
DE502004006200D1 (en) * 2004-09-30 2008-03-27 Brainlab Ag Method and device for generating a CT data record

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1997009690A1 (en) * 1995-09-05 1997-03-13 Northrop Grumman Corporation Data dimensional sieving and fuzzy connectivity for mri image analysis
US6226418B1 (en) * 1997-11-07 2001-05-01 Washington University Rapid convolution based large deformation image matching via landmark and volume imagery
US6009212A (en) * 1996-07-10 1999-12-28 Washington University Method and apparatus for image registration
US5784431A (en) * 1996-10-29 1998-07-21 University Of Pittsburgh Of The Commonwealth System Of Higher Education Apparatus for matching X-ray images with reference images

Also Published As

Publication number Publication date
WO2000036565A1 (en) 2000-06-22
DE69943026D1 (en) 2011-01-20
AU2363200A (en) 2000-07-03
EP1141894A1 (en) 2001-10-10
EP1141894B1 (en) 2010-12-08

Similar Documents

Publication Publication Date Title
US6754374B1 (en) Method and apparatus for processing images with regions representing target objects
Lester et al. A survey of hierarchical non-linear medical image registration
US6909794B2 (en) Automated registration of 3-D medical scans of similar anatomical structures
US9561004B2 (en) Automated 3-D orthopedic assessments
Bankman Handbook of medical image processing and analysis
EP2380132B1 (en) Denoising medical images
US8437521B2 (en) Systems and methods for automatic vertebra edge detection, segmentation and identification in 3D imaging
US6813373B1 (en) Image segmentation of embedded shapes using constrained morphing
US8135189B2 (en) System and method for organ segmentation using surface patch classification in 2D and 3D images
WO2006024974A1 (en) Feature weighted medical object contouring using distance coordinates
Kabus et al. B-spline registration of 3D images with Levenberg-Marquardt optimization
CA2617227C (en) Deformable model for segmenting patient contours versus support structures in medical images
Arezoomand et al. A 3D active model framework for segmentation of proximal femur in MR images
Tang et al. Medical image registration: A review
Liou et al. An approach to three-dimensional image segmentation
Koh et al. Automatic spinal canal detection in lumbar MR images in the sagittal view using dynamic programming
EP1141894B1 (en) Method and apparatus for processing images with regions representing target objects
Loeckx et al. Non-rigid image registration using a statistical spline deformation model
EP1262914A1 (en) Method and apparatus for processing images with regions representing target objects
Leonardi et al. 3D reconstruction from CT-scan volume dataset application to kidney modeling
Sermesant et al. Biomechanical model construction from different modalities: Application to cardiac images
Chica et al. Example-guided segmentation
Khandelwal Spine segmentation in computed tomography images using geometric flows and shape priors
Carlino et al. Geometry description and mesh construction from medical imaging
GaweB et al. Research Article Automatic Spine Tissue Segmentation from MRI Data Based on Cascade of Boosted Classifiers and Active Appearance Model

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ DE DK DM EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
WWE Wipo information: entry into national phase

Ref document number: 1999967332

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 1999967332

Country of ref document: EP

REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

AK Designated states

Kind code of ref document: C2

Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ DE DK DM EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: C2

Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

COP Corrected version of pamphlet

Free format text: PAGES 1/7-7/7, DRAWINGS, REPLACED BY NEW PAGES 1/5-5/5; DUE TO LATE TRANSMITTAL BY THE RECEIVING OFFICE