CA2079094A1 - Three-dimensional graphics simulation and actual imaging data composite display - Google Patents
Three-dimensional graphics simulation and actual imaging data composite displayInfo
- Publication number
- CA2079094A1 CA2079094A1 CA002079094A CA2079094A CA2079094A1 CA 2079094 A1 CA2079094 A1 CA 2079094A1 CA 002079094 A CA002079094 A CA 002079094A CA 2079094 A CA2079094 A CA 2079094A CA 2079094 A1 CA2079094 A1 CA 2079094A1
- Authority
- CA
- Canada
- Prior art keywords
- displaying
- dimensional
- actual
- image data
- original image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000004088 simulation Methods 0.000 title claims abstract description 69
- 238000003384 imaging method Methods 0.000 title claims abstract description 24
- 239000002131 composite material Substances 0.000 title claims abstract description 7
- 238000000034 method Methods 0.000 claims abstract description 53
- 206010028980 Neoplasm Diseases 0.000 claims description 37
- 239000000523 sample Substances 0.000 claims description 30
- 230000005855 radiation Effects 0.000 claims description 29
- 239000007787 solid Substances 0.000 claims description 19
- 239000007943 implant Substances 0.000 claims description 11
- 230000003902 lesion Effects 0.000 claims description 8
- 230000005856 abnormality Effects 0.000 claims description 4
- 206010000269 abscess Diseases 0.000 claims description 4
- 210000004556 brain Anatomy 0.000 description 19
- 238000002591 computed tomography Methods 0.000 description 13
- 238000011282 treatment Methods 0.000 description 9
- 230000001225 therapeutic effect Effects 0.000 description 8
- 238000013507 mapping Methods 0.000 description 6
- 238000002595 magnetic resonance imaging Methods 0.000 description 5
- 238000005481 NMR spectroscopy Methods 0.000 description 4
- 238000001356 surgical procedure Methods 0.000 description 4
- 210000004204 blood vessel Anatomy 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000012877 positron emission topography Methods 0.000 description 3
- 210000003484 anatomy Anatomy 0.000 description 2
- 238000002583 angiography Methods 0.000 description 2
- 210000004289 cerebral ventricle Anatomy 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000002674 ointment Substances 0.000 description 2
- 238000010422 painting Methods 0.000 description 2
- 238000002673 radiosurgery Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 241001085205 Prenanthella exigua Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000001574 biopsy Methods 0.000 description 1
- 238000002725 brachytherapy Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004040 coloring Methods 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000007428 craniotomy Methods 0.000 description 1
- 238000002405 diagnostic procedure Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000002285 radioactive effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000002672 stereotactic surgery Methods 0.000 description 1
- 238000002560 therapeutic procedure Methods 0.000 description 1
- 238000003325 tomography Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000011277 treatment modality Methods 0.000 description 1
- -1 wire frame Substances 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
- A61N5/00—Radiation therapy
- A61N5/10—X-ray therapy; Gamma-ray therapy; Particle-irradiation therapy
- A61N5/103—Treatment planning systems
- A61N5/1031—Treatment planning systems using a specific method of dose optimization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/10—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges for stereotaxic surgery, e.g. frame-based stereotaxis
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Pathology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Radiology & Medical Imaging (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
- Apparatus For Radiation Diagnosis (AREA)
Abstract
21 THREE-DIMENSIONAL GRAPHICS SIMULATION AND ACTUAL IMAGING DATA COMPOSITE DISPLAY ABSTRACT OF THE DISCLOSURE The disclosure is directed to the combination of actual imaging data with simulated graphics to provide a two-dimensional composite display, such as on a computer screen display, of a three-dimensional relationship between actual images and simulated images. The apparatus and method of the invention are useful for medical applications, particularly stereotactic procedures.
Description
PATLNT APPLICATION
TNREE-DIMENSION~L GRAPHICS SIMUL~TION
AND ~CTUAL IM~GING DAT~ COM~OSITE DISPL~Y
CROSS-REFERENCE TO A RELATED APPLICATION
This application is a continuation-in-part application of U.S.
Patent Application Serial No. 07/290,316, entitled Method and Apparatus for Video Presentation from a Variety of Scanner Imaging Sources, to Tyrone L. Hardy, filed on December 23, 1988, further filed as Canadian Patent Application Serial No. 612,019, filed on September 19, 1989, and U.S. Patent Application Serial No. 07,428,242, entitled Three Dimensional Laser Locali7ation Apparatus and Method for Stereotactic Diagnoses or Surgery, to Tyrone L. Hardy, et al., filed on October 27, 1989, the teachings of which are incorporated herein by reference.
BACKGROUND OF THE INVENTION
Field of the Invention (Technical Field):
The present invention relates to an apparatus and method for combining real or actual three-dimensional scanning or imaging data with simulated graphics so that the three-dimensional nature of structures within a given area or region, such as a head, can be viewed in relationship to the original image data.
Background Art:
There are primarily two methods in the art for three-dimensional (3-D) medical image simulation using various diagnostic scanning techniques, e.g., magnetic resonance imaging (MRI) or nuclear magnetic resonance (NMR) imaging, computer tomography (CT), various isotope imaging techniques, other multi-planar scanners, and the like. These two methods are as follows:
1. Line Tracing or Boundary Marking of Data Taken From Image Sections. In this method, line tracing or boundary marking or contouring of image data taken from various image sections are used to create "wire fr~me" graphic simulations of the contour margins of each section and thereby simulate image data in a three-dimensional fashion. Such wire frame simulations are usually rotated so that ~-~ 2 2079094 they -can be viewed from several perspectives to create a three-dimens1Onal i~age. In some cases, the wire frame simulations are shaded to produce solids which can then be simulated in a three-dimensional fashion with associated simulation of light source views froo various directions. The disadvantage of this technique is that the original image, from techniques such as CT, MRI, etc., is discarded once the wire frame tracings have been made. Valuable data with regard to the quality and features in the original image data are lost or not used. For example, this is somewhat like comparing a photograph of an ob~ect to a line tracing of an ob~ect. In essence, the original image is used to create another image which is a rough simulation of some features of the original image.
Examples of this method are disclosed in the following patents.
U.S. Patent No. 4,791,934, entitled Computer Tomography Assisted Stereotactic Surgery System and Method, to Brunnett, discloses a two-dimensional "shadowgraph" comparison of two images to obtain an indication of the relative spacial orientation and position between the patient and a map. The actual CT data is discarded after rendering the images.
TNREE-DIMENSION~L GRAPHICS SIMUL~TION
AND ~CTUAL IM~GING DAT~ COM~OSITE DISPL~Y
CROSS-REFERENCE TO A RELATED APPLICATION
This application is a continuation-in-part application of U.S.
Patent Application Serial No. 07/290,316, entitled Method and Apparatus for Video Presentation from a Variety of Scanner Imaging Sources, to Tyrone L. Hardy, filed on December 23, 1988, further filed as Canadian Patent Application Serial No. 612,019, filed on September 19, 1989, and U.S. Patent Application Serial No. 07,428,242, entitled Three Dimensional Laser Locali7ation Apparatus and Method for Stereotactic Diagnoses or Surgery, to Tyrone L. Hardy, et al., filed on October 27, 1989, the teachings of which are incorporated herein by reference.
BACKGROUND OF THE INVENTION
Field of the Invention (Technical Field):
The present invention relates to an apparatus and method for combining real or actual three-dimensional scanning or imaging data with simulated graphics so that the three-dimensional nature of structures within a given area or region, such as a head, can be viewed in relationship to the original image data.
Background Art:
There are primarily two methods in the art for three-dimensional (3-D) medical image simulation using various diagnostic scanning techniques, e.g., magnetic resonance imaging (MRI) or nuclear magnetic resonance (NMR) imaging, computer tomography (CT), various isotope imaging techniques, other multi-planar scanners, and the like. These two methods are as follows:
1. Line Tracing or Boundary Marking of Data Taken From Image Sections. In this method, line tracing or boundary marking or contouring of image data taken from various image sections are used to create "wire fr~me" graphic simulations of the contour margins of each section and thereby simulate image data in a three-dimensional fashion. Such wire frame simulations are usually rotated so that ~-~ 2 2079094 they -can be viewed from several perspectives to create a three-dimens1Onal i~age. In some cases, the wire frame simulations are shaded to produce solids which can then be simulated in a three-dimensional fashion with associated simulation of light source views froo various directions. The disadvantage of this technique is that the original image, from techniques such as CT, MRI, etc., is discarded once the wire frame tracings have been made. Valuable data with regard to the quality and features in the original image data are lost or not used. For example, this is somewhat like comparing a photograph of an ob~ect to a line tracing of an ob~ect. In essence, the original image is used to create another image which is a rough simulation of some features of the original image.
Examples of this method are disclosed in the following patents.
U.S. Patent No. 4,791,934, entitled Computer Tomography Assisted Stereotactic Surgery System and Method, to Brunnett, discloses a two-dimensional "shadowgraph" comparison of two images to obtain an indication of the relative spacial orientation and position between the patient and a map. The actual CT data is discarded after rendering the images.
2. Use of Original Imaee Data to Create a_Three-Dimensional i_- In this particular method, image sections are combined to produce a three-dimenslonal reconstructed solid of the sub~ect area or ob~ect. The original image data is not lost but there is no transparency; hence various structures within the sub~ect area cannot be viewed in a three-dimensional manner in relationship to surrounding areas. Usually, the reconstructed solid form is processed in such a fashion that various two-dimensional surface sections can be viewed. Although this may be superior to standard two-dimensional imaging, there is still difficulty in viewing three-dimensional relationships of subject parts within the reconstructeds~lid. For example, if a computer simulation of the head with all its contents is reconstructed to produce a solid simulation, one initially has no more information than if they were viewing the head with the naked eye. Thus this kind of reconstruction does not present any more useful data than can be obtained by visual observation. In an attempt to overcome this difficulty, methods for producing computer sections along different planes have been done to .. . .. .. .. .... . . . .... ...... . .. . . . . .
: ; , , -: . :: : , ,: - . . .~: : :~ : ; . .:. . : : . .:: :: ;
207909~ .
_ ~ 3 "seen-inside the solid. As can be appreciated by those skilled in the art, two-dimensional sections from various angles can produce images which are themselves confusing in terms of anatomical relations.
An example of this method includes U.S. Patent No. 4,777,598, entitled Image Processing Systems and Methods, to Kellar, et al., which generates a three-dimensional reconstruction from two-dimensional slices. However, once the solid is rendered, one cannot see structures or features inside the solid. Also, the `598 patent does not disclose mixing of images generated from actual data with simulated graphics.
Other methods are utilized in the art for manipulating actual image data, however, these do not combine actual data image representations with simulated images of selected features of diagnostic, therapeutic, or surgical devices or body ob~ects or features. Examples of such methods are disclosed in: U.S. Patent No. 4,259,725, entitled Cursor Generator for Use in Computerized Tomography and Other Image Display Systems, to Andrews, et al., which is addressed to overlaying a displayed image with another two-dimensional simple image, such as a pointer, arrow, star, circle,rectangle, and the like; and U.S. Patent No. 4,608,635, entitled ~ethod and Apparatus for Tomographic Diagnosis, to Osterholm, which discloses a tomographic diagnosis system utilizing an apparatus which images the radiodensity of various regions of the body and compares actual images with predetermined images generated from actual patient data; and U.S. Patent No. 4,651,732, entitled Three-Dimensional Light Guidance System for Invasive Procedures, to Frederick, which discloses a three-dimensional guidance system which develops a line of light above a patient's body to indicate the entry point and path of an invasive instrument.
None of the methods or devices described above combine real or actual three-dimensional imaging data with simulations of features of actual structures or techniques to be employed pertaining to such structures. For exa~ple, the present invention can combine images representing actual data of a patient's tumor, and com~ine this image with simulations, such as tumor volume; treatment zones to treat the . ,. .; . : , i. ., :.. ~ , . . -4 207~
tumor, such as radiation implant zones; path of a probe to reach the tumor; and the like.
SUMMARY OF TH~ INVENTION
~DISCLOSURE OF THE INVENTION~
The present invention comprises an apparatus for displaying a combined three-dimensional representation. This apparatus comprises means for calling from storage and displaying in a two-dimensional representation, at least one of the selected actual features of an actual three-dimensional object, such as tumors, lesions, abscesses, or abnormalities; means for generating and displaying in a two-dimensional representation a graphic simulation of at least one associated three-dimensional feature (a feature or structure of the actual ob~ect, such as a transparent solid volume or wire frame structure, or a feature or structure of another ob~ect, such as a 15 stereotactic frame, probe, probe trajectory, radiation implant, -radiation zone resulting from a radiation implant, or radiation beam); and means for combining the graphic simulation display with the display of selected actual features as a composite ~mage in a manner suitable for viewing. Useful imaging techniques include CT, NMR, PET, DSA, isotope imaging, and the like.
In the preferred embodiment, the composite image display allows for selection of a two-dimensional planar slice through the actual object and displaying the selected slice in combination with the three-dimensional graphic simulation two-dlmensional display. This two-dimensional planar slice can be displayed at essentially any viewer determined perspective. Also in the preferred embodiment, the graphic simulation provides for the display of a volume above and/or below the planar slice.
The selected actual features and graphic simulations may be displayed in any form, including wire frame representations, transparent solid representations, relative transparencies, and multiple color representations.
The present invention further comprises a method for displaying a combined three-dimensional representation. This method comprises the following steps of:
207909~
-a. imaging an actual three-dimensional object and storing the data;
b. calling from storage and displaying in a two-dimencional representation at least one selected actual feature of the actual ob~ect;
c. generating and displaying in a two-dimensional representation a graphic simulation of at least one associated three-dimensional feature; and d. combining the graphic simulation display with the display of the selected actual feature in a manner suitable for viewing by a user.
The discussion above is applicable to the method of the invention.
It is a primary ob~ect to provide an apparatus and method for combining three-dimensional imaging data with simulated computer graphics for improved diagnostic, therapeutic, and surgical techniques and other clinical procedures.
It is another object to provide an apparatus and method for providing enhanced information about structures in the body to the diagnostician or surgeon.
An advantage of the apparatus and method of the invention is that it improves the utilization of scanning and imaging techniques and devices.
Other ob~ects, advantages and novel features, and further scope of applicability of the present invention will be set forth in part in the detailed description to follow, taken in con~unction wi.h the accompanying drawings, and in part will become apparent to those skilled in the art upon examination of the following, or may be learned by practice of the invention. The objects and advantages of .
the invention may be realized and attained by means of the ;~
instrumentalities and combinations particularly pointed out in the appended claims.
2~79~9~
~ BRIEF DESCRIPTION OF THE DRAWINGS
The accompanying drawings, which are incorporated into and form a part of the specification, illustrate several smbodiments of the present invention and, together with the description, serve to explain the principles of the invention. The drawings are only for the purposes of illustrating a preferred embodiment of the invention and are not to be construed as limiting the invention.
Fig. 1 is a photograph of a computer screen display of an actual CT image section or scan slice combined with a simulated stereotactic frame and a stereotactic surgical probe directed towards a tumor lesion within the confines of the head;
Fig. 2 is a photograph of a computer screen display of an enlarged image section of Fig. 1, showing a wire frame simulation of a tumor volume; ~ -Fig. 3 is a photograph of a computer screen display of an enlarged image section of Fig. 1, showing a simulated wire frame and shading to present the tumor as a three-dimensional shell;
Fig. 4 is a photograph of a computer screen display of an actual scan image slice with simulated wire frame tumor volume above and below the slice and simulated stereotactic probes and radioactive seeds;
Fig. 5 is a photograph of a computer screen display of multiple views of actual scan image slices in relationship to other corresponding CT images and showing simulated radiation treatment zones within a simulated shell of the tumor volume;
Fig. 6 is a photograph of a computer screen display of an enlarged view of an image, such as shown in Fig. 5;
Fig. 7 is a photograph of a computer screen display of multiple views of an example of simulated tumor volumes within the confines of a brain and simulated radiation treatment zones;
.~' ' ' ' ' '', ~ .: '', ;,' ,', ' , ':. ' .. : ' " ' " "',' '' ' " ''' .', '.' ' ' . ., ' '''' '' . ' . '. `
207909~
-Fig. 8 is a photograph of a computer screen display of a wire frame presentation of a stereotactic frane, a tumor, and a probe tra~ectory;
Fig. 9 is a photograph of three-dimensional simulation of a 5 stereotactic frame and actual MRI images in frontal, saggital, and ~ -horizontal vLews using a whole brain voxel proportional method for simulated mapping of each voxel area of the brain viewed; and Fig. 10 is a block diagram of the apparatus and method of the invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
(BEST MODES FOR CARRYING OUT THE INVENTION~
The present invention combines or overlays actual image representations with simulated wire frame or solid graphics such that the three-dimensional nature or features of structures or objects within a given area or region, e.g., head, can be displayed and viewed in relationship to one another. The advantage of the present invention is that the original image data is not discarded to create the three-dimensional simulation, yet associated data is used to construct wire frame or solit contours of three-dimensional forms and structures within the area of interest.
With reference to Fig. 10, the apparatus and method of the invention provide for the following: (a) scanning, acquiring images, or imaging 10 an actual three-dimensional object (e.g., brain), preferably with a multi-slice imaging technique (e.g., from MRI, CT, DSA (Digital Subtraction Angiography), PET (Positron Emission Topography), and other isotope scanners) and storing 12 the data obtained; (b) calling from storage 12 and displaying 14 in a two-dimensional representation (e.g., on a computer display screen or video monitor) a selected feature or structure (e.g., tumor) of the actual object; (c) generating and displaying 14 in a two-di~ensional -representation (e.g., on a computer display screen or video monitor) a graphic simulation of associated three-dimensional features or structure of the actual object or other objects; and (d) combining 14 the graphic simulation display with the display of the selected actual feature or structure in a composite image in a manner suitable 8 2079~
for viewing by a user. The scanner 10 may be directly connected to the storage 12 and display 14 means, such that images are directly acquired, or images may be transferred to the storage 12 and display 14 means via tape, ethernet or camera. Digital data of images may be stored in the storage 12 and display 14 means, such as a computer memory. The digital data in the images is analyzed and displayed to define certain selected features or structures, or to outline the boundaries of certain features or structures. Similar data from other image sections are also analyzed and stored. These data are then utilized to create three-dimensional simulations of selected features or structures, such as wire frame, solid, surface renditions, or transparent three-dimensional renditions. The images created thereby are then displayed with various originally obtained images, such as slices, to create a three-dimensional sLmulation of selected features or structures rendered in various three-dimensional manners, noted above, such that they can be seen in relationship to the actual CT, NMR or other scanning data. Various original scan data can be selectively displayed in relationship to the images created or simulated and rotated and manipulated in various ways.
Images created, as noted above, can be further simulated in various manners to define graphical three-dimensional simulation of other ob~ects (e.g., stereotactic frame, probe, probe tra~ectories, radiation implants, and radiation zones, radiation and laser beams, and the like). Images created and stored can be recalled for further display.
As can be appreciated by those s~illed in the art, the simulation can be of a feature of the actual object (e.g., tumor volume or structure, or other sub-structures, such as blood vessels or ventricular systems within the brain or other body areas), or a feature of another object (stereotactic frame, probe, probe tra~ectories, radiation implants, and radiation zones resulting from proposed radiation implants), which combined with the image from the actual data provides an effective tool for the clinician. The features simulated can be represented as transparent solids, wire frame structures, in varying colors and transparencies, and combinations thereof, and other renditions, depending on the desired visual effect. As an example, a two-dimensional planar slice through the brain generated from actual data, can be combined with graphic 2~79094 simulation of a volumetric entity, such as a tumor, above and below the planar slice. The planar slice may be viewed at essentially any viewer determined perspective.
The apparatus and method of the invention preferably utilize the imaging methods and apparatus of U.S. Serial No. 07/290,316, entitled t~ethod and Apparatus for Video Presentation from a Variety of Scanner Imaging Sources, to Tyrone L. Harty, to generate more accurate images from actual data or actual data combined with known data (e.g., brain maps), although other techniques for generating images from actual data could be utilized in the invention. The resulting images from the actual data is then combined with simulated graphics data to yield three-dimensional information useful for diagnostic, therapeutic and surgical techniques and other clinical procedures. The graphics simulations can be accomplished by methods well known in the art using standard three-dimensional transformation algorithms or methods, or by the methods described in patent application Serial No. 07/290,316. The position of wire frame or other renditions of structures or features within any image is transposed to the actual image by these techniques.
In the preferred embodiment, the computer is connected to a video printer, camera, magnetic tape, ethernet image acquisition interface, or the like, so that black and white or preferably color copies or photos can be printed whenever the viewer desires to make a record of the procedure or images obtained. Also in the preferred embodiment, all resulting images are stored (e.g., on magnetic or video tape) so that an archive record can be maintained by the *
clinician.
The preferred apparatus and method of the invention provide for one or more of the following features: acquisition of images directly from any scanner; pseudo-coloring; edge detection; pixel analysis; precise volume and point-to-point measurements; perspective viewing of multiple image sections; and Flicker-Frame-Overlay for transparency viewing (an image display technique in which the viewer of a computer video monitor screen is presented with alternating frames of different images or images from different sources in a rapidly alternating manner such that images from two different - : .. : . . .. :;: . . : , ...
', '' `, , ' ` ;'' , `.: ', ''-",'," ','` ;, , :-', , , ` ' '' ''. ',' . r,. :: '. , .
207909~
sources appear to be transparent to one another, i.e., two images are presented in the same frame of reference for comparison). For medical applications, the invention may provide for one or more of the following features: preoperative planning and simulation of the operative procedure; and tailored graphic scaling of brain maps, probe tracks, electrophysiological maps, and volumetric and coordinate measurements to patient images. For stereotactic techniques, the invention may provide for one or more of the following features: probe coordinate determination; probe tra~ectory simulation; anatomical or electrophysiological diencephalic mapping;
whole brain mapping (e.g., Talairach/Tournoux); three-dimensional brachytherapy optimization; three-dimensional laser localization for open craniotomy, such as disclosed in co-pending application Serial No. 07,428,242, entitled Three Dimensional Laser Localization Apparatus and ~ethod for Stereotactic Diagnoses or Surgery, to Tyrone L. Hardy, et al.; three-dimensional microsurgical simulation; three-dimensional angiography; three-dimensional radiosurgery; and three-dimensional imaging and graphic simulation of the stereotactic frame.
The mapping systems can be individually scaled and superimposed over any patient's image for specific mapping purposes. For probe simulation, the invention provides for rapid probe coordinate determination and various simulations of the probe's position;
perspective; a graphic simulation of the probe as it travels through a series of stacked slices; a three-way graphic view of the probe on combinations of sagittal, frontal and horizontal images from various scanners; and three-dimensional graphic simulation of the probe, pointing indicators, or the like, within a stereotactic frame.
Throughout the specification and claims, the term "probe" includes other pointing indicators, such as laser beams, X-ray or radiation beams, other beams, tra~ectories, and the like.
Reference is now made to the drawings which illustrate photographs of actual computer screen displays of actual imaging data combined with simulated graphics such as a stereotactic frame, probe and probe tra~ectories, radiation implants, wire frame, solids, shaded volumes, and shells. The icons and numbers shown in the photographs represent co~puter graphics control symbols. Although the Figures show black and white photographs, the computer graphics . ~. ~ : : . :. ,: - ` ~ : . : . :: .
11 207909~
: `
displays are preferably in varying colors so that the information can be more readily discerned.
Fig. 1 shows an actual CT scan planar slice simulated within a stereotactic frame with a stereotactic surgical probe directed toward a tumor lesion within the confines of the head. Figs. 2 and 3 are magnified or enlarged views of the image section shown in Fig. 1.
Fig. 2 is a three-dimensional wire frame simulation of a tumor volume and structure relative to the image slice, and Fig. 3 shows wire frame simulation shading to present the tumor as a three-dimensional shell. In Fig. 3, the wire frame simulation of the tumor within the confines of the brain has been shaded such that it appears as a shell.
As can be seen by the drawings, by using the method and apparatus of the present invention, one can choose a feature of structure of interest within the confines of the imaged area and simulate that structure in wire frame or solid form showing its relationship to the image slices. The relationship of the structure of interest to the surrounding anatomical areas is enhanced. Images created in this fashion can be viewed from a number of perspectives such that simulations of a structure above and below an image slice can be viewed. An important feature of this method of the present invention is the creation of image sections having true transparency.
That is, by example in Figs. 2 and 3, the three-dimensional construction of the tumor volume and structure can be seen above and below the image slice. By analogy, this is somewhat like having a painting on a very fine mesh screen by which one can simultaneously see through the screen to see ob~ects behind the screen, yet also be able to see the painting on the screen.
Fig. 4 is another example of a three-dimensional wire frame simulation of a tumor lesion within the confines of the brain viewed in relation to an actual CT scan image slice. In this particular display, again the tumor volume and structure above and below the image slice can be viewed. Vertical stereotactic surgical probes are simulated to show possible tra~ectories to target areas within the confines of the tumor volume and structure. The bright white spheres on the ends of the si~ulated probes simulate the position of . .
2~7909~
radio-isotope seeds placed within the confines of the tumor as a therapeutic treatment modality. The larger three-dimensional balls or spheres within the confines of the three-dimensional wire frame simulation of the tumor simulates the kill zones or therapeutic zones for some additional isotope seeds. The radioisotope seeds are present at the center of the spheres. This Figure is an example of how multiple areas of interest can be graphically simulated relative to an actual image slice. The surgical or therapeutic techniques can be optimized by varying the position of the radioisotope seeds and by varying the type of radioisotopes, which can be simulated prior to employing the desired procedures.
Fig. 5 shows multiple views of the same three-dimensional tumor volume and structure relative to another corresponding CT image showing three-dimensional simulated radiation treatment zones within shell and solid renditions of the tumor volume and structure. The simulated radiation treatment zones (dark areas) which extend beyond the confines of the simulated tumor volume and structure (light areas) can be readily discerned. For demonstration purposes, an enlarged computer rendition is shown in Fig. 6. Fig. 7 is another rendition showing radiation treatment zones within the wire frame simulation of the tumor volume and structure and where treatment zones extend beyond the confines of the tumor. Such simulated radiation zones around each seed are used in the preliminary stages of radiation treatment optimization before final dose calculations are performed. The grids (e.g., 1 cm square) can be used as a measuring aid. This particular method can be used to optimize treatoent of tumor lesions, that is the goal is to have the radiation treatment zones touch the margins of the tumor volume and structure, but not break out substantially beyond the margins.
This combining of actual image data with graphics simulation of related three-dimensional entities and image transparency can be used to simulate other therapeutic diagnostic and relational properties.
For example, Fig. 8 illustrates a wire frame presentation of a stereotactic frame, a tumor within the confines of the brain and the simulation of a therapeutic probe's tr~a~ectory toward a target area within the confines of the tumor.
:~, , :
~ 13 2~79~9~
-Fig. 9 is another example showing a three-dimensional simulation of MRI images in frontal, saggital, and horizontal views using a whole brain voxel proportional method for mapping each voxel area of the brain viewed. Various image slices can be presented in a single fashion or in various combinations to study specific areas of interest. Although this photograph shows orthogonal images, images at other angles and in varying combinations may be utilized depending upon the area of interest. This method can also be used, as stated before, to show the three-dimensional relationships of various structures within the confines of the brain in relationship to other structures. For example, the ventricular system, certain blood vessels, tumors, and other structural areas, lesions, abscesses, abnormalities of the brain (or other areas of the body), can be simulated in their three-dimensional relationships to other brain (or other body) areas.
In the drawings, actual image data points are utilized in the construction of the wire frame renditions. The lines connect these actual data points to complete the frame. ;.
Although the drawings all show actual imaging data of brains in combination with a stereotactic frame, the invention is not limited to heads or brains, but could be ut$1ized on any body portion wherein accurate and deta$1ed information is desired to be combined with simulation~. For example, the apparatus and method of the invention could be useful in the placement of objects into the body, such as blood vessels during an angiogram, radioisotope therapy, biopsies, radiosurgery, the optimization of laser or stereotactic positioning, surgical or diagnostic methods for other body parts with a frame system suitable for such body parts (e.g., cylindrical coordinate system for a spinal stereotactic frame, or the like). The present invention provides for a high degree of accuracy and one can tell p~ecisely what specific anatomical areas are within the simulated regions. The apparatus of the invention is also useful in con~unction with attachments to frames, such as microscopes, lasers, robots, other stereotactic devices or pointers, and radiation delivery systems.
237~99~
: : 14 "
Although the invention has been described with reference to these preferred embodiments, other embodiments can achieve the same results. Variations and modifications of the present invention will be obvious to those skilled in the art and it is intended to cover in the appended clai~s all such modifications and equivalents.
: ; , , -: . :: : , ,: - . . .~: : :~ : ; . .:. . : : . .:: :: ;
207909~ .
_ ~ 3 "seen-inside the solid. As can be appreciated by those skilled in the art, two-dimensional sections from various angles can produce images which are themselves confusing in terms of anatomical relations.
An example of this method includes U.S. Patent No. 4,777,598, entitled Image Processing Systems and Methods, to Kellar, et al., which generates a three-dimensional reconstruction from two-dimensional slices. However, once the solid is rendered, one cannot see structures or features inside the solid. Also, the `598 patent does not disclose mixing of images generated from actual data with simulated graphics.
Other methods are utilized in the art for manipulating actual image data, however, these do not combine actual data image representations with simulated images of selected features of diagnostic, therapeutic, or surgical devices or body ob~ects or features. Examples of such methods are disclosed in: U.S. Patent No. 4,259,725, entitled Cursor Generator for Use in Computerized Tomography and Other Image Display Systems, to Andrews, et al., which is addressed to overlaying a displayed image with another two-dimensional simple image, such as a pointer, arrow, star, circle,rectangle, and the like; and U.S. Patent No. 4,608,635, entitled ~ethod and Apparatus for Tomographic Diagnosis, to Osterholm, which discloses a tomographic diagnosis system utilizing an apparatus which images the radiodensity of various regions of the body and compares actual images with predetermined images generated from actual patient data; and U.S. Patent No. 4,651,732, entitled Three-Dimensional Light Guidance System for Invasive Procedures, to Frederick, which discloses a three-dimensional guidance system which develops a line of light above a patient's body to indicate the entry point and path of an invasive instrument.
None of the methods or devices described above combine real or actual three-dimensional imaging data with simulations of features of actual structures or techniques to be employed pertaining to such structures. For exa~ple, the present invention can combine images representing actual data of a patient's tumor, and com~ine this image with simulations, such as tumor volume; treatment zones to treat the . ,. .; . : , i. ., :.. ~ , . . -4 207~
tumor, such as radiation implant zones; path of a probe to reach the tumor; and the like.
SUMMARY OF TH~ INVENTION
~DISCLOSURE OF THE INVENTION~
The present invention comprises an apparatus for displaying a combined three-dimensional representation. This apparatus comprises means for calling from storage and displaying in a two-dimensional representation, at least one of the selected actual features of an actual three-dimensional object, such as tumors, lesions, abscesses, or abnormalities; means for generating and displaying in a two-dimensional representation a graphic simulation of at least one associated three-dimensional feature (a feature or structure of the actual ob~ect, such as a transparent solid volume or wire frame structure, or a feature or structure of another ob~ect, such as a 15 stereotactic frame, probe, probe trajectory, radiation implant, -radiation zone resulting from a radiation implant, or radiation beam); and means for combining the graphic simulation display with the display of selected actual features as a composite ~mage in a manner suitable for viewing. Useful imaging techniques include CT, NMR, PET, DSA, isotope imaging, and the like.
In the preferred embodiment, the composite image display allows for selection of a two-dimensional planar slice through the actual object and displaying the selected slice in combination with the three-dimensional graphic simulation two-dlmensional display. This two-dimensional planar slice can be displayed at essentially any viewer determined perspective. Also in the preferred embodiment, the graphic simulation provides for the display of a volume above and/or below the planar slice.
The selected actual features and graphic simulations may be displayed in any form, including wire frame representations, transparent solid representations, relative transparencies, and multiple color representations.
The present invention further comprises a method for displaying a combined three-dimensional representation. This method comprises the following steps of:
207909~
-a. imaging an actual three-dimensional object and storing the data;
b. calling from storage and displaying in a two-dimencional representation at least one selected actual feature of the actual ob~ect;
c. generating and displaying in a two-dimensional representation a graphic simulation of at least one associated three-dimensional feature; and d. combining the graphic simulation display with the display of the selected actual feature in a manner suitable for viewing by a user.
The discussion above is applicable to the method of the invention.
It is a primary ob~ect to provide an apparatus and method for combining three-dimensional imaging data with simulated computer graphics for improved diagnostic, therapeutic, and surgical techniques and other clinical procedures.
It is another object to provide an apparatus and method for providing enhanced information about structures in the body to the diagnostician or surgeon.
An advantage of the apparatus and method of the invention is that it improves the utilization of scanning and imaging techniques and devices.
Other ob~ects, advantages and novel features, and further scope of applicability of the present invention will be set forth in part in the detailed description to follow, taken in con~unction wi.h the accompanying drawings, and in part will become apparent to those skilled in the art upon examination of the following, or may be learned by practice of the invention. The objects and advantages of .
the invention may be realized and attained by means of the ;~
instrumentalities and combinations particularly pointed out in the appended claims.
2~79~9~
~ BRIEF DESCRIPTION OF THE DRAWINGS
The accompanying drawings, which are incorporated into and form a part of the specification, illustrate several smbodiments of the present invention and, together with the description, serve to explain the principles of the invention. The drawings are only for the purposes of illustrating a preferred embodiment of the invention and are not to be construed as limiting the invention.
Fig. 1 is a photograph of a computer screen display of an actual CT image section or scan slice combined with a simulated stereotactic frame and a stereotactic surgical probe directed towards a tumor lesion within the confines of the head;
Fig. 2 is a photograph of a computer screen display of an enlarged image section of Fig. 1, showing a wire frame simulation of a tumor volume; ~ -Fig. 3 is a photograph of a computer screen display of an enlarged image section of Fig. 1, showing a simulated wire frame and shading to present the tumor as a three-dimensional shell;
Fig. 4 is a photograph of a computer screen display of an actual scan image slice with simulated wire frame tumor volume above and below the slice and simulated stereotactic probes and radioactive seeds;
Fig. 5 is a photograph of a computer screen display of multiple views of actual scan image slices in relationship to other corresponding CT images and showing simulated radiation treatment zones within a simulated shell of the tumor volume;
Fig. 6 is a photograph of a computer screen display of an enlarged view of an image, such as shown in Fig. 5;
Fig. 7 is a photograph of a computer screen display of multiple views of an example of simulated tumor volumes within the confines of a brain and simulated radiation treatment zones;
.~' ' ' ' ' '', ~ .: '', ;,' ,', ' , ':. ' .. : ' " ' " "',' '' ' " ''' .', '.' ' ' . ., ' '''' '' . ' . '. `
207909~
-Fig. 8 is a photograph of a computer screen display of a wire frame presentation of a stereotactic frane, a tumor, and a probe tra~ectory;
Fig. 9 is a photograph of three-dimensional simulation of a 5 stereotactic frame and actual MRI images in frontal, saggital, and ~ -horizontal vLews using a whole brain voxel proportional method for simulated mapping of each voxel area of the brain viewed; and Fig. 10 is a block diagram of the apparatus and method of the invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
(BEST MODES FOR CARRYING OUT THE INVENTION~
The present invention combines or overlays actual image representations with simulated wire frame or solid graphics such that the three-dimensional nature or features of structures or objects within a given area or region, e.g., head, can be displayed and viewed in relationship to one another. The advantage of the present invention is that the original image data is not discarded to create the three-dimensional simulation, yet associated data is used to construct wire frame or solit contours of three-dimensional forms and structures within the area of interest.
With reference to Fig. 10, the apparatus and method of the invention provide for the following: (a) scanning, acquiring images, or imaging 10 an actual three-dimensional object (e.g., brain), preferably with a multi-slice imaging technique (e.g., from MRI, CT, DSA (Digital Subtraction Angiography), PET (Positron Emission Topography), and other isotope scanners) and storing 12 the data obtained; (b) calling from storage 12 and displaying 14 in a two-dimensional representation (e.g., on a computer display screen or video monitor) a selected feature or structure (e.g., tumor) of the actual object; (c) generating and displaying 14 in a two-di~ensional -representation (e.g., on a computer display screen or video monitor) a graphic simulation of associated three-dimensional features or structure of the actual object or other objects; and (d) combining 14 the graphic simulation display with the display of the selected actual feature or structure in a composite image in a manner suitable 8 2079~
for viewing by a user. The scanner 10 may be directly connected to the storage 12 and display 14 means, such that images are directly acquired, or images may be transferred to the storage 12 and display 14 means via tape, ethernet or camera. Digital data of images may be stored in the storage 12 and display 14 means, such as a computer memory. The digital data in the images is analyzed and displayed to define certain selected features or structures, or to outline the boundaries of certain features or structures. Similar data from other image sections are also analyzed and stored. These data are then utilized to create three-dimensional simulations of selected features or structures, such as wire frame, solid, surface renditions, or transparent three-dimensional renditions. The images created thereby are then displayed with various originally obtained images, such as slices, to create a three-dimensional sLmulation of selected features or structures rendered in various three-dimensional manners, noted above, such that they can be seen in relationship to the actual CT, NMR or other scanning data. Various original scan data can be selectively displayed in relationship to the images created or simulated and rotated and manipulated in various ways.
Images created, as noted above, can be further simulated in various manners to define graphical three-dimensional simulation of other ob~ects (e.g., stereotactic frame, probe, probe tra~ectories, radiation implants, and radiation zones, radiation and laser beams, and the like). Images created and stored can be recalled for further display.
As can be appreciated by those s~illed in the art, the simulation can be of a feature of the actual object (e.g., tumor volume or structure, or other sub-structures, such as blood vessels or ventricular systems within the brain or other body areas), or a feature of another object (stereotactic frame, probe, probe tra~ectories, radiation implants, and radiation zones resulting from proposed radiation implants), which combined with the image from the actual data provides an effective tool for the clinician. The features simulated can be represented as transparent solids, wire frame structures, in varying colors and transparencies, and combinations thereof, and other renditions, depending on the desired visual effect. As an example, a two-dimensional planar slice through the brain generated from actual data, can be combined with graphic 2~79094 simulation of a volumetric entity, such as a tumor, above and below the planar slice. The planar slice may be viewed at essentially any viewer determined perspective.
The apparatus and method of the invention preferably utilize the imaging methods and apparatus of U.S. Serial No. 07/290,316, entitled t~ethod and Apparatus for Video Presentation from a Variety of Scanner Imaging Sources, to Tyrone L. Harty, to generate more accurate images from actual data or actual data combined with known data (e.g., brain maps), although other techniques for generating images from actual data could be utilized in the invention. The resulting images from the actual data is then combined with simulated graphics data to yield three-dimensional information useful for diagnostic, therapeutic and surgical techniques and other clinical procedures. The graphics simulations can be accomplished by methods well known in the art using standard three-dimensional transformation algorithms or methods, or by the methods described in patent application Serial No. 07/290,316. The position of wire frame or other renditions of structures or features within any image is transposed to the actual image by these techniques.
In the preferred embodiment, the computer is connected to a video printer, camera, magnetic tape, ethernet image acquisition interface, or the like, so that black and white or preferably color copies or photos can be printed whenever the viewer desires to make a record of the procedure or images obtained. Also in the preferred embodiment, all resulting images are stored (e.g., on magnetic or video tape) so that an archive record can be maintained by the *
clinician.
The preferred apparatus and method of the invention provide for one or more of the following features: acquisition of images directly from any scanner; pseudo-coloring; edge detection; pixel analysis; precise volume and point-to-point measurements; perspective viewing of multiple image sections; and Flicker-Frame-Overlay for transparency viewing (an image display technique in which the viewer of a computer video monitor screen is presented with alternating frames of different images or images from different sources in a rapidly alternating manner such that images from two different - : .. : . . .. :;: . . : , ...
', '' `, , ' ` ;'' , `.: ', ''-",'," ','` ;, , :-', , , ` ' '' ''. ',' . r,. :: '. , .
207909~
sources appear to be transparent to one another, i.e., two images are presented in the same frame of reference for comparison). For medical applications, the invention may provide for one or more of the following features: preoperative planning and simulation of the operative procedure; and tailored graphic scaling of brain maps, probe tracks, electrophysiological maps, and volumetric and coordinate measurements to patient images. For stereotactic techniques, the invention may provide for one or more of the following features: probe coordinate determination; probe tra~ectory simulation; anatomical or electrophysiological diencephalic mapping;
whole brain mapping (e.g., Talairach/Tournoux); three-dimensional brachytherapy optimization; three-dimensional laser localization for open craniotomy, such as disclosed in co-pending application Serial No. 07,428,242, entitled Three Dimensional Laser Localization Apparatus and ~ethod for Stereotactic Diagnoses or Surgery, to Tyrone L. Hardy, et al.; three-dimensional microsurgical simulation; three-dimensional angiography; three-dimensional radiosurgery; and three-dimensional imaging and graphic simulation of the stereotactic frame.
The mapping systems can be individually scaled and superimposed over any patient's image for specific mapping purposes. For probe simulation, the invention provides for rapid probe coordinate determination and various simulations of the probe's position;
perspective; a graphic simulation of the probe as it travels through a series of stacked slices; a three-way graphic view of the probe on combinations of sagittal, frontal and horizontal images from various scanners; and three-dimensional graphic simulation of the probe, pointing indicators, or the like, within a stereotactic frame.
Throughout the specification and claims, the term "probe" includes other pointing indicators, such as laser beams, X-ray or radiation beams, other beams, tra~ectories, and the like.
Reference is now made to the drawings which illustrate photographs of actual computer screen displays of actual imaging data combined with simulated graphics such as a stereotactic frame, probe and probe tra~ectories, radiation implants, wire frame, solids, shaded volumes, and shells. The icons and numbers shown in the photographs represent co~puter graphics control symbols. Although the Figures show black and white photographs, the computer graphics . ~. ~ : : . :. ,: - ` ~ : . : . :: .
11 207909~
: `
displays are preferably in varying colors so that the information can be more readily discerned.
Fig. 1 shows an actual CT scan planar slice simulated within a stereotactic frame with a stereotactic surgical probe directed toward a tumor lesion within the confines of the head. Figs. 2 and 3 are magnified or enlarged views of the image section shown in Fig. 1.
Fig. 2 is a three-dimensional wire frame simulation of a tumor volume and structure relative to the image slice, and Fig. 3 shows wire frame simulation shading to present the tumor as a three-dimensional shell. In Fig. 3, the wire frame simulation of the tumor within the confines of the brain has been shaded such that it appears as a shell.
As can be seen by the drawings, by using the method and apparatus of the present invention, one can choose a feature of structure of interest within the confines of the imaged area and simulate that structure in wire frame or solid form showing its relationship to the image slices. The relationship of the structure of interest to the surrounding anatomical areas is enhanced. Images created in this fashion can be viewed from a number of perspectives such that simulations of a structure above and below an image slice can be viewed. An important feature of this method of the present invention is the creation of image sections having true transparency.
That is, by example in Figs. 2 and 3, the three-dimensional construction of the tumor volume and structure can be seen above and below the image slice. By analogy, this is somewhat like having a painting on a very fine mesh screen by which one can simultaneously see through the screen to see ob~ects behind the screen, yet also be able to see the painting on the screen.
Fig. 4 is another example of a three-dimensional wire frame simulation of a tumor lesion within the confines of the brain viewed in relation to an actual CT scan image slice. In this particular display, again the tumor volume and structure above and below the image slice can be viewed. Vertical stereotactic surgical probes are simulated to show possible tra~ectories to target areas within the confines of the tumor volume and structure. The bright white spheres on the ends of the si~ulated probes simulate the position of . .
2~7909~
radio-isotope seeds placed within the confines of the tumor as a therapeutic treatment modality. The larger three-dimensional balls or spheres within the confines of the three-dimensional wire frame simulation of the tumor simulates the kill zones or therapeutic zones for some additional isotope seeds. The radioisotope seeds are present at the center of the spheres. This Figure is an example of how multiple areas of interest can be graphically simulated relative to an actual image slice. The surgical or therapeutic techniques can be optimized by varying the position of the radioisotope seeds and by varying the type of radioisotopes, which can be simulated prior to employing the desired procedures.
Fig. 5 shows multiple views of the same three-dimensional tumor volume and structure relative to another corresponding CT image showing three-dimensional simulated radiation treatment zones within shell and solid renditions of the tumor volume and structure. The simulated radiation treatment zones (dark areas) which extend beyond the confines of the simulated tumor volume and structure (light areas) can be readily discerned. For demonstration purposes, an enlarged computer rendition is shown in Fig. 6. Fig. 7 is another rendition showing radiation treatment zones within the wire frame simulation of the tumor volume and structure and where treatment zones extend beyond the confines of the tumor. Such simulated radiation zones around each seed are used in the preliminary stages of radiation treatment optimization before final dose calculations are performed. The grids (e.g., 1 cm square) can be used as a measuring aid. This particular method can be used to optimize treatoent of tumor lesions, that is the goal is to have the radiation treatment zones touch the margins of the tumor volume and structure, but not break out substantially beyond the margins.
This combining of actual image data with graphics simulation of related three-dimensional entities and image transparency can be used to simulate other therapeutic diagnostic and relational properties.
For example, Fig. 8 illustrates a wire frame presentation of a stereotactic frame, a tumor within the confines of the brain and the simulation of a therapeutic probe's tr~a~ectory toward a target area within the confines of the tumor.
:~, , :
~ 13 2~79~9~
-Fig. 9 is another example showing a three-dimensional simulation of MRI images in frontal, saggital, and horizontal views using a whole brain voxel proportional method for mapping each voxel area of the brain viewed. Various image slices can be presented in a single fashion or in various combinations to study specific areas of interest. Although this photograph shows orthogonal images, images at other angles and in varying combinations may be utilized depending upon the area of interest. This method can also be used, as stated before, to show the three-dimensional relationships of various structures within the confines of the brain in relationship to other structures. For example, the ventricular system, certain blood vessels, tumors, and other structural areas, lesions, abscesses, abnormalities of the brain (or other areas of the body), can be simulated in their three-dimensional relationships to other brain (or other body) areas.
In the drawings, actual image data points are utilized in the construction of the wire frame renditions. The lines connect these actual data points to complete the frame. ;.
Although the drawings all show actual imaging data of brains in combination with a stereotactic frame, the invention is not limited to heads or brains, but could be ut$1ized on any body portion wherein accurate and deta$1ed information is desired to be combined with simulation~. For example, the apparatus and method of the invention could be useful in the placement of objects into the body, such as blood vessels during an angiogram, radioisotope therapy, biopsies, radiosurgery, the optimization of laser or stereotactic positioning, surgical or diagnostic methods for other body parts with a frame system suitable for such body parts (e.g., cylindrical coordinate system for a spinal stereotactic frame, or the like). The present invention provides for a high degree of accuracy and one can tell p~ecisely what specific anatomical areas are within the simulated regions. The apparatus of the invention is also useful in con~unction with attachments to frames, such as microscopes, lasers, robots, other stereotactic devices or pointers, and radiation delivery systems.
237~99~
: : 14 "
Although the invention has been described with reference to these preferred embodiments, other embodiments can achieve the same results. Variations and modifications of the present invention will be obvious to those skilled in the art and it is intended to cover in the appended clai~s all such modifications and equivalents.
Claims (35)
1. An apparatus for displaying a combined three-dimensional representation comprising:
means for calling from storage and displaying original image data in a two-dimensional representation of at least one of the selected actual features of an actual three-dimensional object;
means for generating and displaying in a two-dimensional representation, a graphic simulation of at least one associated three-dimensional feature; and means for combining said graphic simulation display with said display of original image data of selected actual features as a composite image in a manner suitable for viewing by a user.
means for calling from storage and displaying original image data in a two-dimensional representation of at least one of the selected actual features of an actual three-dimensional object;
means for generating and displaying in a two-dimensional representation, a graphic simulation of at least one associated three-dimensional feature; and means for combining said graphic simulation display with said display of original image data of selected actual features as a composite image in a manner suitable for viewing by a user.
2. The invention of claim 1 wherein said original image data in a two-dimensional representation display is obtained from at least one imaging scanner selected from the group consisting of CT, NMR, PET, DSA, and isotope imaging scanners.
3. The invention of claim 1 wherein said graphic simulation of said associated three-dimensional feature comprises a graphic simulation of a feature or structure of the actual object.
4. The invention of claim 3 wherein said graphic simulation of said feature of the actual object comprises a volume.
5. The invention of claim 4 wherein said volume is represented as a transparent solid.
6. The invention of claim 4 wherein said volume is represented in a wire frame structure.
7. The invention of claim 1 wherein said graphic simulation of said associated three-dimensional feature comprises a graphic simulation of a feature or structure of another object.
8. The invention of claim 7 wherein said graphic simulation of said another object comprises a graphic simulation of at least one member selected from the group consisting of stereotactic frames, probes, probe trajectories, radiation implants, radiation zones resulting from radiation implants, and radiation beams.
9. The invention of claim 1 wherein said original image data in a two-dimensional representation of at least one of the selected actual features of an actual three-dimensional object comprises at least one selected actual feature in the body selected from the group consisting of tumors, lesions, abscesses, and abnormalities.
10. The invention of claim 1 wherein said means for displaying original image data in a two-dimensional representation of at least one of said selected actual features of an actual three-dimensional object comprises means for selecting original image data of a two-dimensional planar slice through the actual object and means for displaying said selected slice in combination with said three-dimensional graphic simulation two-dimensional display.
11. The invention of claim 10 wherein said means for selecting and displaying original image data for a two-dimensional planar slice comprises means for selecting and displaying the slice at a viewer determined perspective.
12. The invention of claim 10 wherein said graphic simulation of said associated three-dimensional feature comprises a graphic simulation of a volume in at least one region above and below said planar slice.
13. The invention of claim 1 wherein said means for displaying original image data in a two-dimensional representation of at least one of said selected actual features comprises means for displaying a wire frame representation of said selected features.
14. The invention of claim 1 wherein said means for displaying original image data in a two-dimensional representation of at least one of said selected actual features comprises means for displaying a transparent solid representation of said selected features.
15. The invention of claim 1 wherein said means for displaying said graphic simulation of at least one associated three-dimensional feature comprises means for generating a multiple color representation thereof.
16. The invention of claim 1 wherein said means for displaying original image data in a two-dimensional representation of at least one of said selected actual features comprises means for generating a multiple color representation thereof.
17. The invention of claim 1 further comprising means controllable by a user for selecting aspects of the original image data of the selected actual features for relative transparency.
18. The invention of claim 1 further comprising means controllable by a user for selecting aspects of the original image data of the graphic simulation display for relative transparency.
19. A method for displaying a combined three-dimensional representation comprising the following steps of:
a) imaging an actual three-dimensional object and storing the data as original image data;
b) calling from storage and displaying the original image data in a two-dimensional representation of at least one selected actual feature of the actual object;
c) generating and displaying in a two-dimensional representation a graphic simulation of at least one associated three-dimensional feature; and d) combining the graphic simulation display with the original image data display of the selected actual feature in a manner suitable for viewing by a user.
a) imaging an actual three-dimensional object and storing the data as original image data;
b) calling from storage and displaying the original image data in a two-dimensional representation of at least one selected actual feature of the actual object;
c) generating and displaying in a two-dimensional representation a graphic simulation of at least one associated three-dimensional feature; and d) combining the graphic simulation display with the original image data display of the selected actual feature in a manner suitable for viewing by a user.
20. The invention of claim 19 wherein the step of generating and displaying in a two-dimensional representation a graphic simulation of at least one associated three-dimensional feature comprises the step of generating and displaying a feature [or structure] of the actual object.
21. The invention of claim 20 wherein the step of generating and displaying a feature of the actual object comprises the step of generating and displaying a volume.
22. The invention of claim 21 wherein the step of generating and displaying a volume comprises the step of generating and displaying a transparent solid.
23. The invention of claim 21 wherein the step of generating and displaying a volume comprises the step of generating and displaying a wire frame structure.
24. The invention of claim 19 wherein the step of generating and displaying in a two-dimensional representation a graphic simulation of the associated three-dimensional feature comprises the step of generating and displaying a feature or structure of another object.
25. The invention of claim 24 wherein the step of generating and displaying a feature or structure of the other object comprises the step of generating and displaying at least one member selected from the group consisting of stereotactic frames, probes, probe trajectories, radiation implants, radiation zones resulting from radiation implants, and radiation beams.
26. The invention of claim 19 wherein the step of displaying original image data in a two-dimensional representation of at least one selected actual feature of the actual object comprises the step of displaying at least one structure in the body selected from the group consisting of tumors, lesions, abscesses, and abnormalities.
27. The invention of claim 19 wherein the step of displaying original image data in a two-dimensional representation of at least one of said selected actual features further comprises the step of selecting original image data of a two-dimensional planar slice through the actual object and the step of displaying the selected slice in combination with said three-dimensional graphic simulation two-dimensional display.
28. The invention of claim 27 wherein the step of selecting and displaying original image data of a two-dimensional planar slice further comprises the step of selecting and displaying the slice at a viewer determined perspective.
29. The invention of claim 27 wherein the step of generating and displaying a graphic simulation of the associated three-dimensional feature further comprises the step of generating and displaying a volume or structure in at least one region above and below the planar slice.
30. The invention of claim 19 wherein the step of displaying original image data of at least one of said selected actual features further comprises the step of displaying a wire frame representation of said selected features.
31. The invention of claim 19 wherein the step of displaying original image data of at least one of the selected actual features further comprises the step of displaying a transparent solid representation of the selected features.
32. The invention of claim 19 wherein the step of generating and displaying a graphic simulation further comprises the step of generating and displaying a multiple color representation thereof.
33. The invention of claim 19 wherein the step of displaying original image data of at least one of the selected actual features further comprises the step of generating a multiple color representation thereof.
34. The invention of claim 19 further comprising the step of controlling and selecting aspects of original image data of the selected actual features display for relative transparency.
35. The invention of claim 19 further comprising the step of controlling and selecting aspects of the graphic simulation display for relative transparency.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US50078890A | 1990-03-28 | 1990-03-28 | |
| US500,788 | 1990-03-28 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA2079094A1 true CA2079094A1 (en) | 1991-09-29 |
Family
ID=23990939
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002079094A Abandoned CA2079094A1 (en) | 1990-03-28 | 1991-03-27 | Three-dimensional graphics simulation and actual imaging data composite display |
Country Status (2)
| Country | Link |
|---|---|
| CA (1) | CA2079094A1 (en) |
| WO (1) | WO1991014397A1 (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE4207901C3 (en) * | 1992-03-12 | 1999-10-07 | Aesculap Ag & Co Kg | Method and device for displaying a work area in a three-dimensional structure |
| FR2692466B1 (en) * | 1992-06-17 | 1999-11-05 | Gen Electric Cgr | METHOD FOR MONITORING A SURGICAL OPERATION. |
| US6694163B1 (en) | 1994-10-27 | 2004-02-17 | Wake Forest University Health Sciences | Method and system for producing interactive, three-dimensional renderings of selected body organs having hollow lumens to enable simulated movement through the lumen |
| US5782762A (en) | 1994-10-27 | 1998-07-21 | Wake Forest University | Method and system for producing interactive, three-dimensional renderings of selected body organs having hollow lumens to enable simulated movement through the lumen |
| US5920319A (en) | 1994-10-27 | 1999-07-06 | Wake Forest University | Automatic analysis in virtual endoscopy |
| DE59805342D1 (en) * | 1997-10-30 | 2002-10-02 | Baldeweg Ag Dr | METHOD AND DEVICE FOR PROCESSING IMAGE OBJECTS |
| US7301535B2 (en) | 2004-09-02 | 2007-11-27 | Siemens Medical Solutions Usa, Inc. | 3D summary display for reporting of organ tumors |
| EP2087924B1 (en) | 2008-02-07 | 2018-01-24 | Varian Medical Systems Particle Therapy GmbH | Method for generating and visualizing an ion beam profile, treatment planning system and computer software |
| CN105283221B (en) * | 2013-05-22 | 2019-06-18 | 皇家飞利浦有限公司 | For radiation treatment plan used in newly organize compensator thickness visualize and modifiers |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4608635A (en) * | 1982-08-03 | 1986-08-26 | Thomas Jefferson University | Method and apparatus for tomographic diagnosis |
| GB8507449D0 (en) * | 1985-03-22 | 1985-05-01 | Quantel Ltd | Video image processing systems |
| US4888713B1 (en) * | 1986-09-05 | 1993-10-12 | Cdi Technologies, Inc. | Surface detail mapping system |
-
1991
- 1991-03-27 CA CA002079094A patent/CA2079094A1/en not_active Abandoned
- 1991-03-27 WO PCT/US1991/002102 patent/WO1991014397A1/en active Application Filing
Also Published As
| Publication number | Publication date |
|---|---|
| WO1991014397A1 (en) | 1991-10-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Trobaugh et al. | Three-dimensional imaging with stereotactic ultrasonography | |
| Kessler et al. | Integration of multimodality imaging data for radiotherapy treatment planning | |
| Cline et al. | 3D reconstruction of the brain from magnetic resonance images using a connectivity algorithm | |
| EP1365685B1 (en) | 3d planning target volume | |
| US6466813B1 (en) | Method and apparatus for MR-based volumetric frameless 3-D interactive localization, virtual simulation, and dosimetric radiation therapy planning | |
| Adams et al. | Computer-assisted surgery | |
| CN101677827B (en) | Targeting method, targeting device, computer readable medium and program element | |
| CA2003497C (en) | Probe-correlated viewing of anatomical image data | |
| JP4421016B2 (en) | Medical image processing device | |
| US7817836B2 (en) | Methods for volumetric contouring with expert guidance | |
| Lorensen et al. | Enhancing reality in the operating room | |
| Nishidai et al. | CT simulator: a new 3-D planning and simulating system for radiotherapy: Part 1. Description of system | |
| Sakas | Trends in medical imaging: from 2D to 3D | |
| EP0216156A2 (en) | Dividing cubes and method for the display of surface structures contained within the interior region of a solid body | |
| Henri et al. | Multimodality image integration for stereotactic surgical planning | |
| Farrell et al. | Color 3-D imaging of normal and pathologic intracranial structures | |
| JP2002330958A (en) | Method and device for selecting and displaying medical image data | |
| JPH0632042B2 (en) | Device and method for displaying three-dimensional surface structure | |
| Goksel et al. | Prostate brachytherapy training with simulated ultrasound and fluoroscopy images | |
| RU2171630C2 (en) | Method for matching of three-dimensional images obtained with the aid of computer tomographs operating on the basis of various physical principles | |
| Gorniak et al. | Evaluation of a semiautomatic 3D fusion technique applied to molecular imaging and MRI brain/frame volume data sets | |
| JP4446881B2 (en) | How to generate and display 3D volume images | |
| JP2003199741A (en) | Method for obtaining two-dimensional image in tomographic device, and medical tomographic device | |
| CA2079094A1 (en) | Three-dimensional graphics simulation and actual imaging data composite display | |
| JP2005524896A5 (en) |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| FZDE | Discontinued | ||
| FZDE | Discontinued |
Effective date: 19940927 |