METHOD AND SYSTEM FOR USING BREAST
ULTRASOUND INFORMATION TO FACILITATE
BREAST CANCER SCREENING
Shih-Ping Wang
CROSS-REFERENCE TO RELATED APPLICATIONS
This application claims the benefit of United States Provisional Application No. 60/252,946, filed November 24, 2000, which is incorporated by reference herein.
FIELD
The present invention relates to medical imaging systems adapted to facilitate breast cancer screening. In particular, the present invention relates to the manipulation and display of breast ultrasound information in a manner that is convenient, amenable to standardization, and readily usable by radiologists accustomed to traditional x-ray mammogram analysis.
BACKGROUND
Breast cancer is the most common cancer among women other than skin cancer, and is the second leading cause of cancer death in women after lung cancer. The American Cancer Society currently estimates that there are about 184,200 new cases of invasive breast cancer per year among women in the United States and 41,200 deaths per year from the disease. Prevention and early diagnosis of breast cancer are of foremost importance. Because early breast cancer does not produce symptoms, the American Cancer Society recommends a mammogram and a clinical breast examination every year for women over the age of 40.
X-ray mammography is currently the imaging method for mass screening of breast cancer. In health maintenance organizations (HMO's) and other medical organizations, specialized x-ray mammography clinics designed for high patient throughput are being increasingly used to screen as many women as possible in a time and cost efficient manner. X-ray mammography practice in the United States has become largely standardized. For each x-ray mammogram screening of a patient, two standard x-ray mammogram views of each breast are commonly taken: a top (head-to-toe) view ordinarily called the
craniocaudal view ("CC"), and a lateral view ordinarily called the mediolateral oblique view ("MLO"). Several efficiencies arise by virtue of this standardization, including: the ability for examinations to be conducted by x-ray technicians; the ability of a radiologist to analyze mammograrns without being physically present as the x-rays are taken; the ability to compare and statistically track large numbers of x-ray mammograrns taken from different facilities; the ability to track changes in a single patient over time even if the x- ray mammograrns are taken at different facilities; the ability of radiologists to gain recursive expertise in analyzing the standard x-ray mammogram views; and the repeatability of results. The standardization of x-ray mammograrns also yields benefits in the public health care area, including the ability for the U.S. government to provide a fixed and predictable per-mammogram reimbursement for Medicare patients. Additionally, HMO's and other medical insurers are provided with predictable outlays for breast cancer screening of their member patients using x-ray mammography. A well-known shortcoming of x-ray mammography practice, however, is found in the case of dense-breasted women including patients with high content of fibroglandular tissues in their breasts. Because fibroglandular tissues have higher x-ray absorption than the surrounding fatty tissues, portions of breasts with high fibroglandular tissue content are not well penetrated by x-rays and thus the resulting mammograrns would contain little or no information in areas where fibroglandular tissues reside. A recent study by Lehman et. al. on 46,340 patients, published in the December 1999 issue of the American Journal of Reoentgenology (AJR), reports that the proportion of dense breasts (summing those with "heterogeneously dense" and "extremely dense" breasts) account for about 52% of women with age range of 35-39, 47% of age range 40-49, 32% of age range 50-59, 24% of age range 60-69, 23% of age range 70 or older, and 36% for all ages. Lehman et. al., "Effect of Age and Breast Density on Screening Mammograrns with False-Positive Findings," AJR 173:1651-1655 (December 1999). For the estimated 36% of the female population who have dense breasts, this means that at least a portion of the breast area on the x-ray mammogram cannot be scrutinized for lesions by x-ray mammography alone. As a result, lesions camouflaged by dense breast tissue may go undetected. Indeed, a study by Kolb et al on 18,005 consecutive patients, as reported in Jalali, "Sound Combination: Ultrasound
Paired With a Mammography Can Improve Cancer Detection for Dense-Breasted Women," ADVANCE for Administrators In Radiology and Radiation Oncology, pp. 68- 70, states that x-ray mammography were able to detect only 56 cancers of the 80 cancers in 7,202 patients with dense breasts. It has been found that the use of ultrasound mammography (sonomammography) in conjunction with conventional x-ray mammography can drastically increase the early breast cancer detection rate. Ultrasound mammograms display the acoustic impedance of individual slices of breast tissue, and therefore may allow detection of breast lesions where x-ray mammography alone fails. The study by Kolb, supra, revealed this drastic increase. In a study of 18,005 consecutive screening examinations on 10,200 women who had mammograrns, ultrasound exams, and physical examination during a three-year period at Dr. Kolb's New York City office, 7,202 had dense breasts. Among that group, 80 breast cancers were detected in 75 women. Mammography alone detected 56 of the cancers (70 percent), while ultrasound in addition to mammography detected 75 of the cancers (94 percent). The remaining five cancers (6 percent) were found through physical exam. All cancers were confirmed with surgical biopsy. Thus, the use of ultrasound mammography increased the early breast cancer detection rate by 33 percent as compared with x-ray mammography alone.
Vibrational resonance techniques, such as those discussed in U.S. Pats. 5,919,139 and 6,068,597, have been proposed for analyzing suspect tumors. As discussed in Lowers, J., "Experimental Modes Abound For Detecting Breast Cancer: Vibrational Resonance Technique Among the Contenders," Women's Health Supplement to Diagnostic Imaging (April 2001) at pp. 15-17, an audio speaker is attached to the ultrasound probe to introduce audio-range vibrational tones (e.g., 69-247 Hz) into the patient during the acquisition of power Doppler ultrasound frames. Different tissue types often vibrate by different amounts responsive to the acoustic signals, and the different vibrations result in different power Doppler readings. It has been found that the vibrations can help to clearly differentiate suspect lesions, even those that appear isoechoic with surrounding tissue (i.e., are less noticeable) on B-scan ultrasound frames. However, the Lowers reference supra is directed to the analysis of known suspicious lesions rather than to initial breast cancer screening environments.
It has been found that a major inhibitor to the use of ultrasound mammography in breast cancer screening is the difficulty of integrating it into today's specialized, high- throughput x-ray mammography environment. This is especially important in light of the substantial training and expertise built up over several years in physicians reading the x-ray mammograrns and in x-ray technicians conducting standardized x-ray mammography examinations.
An additional major inhibitor to the use of ultrasound in breast cancer screening is the additional physician time that would be required to conduct the ultrasound examination in addition to analyze the ultrasound. In the current health-care environment, physician time is a crucial concern. Additional physician diagnosis time per patient means additional cost. Efficacy of radiological procedures is today measured by the cost in dollars per quality adjusted life year (QALY), with procedures costing more than $100,000 per QALY being neither encouraged nor prescribed. It would be desirable to provide a screening system that integrates ultrasound mammography into current breast cancer screening methodologies without requiring technological upheaval, without requiring substantial additional physician time to conduct the examination and to interpret the ultrasound data, and without losing the benefits of the many years of professional expertise developed in relation to current x-ray mammography methods. Thus, it has been found that perhaps the most valuable and saleable feature of a screening system is ready and efficient usability by today's medical professionals.
Various methods have been proposed for using ultrasound information in breast cancer screening. U.S. Pat. 5,938,613 to Shmulewitz ("the '613 patent"), which is incorporated by reference herein, discusses a method and apparatus for performing sonomammography and enhanced x-ray imaging in which ultrasound equipment is integrated with mammography equipment to generate ultrasonic images of the internal structure of breast tissue that are in geometric registration with an x-ray mammogram. However, the '613 patent is primarily directed to solving registration problems between the ultrasound and x-ray mammograrns, and does not disclose methods for viewing and analyzing the data that are sufficiently suitable for everyday practice by radiologists, and in particular for radiologists that are accustomed to viewing standard CC and MLO views of x-ray mammograrns in their everyday practice. Additionally, significant cost and
complexity are devoted to the registration of the ultrasound data and the x-ray mammography data, whereas it has been found that useful analysis can be performed without requiring such precise registration.
U.S. Pat. 5,984,870 to Giger et al. ("the '870 patent"), which is incorporated by reference herein, discusses a method and apparatus for the computerized automatic analysis of lesions in ultrasound images. Geometric features associated with cancerous conditions may be extracted from an ultrasound mammogram. Features of the lesion from the ultrasound mammogram can then be fed into an artificial neural network together with features of the lesion from an x-ray mammogram and/or an MRI to derive an estimate of the likelihood of malignancy. However, the '870 patent does not disclose a manner of presenting the ultrasound data itself to the radiologist such that effective results checking and ultimate diagnosis can occur. It is believed that a key question relevant to the effectiveness and saleability of an ultrasound mammography system, a question that appears to be regularly avoided by the prior art, is whether today's doctors will want to use the system in everyday screening, and whether today's breast cancer screening clinics will want to buy the system.
It would be desirable to provide a breast cancer screening system that increases early breast cancer detection by readily incorporating ultrasound mammography into today's existing breast cancer screening infrastructure. It would be still further desirable to provide a screening system in which ultrasound mammography results can be integrated with x-ray mammography results without losing the benefits of the many years of professional expertise developed in relation to current standardized x-ray mammography methods.
It would be still further desirable to provide a screening system that takes little or no special familiarization or training from the radiologist in order to effectively view ultrasound information in combination with the x-ray mammogram information.
It would be still further desirable to provide a screening system that provides standardized, repeatable, and easily interpretable ultrasound mammography results that may be quickly analyzed in combination with x-ray mammogram information such that screening cost per patient is not substantially increased.
It would be still further desirable to provide a screening system wherein upon acquisition of the system, any increase in breast cancer screening costs is offset by savings brought about by an increased early breast cancer detection rate, whereby cost per patient QALY is ultimately reduced. It would be even further desirable to provide an improved method of using combinations of x-ray mammography-based and ultrasound-based data in a computer- aided diagnosis (CAD) system.
It would be still further desirable to provide a screening system that provides standardized, repeatable, and easily interpretable ultrasound mammography results and ultrasound computer-aided diagnosis (CAD) results that may be quickly analyzed in combination with x-ray mammogram information such that screening cost per patient is not substantially increased.
SUMMARY In accordance with a preferred embodiment, a screening system is provided having a display apparatus that simultaneously displays a succession of ultrasound image slices, the ultrasound image slices being taken from successive planes in a breast volume substantially parallel to a plane of a predetermined x-ray mammogram view of the breast. The ultrasound image slices are preferably displayed adjacent to a display of the x-ray mammogram view itself, such that a screening radiologist may view the ultrasound image slices simultaneously with the x-ray mammogram view. The display of ultrasound image slices may be used to inspect areas of dense breast tissue that appear opaque on the x-ray mammogram view. Because they are taken from planes in the breast parallel to the x-ray mammogram view plane, the ultrasound image slices are of immediate and familiar significance to the screening radiologist. The breast cancer screening process is thereby made more thorough while at the same time being about as time-efficient as a screening of the x-ray mammogram view alone. Preferably, the predetermined x-ray mammogram view is a standardized x-ray mammogram view such as the craniocaudal view (CC) or mediolateral oblique view (MLO), whereby further advantages of standardization may be realized, including the ability to share, store, and track the ultrasound image slices in a manner similar to the way x-ray mammogram images are shared, stored, and tracked.
According to a preferred embodiment, ultrasound scans are taken of the breast when flattened or compressed along a plane substantially parallel to the x-ray mammogram view plane. The ultrasound scans are taken by an ultrasound probe, which in a preferred embodiment scans the flattened breast in planes perpendicular to the x-ray mammogram view plane. In such case the screening system is adapted to reconstruct a 3 -dimensional volume of the acoustic impedance of the breast, and then interpolate within the 3- dimensional volume to compute the ultrasound image slices along planes substantially parallel to the x-ray mammogram view plane. It is within the scope of the preferred embodiments, however, for the ultrasound probe to scan the flattened breast in planes parallel to the x-ray mammogram view plane, wherein the ultrasound image slices are directly computed from the ultrasound scan data without the need for 3-dimensionsal volume reconstruction.
According to another preferred embodiment, a breast ultrasound adapter is provided for facilitating the reliable acquisition of breast ultrasound scans. The breast ultrasound adapter comprises an open fluid reservoir defined by side walls and a bottom membrane for contacting a breast surface. The breast ultrasound adapter is designed for placement on the breast of a supine patient such that it can be filled with water and/or a water-containing polymeric gel or other suitable fluid until the bottom membrane is submerged. The bottom of an ultrasound probe is submerged in the fluid, and the probe is swept in a lateral direction across the breast, the bottom of the probe remaining submerged during the lateral sweep. Preferably, the bottom of probe is maintained in a plane parallel to the fluid surface and does not contact the bottom membrane during the lateral sweep. Advantageously, the breast is maintained in a substantially consistent position and in a consistent state of compression during the scanning process, thereby facilitating consistency among the ultrasound slices forming the three-dimensional image volume. Water, water-containing polymeric gel, and/or oil are preferably used to improve the contact between the bottom membrane and the breast. Optionally, acoustic transducers such as audio speakers are fixed to the breast ultrasound adapter for use in vibrational resonance applications. In one preferred embodiment, the ultrasound probe is swept by hand across the top surface while position sensors detect its position. Alternatively, a
mechanically driven probe is coupled to the top of the breast ultrasound adapter, the probe position being sensed by mechanical transducers.
According to another preferred embodiment, the screening system performs computer-aided diagnosis (CAD) algorithms on the ultrasound image slices for detecting suspicious locations in the breast. Highlighting marks are superimposed at the suspicious locations of the ultrasound image slices, or alternatively an additional display device is used to display suspicious lesion information using other methods, e.g., using a three- dimensional display method. Optionally, the screening system includes devices that perform CAD algorithms on the x-ray mammogram view as well. The x-ray mammogram, the ultrasound image slices, the ultrasound CAD outputs, and the x-ray mammogram CAD outputs may be quickly displayed to the user in various combinations according to input commands from the user.
According to another preferred embodiment, additional ultrasound information is acquired using vibrational resonance techniques. For each primary ultrasound slice being displayed to the user, one or more additional corresponding vibrational resonance images is superimposed thereon or provided in an additional display. Optionally, the user may activate a toggle switch to cyclically superimpose/remove the vibrational resonance image on/from the primary ultrasound slice. In another preferred embodiment, the vibrational resonance information is provided to the CAD system in conjunction with the x-ray mammogram and/or the primary ultrasound scan data. Vibrational resonance-based features of a lesion are extracted from the vibrational resonance image, and then fed to a classifier, e.g. an artificial neural network, in conjunction with features extracted from the x-ray mammogram and/or primary ultrasound images, for determining a likelihood of malignancy or other metric of suspiciousness.
BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 illustrates steps taken in accordance with a preferred embodiment; FIG. 2 illustrates a diagram of an ultrasound probe scanning a breast that is flattened along an x-ray mammogram view plane in accordance with a preferred embodiment;
FIG. 3 illustrates a conceptual diagram of 3 -dimensional breast volume reconstruction and ultrasound image slice interpolation in accordance with a preferred embodiment;
FIG. 4 illustrates a user display of a breast cancer screening system in accordance with a preferred embodiment;
FIG. 5 illustrates an ultrasound image slice display of the user display of FIG. 4; FIG. 6 illustrates steps taken in accordance with a preferred embodiment; FIG. 7 illustrates a user display of a breast cancer screening apparatus in accordance with a preferred embodiment; FIG. 8 illustrates steps taken in accordance with a preferred embodiment;
FIG. 9 illustrates a user display of a breast cancer screening apparatus in accordance with a preferred embodiment;
FIG. 10 illustrates steps taken in accordance with a preferred embodiment; FIG. 11 illustrates steps taken in accordance with a preferred embodiment; FIGS. 12A and 12B illustrate steps taken in accordance with a preferred embodiment;
FIG. 13 illustrates a breast ultrasound adapter for facilitating breast ultrasound scans in accordance with a preferred embodiment; and
FIG. 14 illustrates a partially automated apparatus for performing breast ultrasound scans in accordance with a preferred embodiment.
DETAILED DESCRIPTION
FIG. 1 illustrates steps taken in accordance with a preferred embodiment in which ultrasound data is used in conjunction with x-ray mammogram data for a more thorough diagnosis as compared to x-ray mammogram data alone. At step 102, an x-ray mammogram view plane is chosen. More than one x-ray mammogram view plane may be chosen, it being understood that the steps of FIG. 1 are to be repeated for each chosen x- ray mammogram view plane. As used herein, an x-ray mammogram view plane is a plane in which x-ray film is placed for receiving x-ray radiation in an x-ray mammogram view. The x-ray mammogram view plane may be described using any numerical criterion sufficient to describe a plane, for example, the x-y-z coordinates of the tip of a unit vector
normal to the plane. For well-known standardized x-ray mammogram views, such as the CC and MLO views, the x-ray mammogram view plane can be identified by a simple reference to that x-ray mammogram view, it being understood that persons skilled in the art would recognize the proper position and orientation of the plane. At step 104, an x-ray mammogram is obtained for the chosen x-ray mammogram view plane. Generally speaking, step 104 is performed using standard x-ray mammography methods at a breast cancer screening clinic. Alternatively, in the case where the steps of FIG. 1 are being carried out several hours, days, or weeks after all the clinical measurements are taken, step 104 is performed by accessing the appropriate database or accessing the appropriate physical x-ray film containing the x-ray mammogram for the chosen x-ray mammogram view plane. At step 104, the x-ray mammogram is typically a x-ray film acquired with a film-screen system. However, the x- ray mammogram may also be acquired by a digital detector, such as that used in a digital mammography system, in the form of a digital mammogram. At step 106, the breast is flattened along an ultrasound-absorbing plate placed in the chosen x-ray mammogram view plane, and at step 108 ultrasound scan data on the flattened breast is obtained. At step 110, a three-dimensional volume reconstruction is performed using the ultrasound scan data to form a three-dimensional acoustic impedance map of the breast. At step 112, ultrasound image slices are interpolated along planes parallel to the chosen x-ray mammogram view plane using the three-dimensional acoustic impedance map. Finally, at step 114 the x-ray mammogram and the ultrasound image slices for the chosen x-ray mammogram view plane are simultaneously displayed for viewing by the screening radiologist.
Importantly, it is to be appreciated that step 106, together with subsequent step 108, is not required to be performed at the same time, on similarly calibrated equipment, or even at the same screening facility as step 104. Indeed, in many instances it may be more convenient to perform examinations on separate machines or in separate facilities to increase patient throughput. Advantageously, in accordance with the preferred embodiments, precise registration between the ultrasound image slices and the corresponding x-ray mammogram views is not required. It is only preferred that the breast be flattened against a plane substantially parallel to the x-ray mammogram view plane, so
that when ultrasound image slices are derived from the ultrasound scan data, they will generally correspond to the chosen x-ray mammogram view.
FIG. 2 illustrates a diagram corresponding to steps 106 and 108 of FIG. 1, wherein an ultrasound probe 202 scans a breast 204 that is flattened along an ultrasound-absorbing plate 206. The ultrasound-absorbing plate 206 lies in the chosen x-ray mammogram view plane, which for the example of FIG. 2 is the x-y plane in the axis system shown. Ultrasound scan data is obtained using methods known in the art. The breast 204 may be flattened either by hand, or may be flattened by a planar device that is integral with an ultrasound probe, such as that described in U.S. Pat. 5,938,613, supra. In the case of a planar ultrasound probe, an impedance-matching ultrasound gel or gel bag should be placed between the breast and the probe for best results. Because typical hand-held ultrasound probes are about 38 mm wide, it may be desirable to incorporate two or more parallel probes to cover the breast in one scan. Or alternatively, an automatic raster scan mechanism may be used to cover the breast. The dynamic range of the ultrasound data samples is chosen such that approximately 8 bits of gray scale resolution is yielded in the ultrasound image slices.
Also shown in FIG. 2 are fixed sensors 208 and 210, which together with an electromagnetic transmitter embedded into ultrasound probe 202 form an electromagnetic position sensing system. The electromagnetic position sensing system provides the necessary mathematical link between the acoustic reflection data obtained by the ultrasound probe 202 and the corresponding position within the breast 204. Suitable electromagnetic sensing systems that may be used in accordance with the preferred embodiments are described in Cheng et al, "Automated Detection of Breast Tumors in Ultrasonic Images Using Fuzzy Reasoning," Proceedings of the IEEE Computer Society International Conference on Image Processing Volume III, pp. 420-423 (October 26-29, 1997) (hereinafter "Cheng"), the contents of which are hereby incorporated by reference. Other types of ultrasound probe position sensing systems are known in the art, and any of a variety of such systems may be used in accordance with the preferred embodiments, including servo-mechanical sensing systems (see, e.g., U.S. Pat. 5,433,202 to Mitchell et. al., the contents of which are hereby incorporated by reference) and fixed plate scanning systems (see, e.g., U.S. Pat. 5,640,956 to Getzinger et. al., the contents of which are hereby
incorporated by reference). Alternatively, inertial-type of positional sensors, such as those used in the Hewlett Packard CapShare 920 Portable E-Copier hand-held page scanner, may also be used.
Preferably, the ultrasound scans taken by ultrasound probe 202 are taken in planes that are normal to the chosen x-ray mammogram view plane (the x-y plane in FIG. 2). Thus, in FIG. 2 the ultrasound scans are taken for a series of planes parallel to the x-z plane, which is normal to x-y plane, resulting in a succession of two-dimensional ultrasound planes denoted herein as n;, where "i" is a counter variable for the successive ultrasound planes. It is preferable to take the ultrasound scans along planes normal to the x-ray mammogram view plane because the resulting scans are normal to the flattened breast and therefore are of a shallow depth compared to ultrasound scans taken parallel to the x-ray mammogram view plane. Because of the shallower depth, greater ultrasound image resolution can be obtained. As a typical numerical example, ultrasound scans taken normal to a flattened breast would only need to penetrate about 4 cm of breast tissue, whereas ultrasound scans taken parallel to the plane of the flattened breast would need to penetrate about 15 cm of breast tissue. Due to acoustic attenuation in the human body that increases with frequency (a factor of about 0.6 dB/MHz-cm), lower frequency probes operating near 3 MHz would be required for taking ultrasound scans parallel to the plane of the flattened breast, with the use of lower probe frequency yielding a lower spatial resolution of about 0.1 cm. In contrast, higher frequency probes operating at about 7.5 MHz can be used for talcing ultrasound scans normal to the flattened breast, with the use of the higher probe frequency yielding a better spatial resolution of about 0.05 cm.
However, although it is preferable ultrasound probe 202 scan in a direction normal to the chosen x-ray mammogram view plane, it is nevertheless within the scope of the preferred embodiments for scanning to occur parallel to the x-ray mammogram view plane in certain circumstances. This is because even though the resolution is diminished, one advantage of scanning parallel to the x-ray mammogram view plane is that less processing time is required to generate the ultrasound image slices, which are generally directly yielded from the ultrasound scans themselves. When the ultrasound scans are taken by the ultrasound probe 202 parallel to the x-ray mammogram view plane, the ultrasound image slices may be directly computed without the need for performing the three-dimensional
reconstruction and interpolation at step 112. Because the lower end of the target size lesions is usually about 0.5 cm, the lesser spatial resolution of 0.1 cm associated with the lower frequency probe may be sufficient in some clinical environments.
FIG. 3 illustrates a diagram corresponding to steps 110 and 112 of FIG. 1, in particular a conceptual diagram of 3 -dimensional breast volume reconstruction and ultrasound image slice interpolation in accordance with a preferred embodiment. FIG. 3 conceptually shows a 3 -dimensional acoustic impedance matrix 302 constructed from the plurality of two-dimensional ultrasound planes ιij resulting from step 108. The 3- dimensional acoustic impedance matrix 302 may be constructed from the ultrasound planes nj using any of a variety of known 3-dimensional volume formation algoritlims. FIG. 3 also conceptually shows a representative ultrasound image slice 304 resulting from the application of an interpolation algorithm to the 3-dimensional acoustic impedance matrix 302, the ultrasound image slice 304 being substantially parallel to the x-ray mammogram view plane (e.g., the x-y axis in FIGS. 2 and 3), the ultrasound image slices being denoted herein by n j, where j is a counter variable for the successive ultrasound image slices. The successive ultrasound image slices m.j may be computed using any of a variety of known 3- to-2 dimensional projection algorithms. Depending on the interpolation/projection algorithm used, each ultrasound image slice may represent instantaneous values along a single plane within the 3-dimensional acoustic impedance matrix. Alternatively, each ultrasound image slice may be a "thick" one representing an integration of the 3- dimensional acoustic impedance matrix over a predetermined thickness (e.g., 0.5 cm). The latter algorithm is usually preferable because more data within the 3-dimensional acoustic impedance matrix is used to compute the actual result, and less data is "discarded" by the projection algorithm. In accordance with a preferred embodiment, the successive ultrasound image slices m.j correspond to planes in the breast volume that are no greater than a predetermined spacing distance apart. The predetermined spacing distance is selected to reduce the probability of missed lesions due to undersampling along an axis perpendicular to the x-ray mammogram view plane (e.g. , the x-axis in FIGS. 2 and 3). At the same time, however, the predetermined spacing distance should not be so small as to cause the screening radiologist to view an untenably large number of ultrasound image slices. For one
preferred embodiment, a predetermined spacing distance of approximately 0.5 cm is suitable for many applications. However, it is also anticipated that some physicians may prefer higher spatial resolution (more slices with spacing less than 0.5 cm) and some may prefer lower spatial resolution (fewer slices with spacing more than 0.5 cm). In these cases, more or fewer slices will be provided to suit the physician's taste.
FIG. 4 illustrates a user display 400 of a breast cancer screening system in accordance with a preferred embodiment. User display 400 comprises an x-ray mammogram display 402 for showing a chosen x-ray mammogram view 404. The x-ray mammogram display 402 may comprise a TV monitor for displaying a digitized form of the x-ray mammogram view, or alternatively may comprise a mechanical display such as a conveyor-based system and light box for displaying x-ray mammogram film. User display 400 further comprises an ultrasound slice display 406 for simultaneously displaying a film containing a plurality of ultrasound image slices nij adjacent to one another, the display 406 being positioned such that it may be viewed simultaneously with the x-ray mammogram display 402. The ultrasound slice display 406 of FIG. 4 shows a display of eight ultrasound image slices m.j, although as few as four slices and as many as 16 or more slices may be displayed depending on screening radiologist desires. When displaying a digital mammogram on a TV monitor, it is sometimes preferred to also display the ultrasound slices on the same or an adjacent TV monitor. Or, while the x-ray mammogram may be in the film form, the ultrasound slices be displayed on an adjacent TV monitor. Alternatively, while the x-ray mammogram may be displayed on a TV monitor, the ultrasound slices be displayed on a film on an adjacent light box.
User display 400 may further comprise an optional keyboard 408, an optional mouse 410, and a plurality of optional control knobs/switches 412 for controlling the brightness, contrast, zoom, and other functions that may be installed with the user display 400. An example of an x-ray mammogram display apparatus that may be used in conjunction with the preferred embodiments can be found in U.S. Pat. 5,917,929 to Marshall et. al.., the contents of which are hereby incorporated by reference.
The user display 400, and in particular the ultrasound slice display 406, may be advantageously used by the screening radiologist to quickly determine if there are problematic areas of dense breast tissue that would otherwise be camouflaged in the x-ray
mammogram view 404. A conceptual diagram of this is shown in FIG. 4, in which a spherically-shaped lesion 414 does not show up on the x-ray mammogram view 404 because it is camouflaged by an area of dense fibroglandular breast tissue 416. The lesion 414 is shown as a dotted line in x-ray mammogram view 404 to signify that the radiologist cannot see it. In accordance with a preferred embodiment, however, the lesion 414 does indeed show up among the plurality of ultrasound image slices my in ultrasound slice display 406, appearing in three of the ultrasound image slices as areas marked 414a, 414b, and 414c in FIG. 4. Thus, the use of the ultrasound slice display 406 in ready conjunction with the x-ray mammogram view 404 can avoid missed diagnoses due to camouflage of lesions within dense breast tissue.
Conversely, another advantage in accordance with the preferred embodiments can avoid increased false positive screening results due to dense breasts. In particular, (see Lehman et. al., supra), dense breasts typically lower the specificity (increased false positives) of x-ray screening examinations. A lesion may be obscured by the dense fibroglandular tissue or an apparent lesion on the x-ray mammogram view may in fact be due to a coincidental alignment of unrelated breast tissue that appears as a single lesion in the x-ray mammogram view. Such occurrences would readily be revealed by a quick examination of the ultrasound image slices, wherein the screening radiologist would quickly determine that there is no single lesion in the ultrasound image slices but rather a plurality of unrelated tissue segments. Especially in the circumstance in which the chosen x-ray mammogram view is a standardized x-ray mammogram view such as the CC or MLO view, the above diagnoses can be quickly and readily made because the screening radiologist already knows and is familiar with the orientations of the ultrasound image slices. FIG. 5 illustrates a more detailed view of the ultrasound image slice display 406 of the user display of FIG. 4. The medium of the slice display 406 may be any of a variety of display media sufficient to display ultrasound image slice information, such as a TV monitor or a film on a light box. Preferably, the ultrasound image slice display shows a minimum amount of information necessary for the screening radiologist to readily understand the significance of the ultrasound image slices being displayed. Thus, for example, ultrasound image slice display 406 comprises a legend 502 showing the chosen
x-ray mammogram view information ("CC" in the example of FIG. 5), as well as information on the spacing of the ultrasound image slices. The spacing of the ultrasound image slices is of particular importance to the screening radiologist, as it raises awareness of whether a "complete" picture of closely spaced slices is being shown, or whether there is undersampling due to large spaces between the planes.
FIG. 6 illustrates steps taken by a breast cancer screening system in accordance with another preferred embodiment, in which ultrasound data is used to generate ultrasound image slices for display to a screening radiologist, who is familiar with the standard x-ray mammogram views but without the aid of corresponding x-ray mammogram views being shown. At step 602, a standardized x-ray mammogram view plane is chosen, e.g., the CC or MLO view. Preferably, a breast cancer screening system that only shows ultrasound image slice information should show ultrasound image slices corresponding to standardized x-ray mammogram views in order to present familiar orientations to the screening radiologist. Subsequent to step 602, steps 604, 606, 608, and 610 are performed in a manner similar to the steps 106, 108, 110, and 112 of FIG. 1, respectively. Finally, at step 612 the ultrasound image slices for the chosen standard x-ray mammogram view plane are displayed to the screening radiologist.
FIG. 7 illustrates a user display 700 for carrying out step 612 of FIG. 6. User display 700 comprises an ultrasound slice display 706 for simultaneously displaying a plurality of ultrasound image slices m.j adjacent to one another, and additionally comprises elements 708, 710, and 712 similar to elements 408, 410, and 412 of FIG. 4. The ultrasound slice display 706 further comprises a legend 714 similar to the legend 502 of FIG. 5. The embodiments of FIGS. 6 and 7 are advantageous in that they are lower-cost and less complex than the embodiments of FIGS. 1 and 4, and may be of potential use in very low-cost, integrated, portable ultrasound systems for mass breast cancer screening in impoverished areas of the world. Alternatively, the embodiments of FIGS. 6 and 7 may be of use if advances in ultrasound technology yield ultrasound image slice outputs that are of equal or superior resolution than x-ray mammograms. In general, however, it will usually be preferable to combine the ultrasound image slice outputs with x-ray mammogram outputs in accordance with the embodiments of FIGS. 1 and 4, especially since microcalcifications are not generally detectable using known ultrasound methods, in which
case it will be desirable to obtain and analyze x-ray mammogram data in any case to detect microcalcifications .
FIG. 8 illustrates steps taken by a breast cancer screening system in accordance with another preferred embodiment, in which computer aided diagnosis (CAD) algorithms are performed on the ultrasound image slices and/or the x-ray mammogram view and the results are flexibly displayed to the screening radiologist. For example, in the embodiment of FIG. 8, steps 802-812 are carried out in a manner similar to steps 102-112 of FIG. 1, respectively. Prior to a display step, however, a step 813 for performing computer aided diagnosis (CAD) algorithms on the ultrasound image slices mj is carried out. Optionally, a step 815 for performing CAD algorithms on the chosen x-ray mammogram view is carried out. Finally, at step 817 the x-ray mammogram view, the ultrasound image slices, and the ultrasound image slice CAD results and/or the x-ray mammogram view CAD results are displayed to the screening radiologist.
FIG. 9 illustrates a user display 900 for carrying out step 817 of FIG. 8, the user display 900 comprising elements 902-912 similar to elements 402-412 of FIG. 4, respectively. User display 900 also includes, however, a CAD result display 914 for displaying the ultrasound image slice CAD results and/or the x-ray mammogram view CAD results. In accordance with a preferred embodiment, CAD result display 914 displays the ultrasound image slice CAD results until the user presses a toggle control 916, wherein the CAD result display 914 then displays the x-ray mammogram view CAD results, and vice versa. CAD result display 914 is designed to highlight areas of suspicion in the medical image and draw them to the attention of the screening radiologist, who may then more closely examine the original medical image. As a nonlimiting example, FIG. 9 shows highlighting arrows 918 to draw the attention of the radiologist toward the suspicious lesions appearing in the ultrasound image slices.
Any of a variety of ultrasound CAD algorithms can be used at step 813 depending on the desired clinical application including, but not limited to, algoritlims disclosed in Chen et. al., "Computer-aided Diagnosis Applied to Ultrasound of Solid Breast Nodules by Using Neural Networks," Radiology, pp. 407-412 (November 1999)^ Cheng et. al., "Automated Detection of Breast Tumors in Ultrasonic Images Using Fuzzy Reasoning," supra, and U.S. Pat. 5,984,870 to Giger et. al., supra, the contents of each of these
references hereby being incorporated by reference. Also, any of a variety of x-ray mammogram CAD algorithms can be used at step 815 depending on the desired clinical application including, but not limited to, algoritlims disclosed in U.S. Pat. 5,815,591 to Roehrig et. al., the contents of which are hereby incorporated by reference. FIG. 10 illustrates steps taken by a breast cancer screening system in accordance with another preferred embodiment in which vibrational resonance information is flexibly displayed to the screening radiologist. At step 1002 the primary ultrasound scan data is obtained. As used herein, primary ultrasound data refers to any type of ultrasound data acquired while the breast is not being acoustically vibrated by an external transducer. The primary ultrasound data may generally be of any type including B-mode, color Doppler, power Doppler, and other types. At step 1004, a three-dimensional volume reconstruction of the breast is performed using the primary ultrasound scan data. At step 1006, vibrational resonance ultrasound scan data is acquired in a manner similar to that described in Lowers, supra, by introducing acoustic vibrations into the breast, allowing the acoustic vibrations to stabilize, and talcing power Doppler ultrasound scans. At step 1008, the power Doppler image slices are computed. At step 1010, a 3-D volume reconstruction of the breast is performed using the vibrational resonance information. As used herein, vibrational resonance information refers to ultrasound scan data taken while the breast is acoustically vibrated; vibrational resonance image slice refers to an ultrasound image slice derived from the vibrational resonance information; vibrational resonance volume refers to a 3-D volume reconstruction computed from the vibrational resonance image slices; and vibrational resonance image refers to refers to either a vibrational resonance image slices or to an ultrasound image derived from the vibrational resonance volume. The spatial resolution of the vibrational resonance information, and the volumes/images derived therefrom, will usually be less than the spatial resolution of the primary ultrasound information due to the greater amount of processing required for power Doppler images, although the scope of the preferred embodiments is not so limited.
At step 1012, the choice of x-ray mammogram view plane is received (e.g., CC or MLO) and at step 1014 the corresponding mammogram is obtained. Importantly, the steps 1002, 1006, and 1012 may be carried out in any order and may occur at substantially different times. At steps 1016 and 1018, respectively, primary and vibrational resonance
images are computed from their corresponding 3-D image volumes along planes parallel to the chosen x-ray mammogram plane. At step 1020, the x-ray mammogram and primary ultrasound image slices are displayed to the user using methods described supra. At step 1022, to obtain a more thorough view of a given lesion or feature, the user may activate a simple toggle switch or equivalent keyboard or mouse entry that allows the vibrational resonance images to be superimposed upon the primary ultrasound images. The user may easily toggle between "superimposed" and "non-superimposed" states during the examination. In this way, vibrational resonance information easily and intuitively integrated with the primary ultrasound display in a manner readily usable by the screening radiologist or other medical professional.
It has been found that different amounts of tissue type differentiation may be acquired for different vibrational frequencies and/or amplitudes. According to a preferred embodiment, "N" distinct sets of vibrational resonance information are acquired for different acoustic vibration frequencies and/or amplitudes. For example, a first set may be acquired for an acoustic frequency of 100 Hz at 20 watts of acoustic power, a second set for 200 Hz at 20 watts, a third set for 200 Hz at 40 watts, and so on. Each of the "N" data sets yields its own vibrational resonance volume and its own set of vibrational resonance images, which may be separately displayed to the user according to the methods described herein. Moreover, lesion features may be extracted from each of the "N" vibrational resonance data sets and provided to the CAD classifiers, e.g. , artificial neural networks described herein for classifying the lesions. Advantageously, providing "N" different vibrational resonance data sets to the classifier algorithms accommodates the possibility of locating different lesion types that may respond differently to different vibrational frequencies and amplitudes. Thus, it is to be appreciated that while described herein in terms of only a single set of vibrational resonance data for simplicity and clarity of explanation, the scope of the preferred embodiments includes the acquisition and use of "N" sets of vibrational resonance data in data display and CAD algoritlims.
FIG. 11 illustrates steps taken by a breast cancer screening system in accordance with another preferred embodiment, in which computer aided diagnosis (CAD) algorithms are performed using the primary ultrasound image data, the x-ray maimnogram data, and the vibrational resonance information and the results are flexibly displayed to the screening
radiologist. Steps 1102-1114 are carried out in a manner similar to steps 1002-1014 of FIG. 10, respectively. Prior to a display step, however, a step 1116 for performing computer aided diagnosis (CAD) algorithms using the primary ultrasound image data, the x-ray mammogram data, and the vibrational resonance information. Finally, at step 1118 the x-ray mammogram view, the primary ultrasound image slices, the vibrational resonance images, and the CAD results are displayed to the screening radiologist in any of a variety of combinations. In particular, the CAD results may be superimposed on any of the above medical images, or any combination thereof, in accordance with a preferred embodiment. The various steps of FIG. 11 may be carried out in different orders, may occur at substantially different times, and may be carried out by different, physically separated hardware systems.
FIG. 12A illustrates steps for computer-aided diagnosis in accordance with a preferred embodiment corresponding to step 1116 of FIG. 11. At steps 1202, 1204, and 1206, respectively, the initial data for the x-ray mammogram information, primary ultrasound information, and vibrational resonance information is read into the CAD system. At steps 1208, 1210, and 1212, respectively, lesions in the patient are separately localized, i.e., areas containing possible lesions are initially identified, using the respective x-ray, primary ultrasound, and vibrational resonance data. At step 1214, lists of localized lesions are compared and adjusted as necessary. Advantageously, the use of vibrational resonance data may result in localization of lesions that are initially missed by the primary ultrasound localization algorithms because, as discussed supra, some tumors may appear isoechoic with surrounding tissue on the primary ultrasound scans, but may appear hyperechoic or hypoechoic in the vibrational resonance images. Most commonly, the list of localized lesions will simply comprise the logical union of lesions localized at steps 1208-1212, although the scope of the preferred embodiments is not so limited.
At steps 1216, 1218, and 1220, respectively, features of the lesions are separately extracted, using the respective x-ray, primary ultrasound, and vibrational resonance data. The various features extracted from the x-ray mammogram and the primary ultrasound data include those discussed in Giger, supra, including gray-level based features, geometric features, gradient features, and other features. Features extracted from the vibrational resonance data include a first set similar to those extracted from the primary
ultrasound data, and a second set based on vibrational resonance parameters. According to a preferred embodiment, this second set of features includes differential gray scale metrics, differential geometric feature metrics, and differential gradient metrics measured versus changes in vibrational resonance frequency and vibrational resonance power. Differential metrics further include differences in a given metric between a vibrational resonance image and the primary ultrasound image (for which the vibrational resonance frequency and vibrational resonance power are both equal to zero).
By way of example, gray-level based features (which for ultrasonic data refers to acoustic impedance amplitude, power Doppler amplitude, etc.) include: average level within a region of interest (ROI) or lesion; standard deviation of the levels within a ROI or lesion; minimum level within a ROI or lesion; maximum level within a ROI or lesion; minimum 5% level corresponding to the level below which yields 5% of the area under the level histogram; maximum 5% level corresponding to the level above which yields 5% of the area under the level histogram; and contrast as given by the difference between levels within the lesion relative to those in the surrounding area. Geometric features include, for example, ratio of lesion surface area to lesion volume.
By way of example, differential gray scale metrics include: a ratio of primary ultrasound image contrast for the lesion versus vibrational resonance image contrast for the lesion; ratio of standard deviation in the lesion for the primary ultrasound image versus standard deviation in the lesion for the vibrational resonance image; first and second derivatives of vibrational resonance image contrast versus vibrational resonance frequency; first and second derivatives of vibrational resonance image contrast versus vibrational resonance power; and any of a variety of metrics that compare (i) the primary image versus a vibrational resonance image, or (ii) different vibrational resonance images at different vibrational resonance frequencies and powers. It is to be appreciated that differential metrics based on the primary image versus the vibrational resonance image will require communication of results between hardware carrying out steps 1218 and 1220. Differential geometric features include, for example, the surface area/volume ratio for the ultrasound image divided by the surface area/volume ratio for the vibrational resonance image.
At step 1222, classifier algorithms, e.g. artificial neural network algorithms, are executed on the features extracted at steps 1216-1220, and at step 1224 a degree of user attention required for that feature (e.g. in the form of a highlighting symbol type, symbol brightness, etc.) is determined. Finally, at step 1226 the CAD results are output for display in accordance with step 1118 of FIG. 11 , supra.
FIG. 12B illustrates steps for computer-aided diagnosis in accordance with an alternative preferred embodiment corresponding to step 1116 of FIG. 11, wherein CAD algorithms on the x-ray mammogram information are carried out independently of the ultrasound/vibrational resonance CAD algoritlims. As indicated in FIG. 12B, steps 1202- 1212 are carried out in the same manner as for FIG. 12 A. However, beginning at step 1214A, the x-ray mammogram CAD algorithm steps are carried out independently, while the ultrasound/vibrational resonance CAD algorithms are still executed together. Thus, at step 1214A, only lists of localized lesions from the ultrasound and vibrational resonance images are compared and adjusted as necessary. Feature extraction step 1216A, classification step 1222 A, user attention determining step 1224 A are separately executed on the x-ray mammogram data without using any data from the ultrasound/vibrational resonance CAD algoritlims. At step 1226A, the x-ray mammogram CAD results are output separately for display to the user. Likewise steps 1218 A, 1220 A, and 1222B-1226B are carried out without using x-ray mammogram data or CAD results. Accordingly, in the preferred embodiment of FIG. 12B, the first integration between the x-ray mammogram CAD results and the ultrasound/vibrational resonance CAD results is performed by the user when simultaneously viewing the respective displays.
FIG. 13 illustrates a breast ultrasound adapter 1300 for facilitating breast ultrasound scans in accordance with a preferred embodiment. As known in the art, it is required that a close acoustical coupling be maintained between the breast skin surface and the ultrasound probe at all times during the scan. In particular, there should be no vacuum or air pockets anywhere along the path of the interrogating ultrasound pulses. This is usually achieved by keeping the breast surface wet with a water-containing polymeric fluid or gel at all points of contact with the ultrasound probe. Alternatively, gel bags have been placed between the ultrasound probe and skin surface to maintain an acoustic coupling.
With both of these methods, however, the breast itself is necessarily shifted around and compressed by differing amounts at differing places as the ultrasound probe is moved.
Breast ultrasound adapter 1300 comprises an upper frame 1302, side walls 1306, and a bottom membrane 1310 forming a fluid reservoir volume 1308 for holding a water- containing polymeric fluid or gel. As shown in FIG. 13, breast ultrasound adapter 1300 has been placed over a breast 1320 and filled with fluid 1311, and the bottom membrane 1310 has conformally adapted to the shape of the breast 1320. The fluid level should rise above the highest point of the breast 1320. The bottom of an ultrasound probe 1314 is submerged in the fluid 131 land is swept in a lateral direction across the breast as indicated in FIG. 13, the bottom of the ultrasound probe 1314 remaining submerged during the lateral sweep. Preferably, the bottom of ultrasound probe 1314 is maintained in a plane parallel to the fluid surface and does not contact the bottom membrane 1310 during the lateral sweep. Position sensors (not shown) are used to track the position of the ultrasound probe 1314 as it is swept, usually by hand, across the fluid surface. Preferably, water- containing polymeric gel, water, and/or oil are used to improve the contact between the bottom membrane 1310 and the breast 1320. Advantageously, the breast is maintained in a substantially consistent position and in a consistent state of compression during the scanning process, thereby facilitating consistency among the ultrasound slices forming the three-dimensional image volume. The breast ultrasound adapter 1300 may comprise any of a variety of material configurations that facilitate the presence of an open fluid reservoir above the breast of a supine patient, the fluid reservoir having a lower surface that conforms to the breast shape such that an ultrasound probe is acoustically coupled to the breast skin surface when immersed in the fluid. By way of example, upper frame 1302 is preferably a rigid or semi- rigid compression-molded silicone rubber material, or an equivalent material, such that the breast ultrasound adapter 1300 can be supported and moved by manipulating the upper frame 1302 even when full of fluid. Side walls 1306 may also be rigid or semi-rigid. In the embodiment of FIG. 13, side walls 1306 are semi-rigid and sufficiently flexible such that they are conformal with the patient's skin surface along points of intersection 1304 therewith. However, in alternative preferred embodiments, the side walls 1306 do not contact the skin surface directly and therefore may be rigid. In this case, the bottom
membrane protrudes downward from the side walls and hangs down like a plastic bag over the breast when filled with fluid. In still other preferred embodiments, the side walls 1306 are not present at all, and the bottom membrane hangs down directly from the upper frame 1302 over the breast when filled with fluid. In preferred embodiments in which the side walls 1306 do not contact the skin surface directly or are not present, the breast ultrasound adapter 1300 is supported by the upper frame 1302 and laid gently over the breast, which does not support its entire weight. The bottom membrane 1310 preferably comprises a flexible, watertight, conforming material. An at least partially distensible characteristic assists in ensuring bubble-free contact with the breast skin. Suitable materials include protective latex, synthetic elastomers, cellophane, or other protective sheath-type materials described in U.S. Pat. 6,039,694, which is incorporated by reference herein. The respective pieces are fastened together by a suitable adhesive such as silicone RTV.
FIG. 14 illustrates a breast ultrasound adapter 1402 having additional features in accordance with a preferred embodiment that, in conjunction with a mechanical probe translation assembly, forms a semi-automatic breast ultrasound scanning device 1400. Breast ultrasound adapter 1402 comprises an integrated audio speaker 1410 coupled to an electrical source 1412 for use in vibrational resonance applications. The apparatus of FIG. 14 also comprises a conceptual diagram of an ultrasound probe 1404 as it is guided by a mechanical translator 1406, for example by means of a slot 1408. Advantageously, the probe position may be sensed by mechanical transducers instead of position sensors as required in the embodiment of FIG. 13, supra. The mechanical translator may be affixed to the frame of the breast ultrasound adapter 1402, or may alternatively be externally supported.
Finally, breast ultrasound adapter 1402 further comprises a fluid transport opening and fluid conduit 1414 for coupling to a gravity-feed reservoir (not shown) that is functionally similar to an intravenous (IV) fluid container. In accordance with a preferred embodiment, the gravity-feed reservoir is raised, either by hand or by an automated mechanical assembly, to introduce fluid into the breast ultrasound adapter 1402 after it has been placed on a patient's breast. Following the scan procedure, the gravity-feed reservoir is lowered to cause fluid to drain out, thereby emptying the breast ultrasound adapter 1402.
The vertical position of the gravity-feed reservoir may also be adjusted used to regulate the level of the fluid during the scan.
Whereas many alterations and modifications of the present invention will no doubt become apparent to a person of ordinary skill in the art after having read the foregoing description, it is to be understood that the particular embodiments shown and described by way of illustration are in no way intended to be considered limiting. By way of example, while vibrational resonance images are described supra as comprising power Doppler images, in other preferred embodiments the vibrational resonance images may comprise color Doppler images or other ultrasound image types taken while the breast is being vibrated. By way of further example, while the breast ultrasound adapter described supra covers a single breast, in other preferred embodiments it is configured to cover both breasts simultaneously, thereby further expediting the ultrasound scanning process. Therefore, reference to the details of the preferred embodiments are not intended to limit their scope, which is limited only by the scope of the claims set forth below.