WO2002015113A2

WO2002015113A2 - Mammography screening to detect and classify microcalcifications

Info

Publication number: WO2002015113A2
Application number: PCT/US2001/025421
Authority: WO
Inventors: Chein-I Chang; Chien-Shun Lo; Pau-Choo Chung; San Kan Lee; Ching-Wen Yang
Original assignee: University Of Maryland, Baltimore County
Priority date: 2000-08-14
Filing date: 2001-08-13
Publication date: 2002-02-21
Also published as: WO2002015113A3; AU2001286469A1

Abstract

A novel system for processing mammography images to detect and classify microcalcifications and other abnormalities is provided in which a computer-assisted method and system allows real-time processing of mammography screening to detect and classify microcalcifications based on their likelihood of malignancy. The method extracts the breast region from digitized mammograms (100), finds suspicious areas within this region (200), enhances certain characteristics of the image in a specified region of interest (300), and classifies the microcalcifications based on the geometric shape of the enhanced image (400).

Description

TITLE: Computer Assisted Marnmography Screening to Detect and Classify

Microcalcifications

FIELD AND BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates in general to image analysis and more particularly to computer- assisted real-time processing of marnmography images to detect and classify microcalcifications.

Background Information

The invention described and claimed herein comprises a novel system for processing marnmography images to detect and classify microcalcifications and other abnormalities

Marnmography is currently the most effective screening technique capable of detecting breast cancer at an early stage. Breast cancer mortality rates decrease significantly when using mammograms to detect cancer at an early stage. (Zhou, X. and Gordon, R., Critical Review in Biomedical Engineering: "Detection of early breast cancer: an overview and future prospects, 1989). A 1987 study revealed that the five-year survival rate of women who had used mammograms to discover tumors at an early stage was 82% compared to the control group survival rate was 60%. The study also showed, however, that the accuracy of these mammograms needs to be improved. Eight out of ten masses detected by marnmography turn out to be false alarms and as many as 9% of actual tumors are missed. Missed detections can be attributed to several factors, a major one being the constraint placed on radiologists to review large volumes of mammograms produced for a small number of occasional abnormalities. Under this constraint and with a shortage of trained radiologists, detections may be missed as a result of eye fatigue, oversight, or difficulties in maintaining interest and concentration when viewing large volumes of mammograms.

Microcalcifications present an early sign of breast cancer. On screening studies, 90% of nonpaipable in situ ductal carcinomas and 70% of nonpaipable minimal carcinomas are visible on microcalcifications alone. As a result, detecting nonpaipable malignant calcifications with the breast before they become metastasized is of great importance.

Calcifications, however, are the smallest structures identified on mammograms. They are tiny, sometimes clustered particles, that are most easily seen using high-resolution imaging techniques or direct radiological magnification. In order, to assist radiologists in detecting such microcalcifications, developing reliable computer aided diagnostic (CAD) systems for microcalcifications detection is highly desirable. Although early reports from 1960s suggested that clustered microcalcifications associated with benignancy and malignancy usually have distinct characteristics, more recent studies in the 1980s involving a large number of cases indicated that these characteristics do considerably overlap. As a result, most radiologists encourage biopsies, even when only 20%-30% of cases proved to be cancer [E.A. Sickles, Breast calcifications: mammographic evaluation, Radiology 160 (1986) 289-293]. It is a challenge for radiologists not only to recognize the presence of these tiny particles, but also to assess the likelihood of malignancy in order to avoid unnecessary biopsies [M. Lanyi, Microcalcifications in the breast-a blessing or a curse?, Diagnostic Imag. Clin. Med. 54 (1985) 126-145]. Therefore, an important issue is how to reduce the false-positive biopsy rate for mammographically detected abnormalities such that the number of mammography-generated biopsy for benign lesions can be substantially decreased.

Thus, correct image analysis of calcifications in mammograms will decrease the biopsy rate in situations where a mammogram reveals a calcification. (Giger, 1993; Vyborny et al., 1994; Adler et al, 1995). Although radiologists use considerable interpretive expertise as well as fine- resolution images to rate the need to biopsy for calcifications that are likely to be malignant, difficulties in interpretation primarily arise where both benign and malignant lesions have similar mammographic appearances. A computer-aided system, such as this invention, may provide not only a means for detecting regions of interest that could be missed by a human interpreter but also may provide a second opinion to assist the radiologist, particularly those who are inexperienced, to diagnose the calcification and also may be used as a training tool.

Many CAD algorithms and methods have been proposed for detection and segmentation of microcalcifications, such as global and local thresholding techniques, artificial neural network and wavelet approaches. A sample of such techniques are reported in:

B.W. Fam, S.L. Olson, P.F. Winter, F.J. Scholz, Algorithm for the detection of fine clustered calcifications on film mammograms, Radiology 169 (1988) 333- 337.

D.H. Davies, D.E. Dance, Automatic computer detection of clusters of calcifications in digital mammograms, Phys. Med. Biol. 35 (1990) 1111-1118. H.P. Chan, K. Doi, S. Galhotra, C.J. Vyborny, H. MacMahon, P.M. Jokich, Image feature analysis and computer-aided diagnosis in digital radiology. I. Automated detection of microcalcifications in marnmography, Med. Phy. 14 (1987) 538-548. H.P. Chan, K. Doi, C.J. Vyborny, B A! Schmidt, C.E. Metz, K.L. Lam, T. Ogura, Y. Wu, H. MacMahon, Improvement in radiologists' detection of clustered microcalcifications on mammograms, Invest. Radiol. 20 (1990) 1102-1110. J. Dengler, J. Behrens, J.F. Desaga, Segmentation of microcalcifications in mammograms, IEEE Trans, on Medical Imaging 12 (1993) 634-642. L. Shen, R.M. Rangayyan, J.E.L. Desautels, Application of shape analysis to mammographic calcifications, IEEE Trans. Med. Imaging 13 (1994) 263-274. Y. Wu, K. Doi, M.L. Giger, R.M. Nishikawa, Computerized detection of clustered microcalcifications in digital mammograms: applications of artificial neural networks, Med. Phy. 19 (1992) 555-560.

Y. Wu, M.L. Giger, K. Doi, R.A, Schmidt, C.E. Metz, Artificial neural networks in marnmography: application to decision making in the diagnosis of breast cancer, Radiology 187 (1993) 81-87.

W. Zhang, K. Doi, M.L. Giger, Y. Wu, R.M. Nishkawa, R.A. Schmidt, Computerized detection of clustered microcalcifications in digital mammograms using a shift-invariant artificial neural networks, Med. Phy. 21 (1994) 517-524. W. Zhang, K. Doi, M.L. Giger, Y. Wu, R.M. Nishkawa, R.A. Schmidt, An improved shift-invariant artificial neural networks for computerized detection of clustered microcalcifications in digital mammograms, Med. Phy. 23 (1996) 595- 601. S.C.B. Lo, H.P. Chan, J.-S. Lin, H. Li, M.T. Freeman, S.K. Mun, Artificial convolution neural network for medical image pattern, Neural Networks 7/8 (1995) 1201-1214.

H.P. Chan, S.-C. Lo, B. Sahiner, K.L. Lam, M.A. Helvie, Computer-aided detection of mammographic microcalcifications: pattern recognition with an artificial neural network, Med. Phys. 22 (1995) 1555-1567. A.F. Laine, S. Schuler, J. Fan, W. Huda, Mammographic feature enhancement by multiscale analysis, IEEE Trans. Med. Imag. 13 (1994) 725-740. IEEE Mag. Eng. Med. Biology. 14 (1995) 536-577.

However, each method has its own strengths and weaknesses. This is mainly due to the nature of mammographic characteristics and appearances. In order for a CAD systems to be diagnostically useful in detection and segmentation of microcalcifications, it is very important to design and develop effective preprocessing techniques to smooth inhomogeneous background and remove structured noise that is caused by parenchyma tissues and texture variations.

In addition, individual modules for automated detection or classification of microcalcifications have been reported [see, e.g.,:

P.-S. Liao, B.C. Hsu, C.-S. Luo, P.-C. Chung, T.-S. Chen, S.-K. Lee, L. Cheng, C.-I Chang, Automatic detection of microcalcifications in digital mammograms, 18th Annual Int. Conf. IEEE Eng. in Med. and Bio. Society (1996) 88-89.

B.-C. Hsu, P.-C. Chung, C.-I Chang, Automated system for detection and classification of microcalcifications in digital mammograms, Proc. CVGIP'96, Taiwan (1996) 127-134.

S.-C. Lo, P.-C. Chung, B.-C. Hsu, C.-I Chang, S.K. Lee, B.-S. Liao, An algorithm for detection and segmentation of clustered microcalcifications on mammograms, Proc. 2nd Medical Eng. Week of the World, 3rd Asian-Pacific Conf. on Medical and Biological Engineering, Taiwan (1996) p. 102. C.-S. Lo, P.-C. Chung, C.-I Chang, S.K. Lee, A computerized system for detection and segmentation of clustered microcalcifications, Joint Conf. 1996 International Computer Symposium, Taiwan (1996) 247-253.

C.-S. Lo, S.-K. Lee, P.C. Chung, C.-I Chang, An automatic computerized system for detection and segmentation of clustered microcalcifications on mammograms, Proc. the 46th Annual Meeting of Radiological Society of Republic of China, (1997) p. FI 57],

but not integrated as a single system for clinical applications. One of the advantages of the invention is its open architecture where each module is an object-oriented, plug-in component and can be upgraded individually to improve the whole CAD system. Another is that the prototype system can be included in the Picture Archiving and Communication System (PACS) currently implemented in TaiChung Veterans General Hospital (TCVGH), Taichung, Taiwan, Republic of China, as described in C.-W. Yang, P.-C. Chung, C.-I Chang, An image capture and communication system for emergency computed tomography, Computer Methods and Programs in Biomedicine 52 (1997) 139-145 and C.-W. Yang, P.-C. Chung, C.-I Chang, S.-K. Lee, L.-Y. Kung, A hierarchical model for PACS, Computerized Methods in Medical Images and Graphics 21(1) (1997) 29-37. This integration allows radiologists to have easy access to the system and assist them in making their diagnosis.

SUMMARY OF THE INVENTION

This invention provides a computer-assisted method and system for real-time processing of marnmography screening to detect and classify microcalcifications or other physical abnormalities such as lung nodules or venous beading. The system is made up of four modules, each of which is designed for a particular task. The first module is called the Mammogram Preprocessing Module which takes a mammogram and digitizes it into an 8-bit image of size 2048 x 2048. It then extracts the breast region from the mammogram, enhances the extracted breast and stores the processed mammogram along with the original unprocessed mammogram in a database of the system for future reference.

The second module, called the MCCs Finder Module, is designed to find and locate suspicious clusters of microcalcifications ("MCCs") and then segment these clustered MCCs from the background as regions of interest ("ROIs") that will be used for further MCC detection. This module identifies and feeds these ROIs into the third module.

The third module, called the MCCs Detection Module, is designed to detect MCCs in the identified ROIs. The MCCs Detection Module is a real-time processing system that uses two different window sizes to extract MCCs. It begins with a large window of size 64 x 64 to fast screen mammograms to find large calcified areas. Then a smaller window of size 8 x8 extracts small and tiny MCCs. These segmented clustered MCCs are then sent to the fourth module.

The fourth module, called the MCCs Classification Module, classifies each of the segmented clustered MCCs into five categories as suggested by Breast Imaging Reporting and Data System (BI-RADS) generally utilized by the medical profession: negative (no further operation), benign finding (MCCs found to be negative), probably benign finding (short interval follow-up suggested), suspicious abnormality (biopsy should be considered), and highly suggestive of malignancy (appropriate action should be taken).

Some of the unique aspects of this method and system are the integration of all four modules as a single entity for clinical applications, the open architecture where each module is an object- oriented and plug-in component with the ability to be upgraded, the ability to implement the system in the Picture Archiving and Communication System (PACS) currently utilized in other areas of the world, and the ease of access for radiologists to the system and its assistance in diagnosis.

It is an object of the invention to provide a system for evaluation of mammograms which can assist in detecting microcalcifications or other abnormalities.

It is a further object of the invention to provide a modular system, designed so that individual modules may be upgraded and integrated with other modules.

A principal feature of the invention is effective preprocessing techniques which smooth inhomogeneous background and remove structured noise that is caused by parenchyma tissues and texture variations.

Among the advantages of the invention are rapid identification of segments of a mammogram suggestive of microcalcifications or other abnormalities.

These and other objects, features and advantages which will be apparent from the discussion which follows are achieved, in accordance with the invention, by providing a novel system for processing marnmography images to detect and classify microcalcifications and other abnormalities

The various features of novelty which characterize the invention are pointed out with particularity in the claims annexed to and forming a part of this disclosure. For a better understanding of the invention, its advantages and objects, reference is made to the accompanying drawings and descriptive matter in which a preferred embodiment of the invention is illustrated.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and still other objects of this invention will become apparent, along with various advantages and features of novelty residing in the present embodiments, from study of the following drawings, in which:

FIG. 1 is a block diagram of the marnmography screening system.

FIG. 2 is a diagram of four quadrants of a co-occurrence matrix divided by a threshold t.

FIG. 3 illustrates the structure of the Shape Cognitron.

FIG. 4 illustrates the spatial patterns and their weight assignments used for identifying the classification of MCCs including: (a) weight assignments of 20 orientation spatial patterns in

layer ^i corresponding to 8 degree spatial patterns; (b) 8 orientation spatial patterns in layer Cι

obtained by merging the 20 orientation spatial patterns in layer S, ; (c) Weight assignments of 8

orientation spatial patterns in layer C_{ obtained by summing each column to represent 8 degree spatial patterns corresponding to < 45°, 45° , 90 °, 135 °, 180 °, 225 °, 270 °, 360 °.

FIG. 5 illustrates (a) Breast regions divided into small blocks with size 64 x 64 ; (b) ROC curve

of TPF versus FPF.

FIG. 6 illustrates (a) A system frame of the demonstration; (b) A selected mammogram and an ROI shown in the windows.

FIG. 7 illustrates (a) Suspicious MCCs area located by the MCCs Finder Module; (b) Detection result of Fig. 8(a) resulting from applying the JRE to the image in Fig. 7(a).

FIG. 8 illustrates (a) A representative mammogram from the Nijmegen data base; (b) Background elimination by the block region growing method; (c) Breast region extracted by the K-means clustering-based thresholding method

FIG. 9 is an image resulting from applying blanket method to the image in Fig. 8(c)

FIG. 10 includes images resulting from enhancements: (a) upon applying the gradient enhancement to Fig. 9; (b) upon applying the contrast enhancement to Fig. 10(a); (c) upon applying the Gaussian filtering to Fig. 10(b)

FIG. 11 displays the detection and classification results of Fig. 10(c)

FIG. 12 consists of tables showing experimental results DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring to the drawings, the invention is a novel system for processing marnmography images to detect and classify microcalcifications and other abnormalities using a system shown in overview in Figure 1.

The system is made up of four modules, each of which is designed for a particular task.

Module 1

The first module, called the Mammogram Preprocessing Module (1.00), takes a mammogram (1) and digitizes it in a mammogram digitization step (101). It then extracts the breast region from the mammogram in a breast region extraction step (102), enhances the extracted breast image and stores the processed mammogram along with the original unprocessed mammogram in a database (103) of the system for future reference. All mammograms are first digitized into an 8- bit image of size 2048 x 2048 by a Travel film digitizer made by Vidar System Corporation with

260 DPI (approximately 0J mm/pixel).

The need of breast region extraction arises from several main reasons. One is that approximately or more than one-third of a mammogram is dark breast background which provides very little information for diagnosis. Additionally, for the purpose of storage and fast retrieval, this background should not be included for diagnosis. Another is computational efficiency. Since the size of a mammogram is generally 16 times as large as CT and MR images, it will be highly desirable if the dark breast background can be removed while retaining only the breast region for future data processing. More importantly, in order to make the system more efficient, extracting regions of interest (ROIs) is the first step of computer automation. In marnmography screening, the breast is the region in which radiologists are interested.

Extracting the breast region from a digitized mammogram involves: a) dividing a mammogram into blocks; b) computing the mean, variance, and energy function for each block; c) identifying a region of blocks with an energy function below a proscribed tolerance; d) eliminating the blocks in this region; e) calculating the minimum and maximum energy functions for the remaining blocks of the mammogram; and f) extracting the area where the energy function is at least one half of the summation of the minimum and maximum energy values.

In step (c) above, a block region growing method is used to eliminate the breast background. The process can be begun with either the block that has the lowest average intensity based on the assumption, which is not always the case with lepto-breasts, that the darkest image block must be part of the breast background or to picking one of four corner blocks of a mammogram as a seed block. In steps (e) and (f), a K-means clustering-based thresholding technique is applied to further refine and smooth the estimated breast region obtained by the previous steps.

MODULE 2

Since only clustered MCCs provide useful diagnostic information about malignancy, the second module, the Finder Module (200), is designed to find and locate suspicious clusters of MCCs and then segment these clustered MCCs from the background as regions of interest (ROIs) that will be used for further MCCs detection. Detection of MCCs is crucial to success in detecting early breast cancer and has been investigated extensively. However, in most cases, the regions of interest for possible MCCs are pre-selected manually by radiologists. From a diagnostic point of view, only clustered MCCs are of interest because single MCC blobs or sparse MCCs are generally caused by breast tissues and noises and do not provide much useful information for diagnosis.

The second module finds and locates areas that contain possible clustered MCCs (201). The process, known as the blanket method, is used to automatically find and locate regions of interest which may have MCCs. In this process, the surface area of an object for a specific distance above or below the surface is calculated. Next, a fractal dimension value is calculated based on the fact that the surface area is proportional to the distance above or below the surface. Details of the calculation and its theoretical basis may be found in B.B. Mandelbrot, The Fractal Geometry of Nature, New York: Freeman, 1977, and S. Pleg, J. Naor, R. Hartley, D. Avnit, Mutiple resolution texture analysis and classification, IEEE Transaction on Pattern Analysis and Machine intelligence 6 (1984) 518-523. Next a measure, D, of image texture characterization is derived. Since clustered MCCs usually have high gray gradients and variances in texture, in this case, D provides an important indication of the existence of clustered MCCs. More importantly, a cluster of MCCs matches the fractal property which is directly determined by the scaling factor r of the fractal model. The fractal dimension value provides an important indication of the existence of clustered MCCs. Because a cluster of MCCs matches the fractal property that is directly determined by the scaling factor of the fractal model; two window sizes, 6 x 64 (t _e large window)

and 8 x 8 (the small window) are used in this module to detect and find suspicious areas of

clustered MCCs, referred to as regions of interest. Using the 64 x 64 window enables users to

quickly screen mammograms to find large calcified areas, while the x 8 window is used to detect small, isolated microcalcifications. However, a small lesion within a large window may sometimes fail to satisfy the fractal property. In this case, they may be passed and go undetected by a large window such as 64 x 64. T e use of a small window ensures that such small lesions will meet the fractal property and can be extracted by the fractal dimension. The concept of the fractal dimension is also very useful for CT liver image classification where it was used to detect three different types of liver regions, normal liver, hepatoma and liver boundary. [E.-L Chen, P.C. Chung, C.-L. Chen, H.M. Tsai, C.-I Chang, An automatic diagnostic system for CT liver image classification, IEEE Trans. Biomedical Engineering 45 (6) (198) 783-794].

MODULE 3

As ROIs are identified, they are fed to the third module, the Detection Module (300), whose task is to detect MCCs in ROIs. It is a real-time processing system that uses two different window sizes to extract MCCs. It begins with a large window to quickly screen mammograms to find large calcified areas. This is followed by a smaller window o extract small and tiny MCCs.

The goal of developing the fractal dimension in MCCs Finder Module is to find and locate suspicious clustered MCCs and to provide radiologists with regions of interest that require their attention. Every region of interest, however, is not calcified. In particular, some detected pixels may be noise or breast tissues and some MCCs are embedded in or obscured by the inhomogeneous background within the breast. The MCCs Detection extracts possible MCCs from these regions of interest for diagnosis.

The technique is based on the assumption that the gray-level intensity of calcified pixels is generally brighter than that of uncalcifϊed pixels. The problem is that calcified pixels, although brighter than uncalcifϊed pixels, also have low intensities, and that when calcified pixels have higher intensities, their neighboring pixels may also have high intensities so that the relative contrast of these calcified pixels is significantly reduced.

In summary, the Detection Module begins with enhancing the low intensity of calcified pixels (step 301). Next, the low contrast'of the enhanced calcified pixels is improved (step 302). Finally, a Gaussian filter is used to remove suppressed undesired high intensity uncalcified pixels, particularly noise pixels (step 303). Since the resulting Gaussian filtered images are generally gray scaled, and these MCCs can only be detected by visual inspection; entropic thresholding methods are used to produce binary images that show the locations of MCCs (step 304).

More specifically, Gradient enhancement of each pixel is accomplished by finding the average gradient between each specific pixel and all adjacent pixels and adding the gray level at the location to this average gradient. Next, Contrast enhancement is accomplished by averaging the gray level intensity of each pixel by using a 3 x 3 window to average 8-neighbor connectivity pixels of each pixel. This increases the contrast between pixels by reducing the intensities of uncalcified pixels. This averaging processing can be repeatedly applied until a desired outcome is achieved. Finally, a Gaussian filter is applied to eliminate noisy and interfering pixels that can be caused by breast tissues.

Although the image quality of the mammograms has been enhanced by these steps, the limitation that the processed mammograms are gray scale and require the visual inspection of radiologists remains. In order for a system to be fully computer automated and not require human intervention, an automatic segmentation method is needed to segment detected MCCs from the background. Entropy-based thresholding methods performed the function of capturing the characteristics of MCCs better than other popular thresholding methods such as Otsu's method [described in N. Otsu, A threshold selection method from gray-level histograms, IEEE Trans. System Man Cybernetics 9 (1) (1979) 62-66] .

Entropic thresholding is a technique that adopts entropy as a criterion to threshold an image. The concept of entropy has been widely used in data compression to measure information content of an information source. In this technique, an image is viewed as an information source with the probability distribution given by its gray-level image histogram. This digital image can be represented by a matrix based on gray levels or a co-occurrence matrix, which is a square matrix that considers only the number of transitions between gray levels. Several different thresholding methods are used to aid in the detection of MCCs.

A widely used co-occurrence matrix is an asymmetric matrix that only considers the gray level transitions between two adjacent pixels. This matrix considers only the pixels on the right and bottom transitions since it was found that including the pixels on the left and top transitions does not provide significant information or improvement. A desired transition probability from one gray level to another is obtained by normalizing the total number of transitions in this cooccurrence matrix. These transitions are used for thresholding the image so that For a particular threshold t, the co-occurrence matrix is partitioned into four quadrants, A, B, C, and D, shown in Fig. 2. These four quadrants are grouped into two classes. Pixels with gray levels equal to or below the threshold are assigned to the background quadrants A, those with gray levels above the threshold value are assigned to the foreground (objects) and quadrant C, and transitions across boundaries between background and foreground are assigned to quadrants B and D. Probabilities that are conditioning on a specific quadrant called "cell probabilities".

Three different types of entropic thresholding can be used.

First, a Local Entropy denoted by H_LE{t), which was derived by N.R. Pal and S.K. Pal, infra,

sums the local within-class transition entropies of the foreground and the background. It contains both the local transitions from background to background (BB) and objects to objects (FF). A second entropic thresholding method, Joint Entropy (JE) provides edge information about transitions from background to foreground (BF) and foreground to background (FB) as assigned

to quadrant B and quadrant D. A third, the global entropy H_0E (t) is simply the sum of the local

entropy H_LE(t) and the joint entropy.

Next, relative entropy-based thresholding for each of the three thresholding methods provides a means of comparing the co-occurrence matrices of an original image and a thresholded image. For the above thresholding techniques they are called Local Relative Entropy, Joint Relative Entropy, and Global Relative Entropy.

Because entropic thresholding and relative entropic thresholding perform differently and have different advantages and so that radiologists are able to make various comparisons among different thresholding methods, this module includes 3 entropic thresholding methods, 3 relative entropic thresholding methods, one popular threshold method (Otsu's method) and a manual threshold adjustment that allows radiologists to be able to manually adjust the threshold value themselves.

Details of the above methods may be found in N. R. Pal, S. K. Pal, Entropic thresholding, Signal Processing 16 (1989) 97-108; C.-I Chang, K. Chen, J. Wang, M.L.G. Althouse, A relative entropy approach to image thresholding, Pattern Recognition 27 (9) (1994) 1275-1289; and C- W. Yang', P.-C. Chung, C.-I Chang, J. Wang, M.L.G. Althouse, Entropic and relative entropic thresholding, Joint Conf. 1996 International Computer Symposium (1996) 82-89. Module 4

Finally, the fourth module is the MCCs Classification Module (400) that classifies each of the segmented clustered MCCs into five categories, "negative" (no further operation), "benign finding" (MCCs found to be negative), "probably benign finding" (short interval follow-up suggested), "suspicious abnormality" (biopsy should be considered) and "highly suggestive of malignancy" (appropriate action should be taken) to represent different stages of MCCs as suggested in BI-RADS (Breast Imaging Reporting and Data System), 3rd edition, American College of Radiology {1998).

After clustered MCCs have been identified by the MCCs Detection Module, the segmented clustered MCCs must be classified. This classification method is called Shape Cognitron ("SC") (401). The SC was originally developed for classification of MCCs, but later used for venous beading detection in retinal images. The structure of SC is given by Fig. 3. It has an input unit

^o (402), two simple layer-complex layers combined units, referred to as (*-*' , i ) (called shape

orientation extraction unit) and

(called shape feature extraction-classification unit), and

one 3-D figure layer (405), called shape orientation display layer lying between

and

units. Each of the units is briefly described as follows.

Shape Cognitron (SC) is derived from Tricognitron and Fukushima's Neocognitron. It was particularly designed to classify clustered microcalcifications into malignancy and benignancy using a set of shape features it generates. It is known that malignant clustered microcalcifications generally have irregular shapes as opposed to round shape or egg-shaped benign clustered microcalcifications. SC captures the shape curvatures of clustered microcalcifications and provides a crucial indication of malignancy. ■

The SC is a neural network-like system and consists of two major components, each of which has two layers, called simple layer (403 and 406) and complex layer (404 and 407) and a mid-layer between them, called 3-D figure layer. The first component is similar to that used in Neocognitron, but it uses 20 orientation spatial patterns to specify 8 degree spatial patterns,

0° = 360° .- East (E), 45° _: Northeast (NE), 90 ° _: North (N), 135 °_: Northwest (NW), 180 ° :

(West), 225 ° _: Southwest (SW), 270 ° ^■ South (S), 335" _: Southeast (SE). It implements a two-

layer operation, a simple layer denoted by layer ^i and a complex layer denoted by layer X .

Layer "ι contains 20 cell planes of size N x N resulting from 20 orientation spatial patterns

operating on the input pattern. Layer Ci contains 8 cell planes of size N x N obtained by

merging 20 cell planes in layer ^ι that represent 20 orientation spatial patterns. They result from

different weight assignments generated by a masking process using the 20 orientation spatial

patterns in layer ι and a particularly designed merging procedure in layer Ci . The layer

following the first component is 3-D figure layer. It is a feature extraction-display layer that extracts and stores the information of the shape orientations of an input pattern in the third dimension. It displays the input pattern as a 3-D figure using the numeric values generated in

layer i as the elevation of the pattern to represent 8 different degrees in the third dimension.

The second component can be viewed as a joint feature selection and classification system that is

made up of a feature selection layer, layer ^2 which generates a desired set of shape features

from the 3-D figure layer and a classification layer, layer ^2 which employs a probabilistic neural network (PNN) as a classifier with the shape features produced by layer ^2 as inputs.

More specifically, the input unit ^υo (402) takes an input pattern of size N x N that may vary

with applications. In our case, the input patterns are clustered MCCs of size 256 x 256 produced

by the MCCs Detection Module (300). The (^1 ,^1) unit contains a simple layer ^ (403) followed

by a complex layer Ci (404). It is a shape information extraction unit that extracts the geometric

shape orientations of an input pattern, then converts them into numeric values for computer

processing. The idea of using (^1 ,^1 ) is similar to that used in Neocognitron and Tricognitron.

Layer ^1 uses a set of 20 orientation spatial patterns (8 2 x 2 spatial patterns numbered from 1 to

8 and 12 x 3 spatial patterns numbered from 9 to 20) shown in the first column of Fig. 4(a) operating on the input pattern to capture 8 degree spatial patterns as shown in the top row of Fig. 4(a) that represent 8 different degrees: < 45° (i.e., degrees less than < 45°), 45° , 90 ° , 135 °,

180 " , 225 " , 270 " and 0 " = 360" in the second top row. The pixel labeled by "x" is the seed

pixel currently being examined during masking processing. The first 12 orientation patterns in

layer ^1 of Fig. 4(a) are designed to extract 8 different degree spatial patterns that correspond to

multiples of 45° , 45° , 90 " , 135 °, 180 ", 225 ", 270 °, 335", 360 " . If there is a match between an

orientation spatial pattern and a degree spatial pattern, a weight "1" is assigned; a weight "0", otherwise. For example, pattern 1 specifies the East orientation. It matches all degree spatial

patterns except those representing degrees < 45° and 45" . As a result, in the row of pattern 1,

there are 2 "0"s and 6 's. The next 8 orientation spatial patterns in layer ^sι from pattern 13 to pattern 20 are designed to extract degrees 167.5°, 22.5", 202.5", 247.5°, 67.5°, 112.5°, 292.5"

and 337.5° respectively. Since these 8 patterns describe smaller degrees that are multiples of

22.5° but not multiples of 45° that are already specified by the first 12 orientation patters, they

are crucial to measure subtle differences among geometric shape features. So, if there is a match, a weight "2" will be assigned; a weight "0", otherwise. For example, pattern 13 matches degree

spatial patterns, 225 " , 270 ° , 335 " . So, there are 3 "2"s appearing in the row of pattern 13 under

the columns of degrees, 225 ° , 270 " , 335° . As a result, a total of 20 256 x 256 cell planes are

produced in layer ^ι , each of which represents a specific orientation spatial pattern with assigned

weights given in Fig. 4(a). These 20 cell planes will be then input to the next complex layer

.

The task of layer

i_s to fuse the shape orientation information produced by Fig. 4(a) by

merging the 20 cell planes generated in layer ^i into a set of 8 cell planes so that each cell plane

in layer represents the information generated by one specific orientation spatial pattern that

corresponds to one of 8 degrees, < 45" , 45" , 90 " , 135 " , 180 " , 225 " , 270 " , 360 ° . Fig. 4(b)

shows how the 8 orientation spatial patterns in layer ^I are generated by merging the 20

orientation patterns in layer ^i where a merge is described by a © . It should be noted that the

weight of a merged pattern is not obtained by summing all the weights of merging patterns. Instead, we adopt a rule that when two patterns are merged, the higher weight of the pattern will be assigned to the weight of the pattern obtained by merging these two patterns. Fig. 4(c) shows

the weights obtained for these 8 orientation spatial patterns in layer ^l resulting from the merging process shown in Fig. 4(b). For example, pattern 1 in layer ^ι in Fig. 4(b) is a result of

merging 6 patterns 1-4, 14-15 in layer shown in the first row in Fig. 4(b). Therefore, the

weight of the degree pattern corresponding to 1 5 " of pattern 1 in layer X was obtained by the

highest weight produced by pattern 15 in layer ^ι , which was "2" according to Fig. 4(a). As

another example, the weight of the degree pattern corresponding to 90 " of pattern 1 in layer X

was "1" because from Fig. 4(a) the highest weight among the 6 merging patterns, 1-4 and 14-15

in layer "ι is "1" produced by the weight of pattern 1 in layer ^ι . Similarly, the weights of the

remaining degree patterns of pattern 1 corresponding to < 45", 180 °, 225°, 270 °, 360° _were

obtained by 0, 0, 2, 2, 2, 2 respectively as shown in Fig. 4(c).

The layer following the first (^ι , ) unit is a 3-D figure layer (405), which is a shape information

display layer. It uses a 3-D figure to represent the shape orientations of an input pattern in the third dimension, called elevation. The magnitude of the elevation of each degree spatial pattern is

expressed by the sum of the weights produced by the 8 orientation patterns in layer that

correspond to this particular degree pattern. For example, the degree spatial pattern represented

by < 45" only matches patterns 2 and 3 in layer

. So, the elevation of < 45° degree spatial

pattern is the sum of the column under < 45° degree spatial pattern, which is 1 + 1 = 2. Similarly,

3, 4, 5, 7, 9, B(l 1) and C(12) are obtained for degree spatial "patterns corresponding to 45° , 90 ° ,

135 ° 180 " 225 ° 270 " 360 " The second simple-complex layers combined unit, i_{s a} shape feature extraction-

classification unit where layer ^2 (406) extracts and selects an appropriate set of shape features

for classification and layer C₂ (407) performs classification based on features generated by layer

^2. Layer ^2 produces a set of shape features on the basis of the shape orientation information

provided by the 3-D figure in 3-D figure layer. Each of these features represents a specific shape

characteristic. It should be noted that layer ^2 is very flexible and varies with feature selections.

^2 layer is a classification layer that employs a probabilistic neural network (PNN) (408) as a

classifier. It takes as input the shape features produced by layer ^2 and its outputs are used for

classification. According to the MCCs assessment categories, five outputs (Figure 1, 500) are

used in layer ^2, which are "negative" (no further operation), "benign finding" (MCCs found to

be negative), "probably benign finding" (short interval follow-up suggested), "suspicious abnormality" (biopsy should be considered) and "highly suggestive of malignancy". Each of these outputs reflects different stages of MCCs. It is worth noting that the number of cell planes

in layer C₂ is determined by the number of patterns needed to be classified or recognized.

Because layer C₂ can be designed separately, it offers S-Cognitron great flexibility to adapt

different applications. A PNN may be implemented to perform classification task. However, a backpropagation neural network (BNN) can also be used to detect venous beading in retinal images, as described in C.-W. Yang, D.-J. Ma, S.-C. Chao, C.-M. Wang, CH. Wen, S.C. Lo, P.- C. Chung, C.-I Chang, A computer-aided diagnostic detection system of venous beading in

retinal images, Optical Engineering 39 (5) (2000). In addition, layer ^C2 is an application-

dependent layer and can be designed by specific criteria for classification. In summary, S-Cognitron works as follows:

(1) It first takes the clustered MCCs (305) produced by the MCCs Detection Module (300) as

input patterns in layer ^u<> (402).

(2) It employs the shape orientation extraction unit (^1 ,^1) (403 and 404) to extract shape

orientations and convert them to numeric representations.

(3) It then displays the numeric values of shape orientations in the 3-D figure layer (405).

(4) The shape feature extraction-classification unit ("2,C₂) _{ls use}d to extract and select shape

features in ^2 layer (406) and then classifies clustered MCCs in layer ^C (407). The outputs of

layer ^2 produce five-category classification results output (500) for diagnosis of the input

clustered MCCs.

Experimental Results Using Nijmegen Database

The Nijmegen database was used for experiments. The choice of this database is based on the availability of the Nijmegen database in the public domain and biopsy results are also provided for each case in the database. There are 40 mammograms from 21 patients in Nijmegen database collected by the Department of Radiology, Nijmegen University Hospital, Netherlands. Each of the mammograms in the database was digitized by an Eikonix 1412 12-bit CCD camera with a fixed calibration. The maximum output level (4095) corresponds to the optical density 0.18. A sample aperture of 0.5 mm in diameter and 0.1 mm sampling distance were used for digitization. All the mammograms were corrected for inhomogeneity of the light source (Gordon planar 1417) and recorded by a Kodak MINR/SO1777 screen/film combination. Each mammogram shows one or more clustered MCCs. The 40 mammograms contain a total of 102 clustered MCCs and the detailed locations and radii of these clustered MCCs were also provided by radiologists.

The invention was embodied in a general purpose computer coupled to storage, user interface and display devices as shown in Figures 13 through 15.

Since the first module is a Mammogram Preprocessing Module designed to segment the breast region from the entire mammogram, it has little impact on the system performance. So, the experiments were specifically designed to evaluate the performance for the following three modules, the second module-MCCs Finder Module, the third module-MCCs Detection Module and the fourth module-MCCs Classification Module. In this case, 1 4 positive ROIs and 41 negative ROIs were selected from the Nijmegen data base where a positive ROI means that it contains clusters of microcalcifications and a negative ROI implies that no cluster of microcalcifications is found in the region.

As noted in the MCCs Finder Module, the areas of containing suspicious MCCs were located by the fractal dimension using a window to screen the breast region extracted by the Mammogram Preprocessing Module as shown in Fig. 5(a) where the breast region was divided into blocks of small regions. Then the MCC Finder Module determined whether or not each small region contains MCCs. In order to demonstrate its detection performance, the receiver operating characteristic (ROC) analysis [described in J.A. Swets, R.M. Pickett, Evaluation of Diagnostic Systems: Methods from Signal Detection Theory, New Yok: Academic (1982) and C.E. Metz, ROC methodlogy in radiological imaging, Radiology 21 (1986) 720-733] was used for performance evaluation.

Two criteria were used for evaluation, referred to as true positive fraction (TPF) and false positive fraction (FPF) which are defined in BI-RADS (Breast Imaging Reporting and Data System), 3rd edition, American College of Radiology (1998). The ROC curve of TPF versus FPF was plotted in Fig. 5(b) where the detection rate can reach as high as 90%) at the false alarm rate 1%.

To evaluate the performance of the MCCs Detection Module, here, we adopt the criteria suggested in BI-RADS to define sensitivity (SS), specificity (SP) and positive predictive value (PPV) for performance evaluation, defining TPN to be the number of ROIs which contains clusters of microcalcifications and are actually detected, and FPN to be the number of ROIs which contains no clusters of microcalcifications but were falsely detected. The JRE was used for evaluation and compared to Otsu's method. The detection results produced by the MCCs Detection Module are tabulated in Table 1 of Figure 12, which shows that JRE is indeed a better method than Otsu's method.

Finally, these detected clustered MCCs were then fed to the MCCs Classification Module for benign-malignancy classification. It should be noted that all the 41 negative ROIs were not included here because they were filtered out by the second module, MCCs Finder Module, which located potential positive ROIs for MCCs while eliminating negative ROIs which contain no MCCs. As a result, only.104 ROIs needed to be classified. Among these 104 positive ROIs 29 were benign and 75 malignant. Three sets of training data were selected to evaluate the classification performance. The first training data set consists of 7 benign and 19 malignant cases, whereas the second and third training data sets were made up of 8 benign+25 malignant cases and 10 benign+31 malignant cases, respectively. These training cases were selected from the 104 ROIs. The resulting confusion matrix is given in Table 2 of Figure 12, where the 104 positive ROIs were classified according to the following five categories: "negative", "benign finding", "probably benign finding", "suspicious abnormality" and "highly suggestive of malignancy". Since the information provided by the Nijmegen data base about each cluster of MCCs is based on its biopsy report, all the clusters of MCCs in the Nijmegen data base must be classified into either benign or malignant and cannot be classified in accordance with five categories suggested by the MCCs Classification Module. In this case, we declared a cluster of MCCs to be malignant only if it fell in the categories of "suspicious abnormality" and "highly suggestive of malignancy"; benign, otherwise.

Using TPN, FPN, TNN and FNN defined above, we can derive three rates for system performance evaluation, which are detection rate (DR), false alarm rate (FAR) and correct classification rate (CR). These results are also tabulated in Table 3 of Figure 12.

This experiment showed that if 41 training cases were used to test 104 cases, we could achieve as high as 95% classification rate with 93% detection rate and 0% false alarm rate. As we can see from Tables 2-3 (Figure 12), the more training data that were used, the better the classification. It is worth noting that the classification performance is not linearly proportional to the number of training cases used.

The processing time of each module in the system using PC-Pentium 200 MHz is given in Table 4 of Figure 12. From the table, the total amount of the time required for the entire system to process the mammogram in Fig. 6(b) averaged about 72 seconds.

Thus, there has been described a novel system for processing marnmography images to detect and classify microcalcifications and other abnormalities that has a number of novel features, and a manner of making and using the invention.

While a specific embodiment of the invention has been shown and described in detail to illustrate the application of the principles of the invention, it will be understood that the invention may be embodied otherwise without departing from such principles and that various modifications, alternate constructions, and equivalents will occur to those skilled in the art given the benefit of this disclosure. Thus, the invention is not limited to the specific embodiment described herein, but is defined by the appended claims.

Claims

CLAIMSWe claim:

1. A method for processing digitized mammogram images comprising the steps of:

(a) providing a digitized mammogram image;

(b) extracting a region of interest from said image tlirough block region growing;

(c) identifying suspicious areas within the region;

(d) using entropic thresholding to detect the presence or absence of clusters of possible abnormality within selected regions of interest; and

(e) classifying said clusters, if present, based on their likelihood of abnormality.

2. A method for detecting, and assisting in the diagnosis of, microcalcifications in a digitized mammogram comprising the steps of:

(a) providing a mammogram;

(b) extracting a breast region from the mammogram through block region growing;

(c) identifying suspicious areas within the breast region;

(d) using entropic thresholding to detect clusters of possible abnormality within selected regions of interest; and

3. A method for detecting and assisting in the diagnosis of microcalcifications in a digitized mammogram comprising the steps of:

1

(a) providing a digitized mammogram;

(b) extracting a region of interest from said digitized mammogram through block region growing; (c) identifying suspicious areas and segmenting said areas into suspicious subareas;

(e) classifying said clusters, if present, based on likelihood of malignancy by:

(1) inputting a cluster of suspected microcalcifications,

(2) extracting a set of geometric shape orientations for the cluster,

(3) generating a numeric weight representation for each of these shape orientations,

(4) creating a third dimension, referred to as elevation, to store the numerical representations generated in step (3),

(5) forming a three-dimensional figure pattern based on the third dimension elevation,

(6) extracting and generating a set of shape features based on the shape orientation information, and

(7) classifying said shape features.

4. A method for extracting the breast region from a digitized mammogram comprising the steps of:

(a) dividing a mammogram into blocks;

(b) computing the mean, variance, and energy function for each block;

(c) identifying a region of blocks with an energy function below a predetermined level;

(d) eliminating the blocks in this region;

(e) calculating the minimum and maximum energy functions for the remaining blocks of the mammogram; and

(f) extracting the area where the energy function is at least one half of the sum of the minimum and maximum energy function levels.

5. A method for detecting microcalcifications in a digitized mammogram and segmenting said clusters into suspicious areas comprising the steps of:

(a) dividing the digitized mammogram into a first set of separate windows of a selected size to be used for identifying suspicious areas in a digitized mammogram; and

(b) scamiing each window in the digitized mammogram by:

(1) calculating a fractal dimension value for each window, and

(2) selecting suspicious areas in the mammogram based on the fractal dimension value.

6. The method of claim 5 further comprising selecting one or more subsequent sets of separate windows of different size than said first set of separate windows, based on the size of probable calcifications and repeating steps (a) and (b) for each such subsequent sets of separate windows.

7. A method for detecting microcalcifications in a digitized mammogram comprising the steps of:

(a) providing a digitized mammogram;

(b) selecting regions of interest of said digitized mammogram, said region of interest comprising one or more pixels, for analysis;

(c) enhancing the gradient and contrast between each pixel and all adjacent pixels within that reason of interest;

(d) suppressing interference and noise by applying a Gaussian filter; and

(e) using entropy-based thresholding to segment microcalcifications from the background.

8. The method of claim 7 wherein said entropy-based threshholding comprises relative entropic thresholding.

9. A method for classifying microcalcifications in a digitized mammogram comprising the steps of:

(a) inputting from a digitized mammogram a cluster of suspected microcalcifications;

(b) extracting a set of geometric shape orientations for the cluster;

(c) generating a numeric weight representation for each of these shape orientations;

(d) creating a third dimension, referred to as elevation, to store the numerical representations generated in step (c);

(e) forming a three-dimensional figure pattern based on the third dimension elevation;

(f) extracting and generating a set of shape features based on the shape orientation information; and

(g) classifying said shape features.

10. A method for classifying digitized images comprising the steps of:

(a) inputting a digitized image;

(b) extracting a set of geometric shape orientations for suspicious areas in the image;

(g) classifying said shape features.

11. The method of claim 3, 9 or 10 wherein the three-dimensional figure pattern is generated by a weight in a simple layer that represents a two-dimensional orientation spatial pattern and a weight from a complex layer that represents a spatial orientation in the third elevation dimension.

12. A device for processing digitized mammogram images comprising: a) means for acquiring a digitized mammogram image; b) processing means for processing said digitized mammogram image, said processing means comprising a mammogram preprocessing module for extracting a breast region from said digitized mammogram image;

a microcalcification finder module for finding and locating suspicious areas within said breast region;

a microcalcification detection module for gradient enhancement, contrast enhancement, noise and interference suppression and entropy-based threshholding so as to detect and produce an image of clustered microcalcifications within said suspicious areas; and

a microcalcification classification module for classifying said clustered microcalcifications using a shape cognitron.