WO2010020921A2

WO2010020921A2 - Blanking of image regions

Info

Publication number: WO2010020921A2
Application number: PCT/IB2009/053588
Authority: WO
Inventors: Iwo W. O. Serlie; Dolph Martherus
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2008-08-20
Filing date: 2009-08-14
Publication date: 2010-02-25
Also published as: WO2010020921A3

Abstract

A method of masking part of an image for display comprises assigning (506) pixel values to respective pixels outside a region of interest (2) to obtain a masked region (3), the pixel values being determined according to a mask function, the mask function being dependent on a distance of a position of a respective pixel to the region of interest (2), for obtaining a smooth transition region (5) in the masked region (3) at a boundary (6) of the masked region towards the region of interest (2). The mask function may be a darkening filter. The pixels in the transition region (5) are assigned brighter pixel values at pixel locations closer to the region of interest and darker pixel values at pixel locations further away from the region of interest.

Description

Blanking of image regions

FIELD OF THE INVENTION

The invention relates to a method, system, and computer program product for masking part of an image for display.

BACKGROUND OF THE INVENTION

In mammography, a significant part of the image corresponds to non-tissue, more particularly, air. This air may result in a dark background when the image is displayed. To improve interpretation of such images, they may be subjected to several image manipulation operations. For example, images may be inverted, which means that dark image areas become bright, and bright image areas become dark. A problem occurs when manipulation of the image leads to transformation of dark background values into bright background values. A very bright background region in the image can be dazzling, and thereby severely hinder interpretation of the image by a radiologist.

To remove the bright background region, it may be masked off by converting the bright background region to a uniform density value, as disclosed in EP0523771B1. However the resulting images are not always satisfying.

SUMMARY OF THE INVENTION

It would be advantageous to provide improved masking of a part of an image for display. To better address this concern, in a first aspect of the invention, a method is presented that comprises identifying a region of interest in the image, and assigning pixel values to respective pixels outside the region of interest to obtain a masked region, the pixel values being determined according to a mask function, the mask function being dependent on a distance of a position of a respective pixel to the region of interest, for obtaining a smooth transition region in the masked region at a boundary of the masked region towards the region of interest.

The smooth transition improves the evaluation of the image, because the smooth transition allows detecting any irregularities in the region of interest more easily. In particular, irregularities close to the border of the masked region are detected more easily. Especially when a user is looking for irregularities close to the border of the masked region, this would be hindered by any hard transition between the masked region and the non- masked region, which would introduce an undesirably high local contrast. By using a smooth transition instead of a hard transition, the evaluation of the image by a human observer is made easier.

In an embodiment, the mask function is a darkening filter which is also dependent on an existing pixel value of the respective pixel. This way, at least some of the information of the original image is preserved in the masked region. In an embodiment, the pixels in the transition region (5) are assigned brighter pixel values at pixel locations closer to the region of interest and darker pixel values at pixel locations further away from the region of interest. The transition region thus provides a smooth transition from bright pixels in the region of interest towards darker pixels in the remainder of the masked region. In an embodiment, the transition region extends from the boundary of the masked region to approximately 0.5 to 1 cm from the boundary of the masked region. Experiments have shown that this may provide an image which can be evaluated efficiently by a clinician.

An embodiment comprises preserving an annotation in the masked region of the image. Annotations are useful for identifying the correct images and for providing additional information. It is advantageous to preserve such annotations also in the case of masking part of an image.

In an embodiment, the image comprises a mammogram and the region of interest comprises a depiction of a breast. Mammography is a field of radiology in which the problem of a dazzling background may occur and in which it is necessary to evaluate the images up to close to the skin surface. Consequently a smooth transition from the skin surface towards the masked region helps to diagnose the mammograms in a reliable and/or efficient way. In an embodiment, a boundary of the region of interest corresponds to a skin surface. This refers to an application of medical imaging in which the problem of a dazzling background may occur, for example the part of the image not showing any tissue may contain a bright background which may be masked. In an embodiment, the step of assigning comprises darkening a bright region of the image. However, this is not a limitation. For example, it is also possible to provide a masking operation which brightens a dark region of the image. In such a case, the transition region may provide a smooth transition from a relatively dark region of interest towards a brighter masked region, for example. In an embodiment, the step of assigning comprises assigning pixel values corresponding to a medium brightness to a majority of the pixels in the bright region. Medium brightness masks may be more pleasant and/or efficient to evaluate than fully dark or fully bright masked regions. For example, medium brightness corresponds to an average brightness in the region of interest. An embodiment comprises establishing a first mean and a first variance of pixel values in the region of interest; establishing a second mean and a second variance of pixel values in a corresponding region of interest in another image; setting a window level and a window width of the image for adapting the first mean to the second mean and adapting the first variance to the second variance; displaying the image taking into account the window level and/or window width; and displaying the second image.

This helps to compare the first image and the second image, for example in a follow-up study. Differences in tissue structure may become easier to identify when the window level and/or window width are set in this way. In a further aspect of the invention, a system for masking part of an image for display is provided, the system comprising identifying means for identifying a region of interest in the image, and assignment means for assigning pixel values to respective pixels outside the region of interest to obtain a masked region, the pixel values being determined according to a mask function, the mask function being dependent on a distance of a position of a respective pixel to the region of interest, for obtaining a smooth transition region in the masked region at a boundary of the masked region towards the region of interest.

In a further aspect of the invention, a medical imaging workstation comprising the system of the invention is provided.

In a further aspect of the invention, a medical image acquisition apparatus is provided, the acquisition apparatus comprising a sensor for obtaining medical image data, and the system set forth for masking part of an image acquired by means of the sensor.

In a further aspect of the invention, a computer program product is provided comprising instructions for causing a processor system to perform the method set forth.

It will be appreciated by those skilled in the art that two or more of the above- mentioned embodiments, implementations, and/or aspects of the invention may be combined in any way deemed useful. Modifications and variations of the image acquisition apparatus, of the workstation, of the system, and/or of the computer program product, which correspond to the described modifications and variations of the system, can be carried out by a person skilled in the art on the basis of the present description. A person skilled in the art will appreciate that the method may be applied to multidimensional image data, e.g., to 2-dimensional (2-D), 3-dimensional (3-D) or 4- dimensional (4-D) images, acquired by various acquisition modalities such as, but not limited to, standard X-ray Imaging, Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Ultrasound (US), Positron Emission Tomography (PET), Single Photon Emission Computed Tomography (SPECT), and Nuclear Medicine (NM).

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of the invention will be further elucidated and described with reference to the drawing, in which Fig. 1 shows a diagram representing a mammogram;

Fig. 2 shows a diagram representing a partially masked mammogram; Fig. 3 shows a diagram representing a partially masked mammogram with a smooth transition;

Fig. 4 illustrates a transition function; Fig. 5 illustrates a method of masking part of an image for display; and

Fig. 6 illustrates a system for masking part of an image for display.

DETAILED DESCRIPTION OF EMBODIMENTS

In (digital) mammography, it is important to inspect gray- values up to the skin: up to the outer part of the region of interest. Inverse mapping is applied in (digital) mammography for improved interpretation of tissue.

Inverse mapping in the mammography image may be applied to the full image. However, the bright white background, which may be the result of applying this inverse mapping, may be dazzling to the person who interprets the image. This dazzling may be so severe that the person chooses to use the non- inverted image, which has a black background. For example, the person is a radiographer who desires to establish a diagnosis based on the image.

A solution is to remove the bright background by applying a dark mask to the portion of the image not showing the breast. To that end, the boundary of the breast is identified in the image, and the mask is applied up to the boundary of the breast. However, if an error is made in the detection of the boundary of the breast, clinical information may be lost. And even if the boundary is detected correctly, it still remains difficult to evaluate the image, in particular near the boundary, because of the high contrast edge at the boundary of the mask. A smooth transition between the masked region and the non-masked region allows the observer to identify an error in the detection of the boundary of the breast.

In mammography, diagnostic information is captured in small changes in grey value up to the edge of the object. Blanking pixels outside the area of interest to a low value may (1) hinder the interpretation of small changes due to the introduction of a huge change in grey value, or (2) even damage the gray values at the edge such that information is lost.

Fig. 1 illustrates diagrammatically an x-ray image 1, in particular a mammogram. The image 1 comprises a region of interest 2 which captures a breast and a background region 7. Also, the image contains annotations 4. The background region 7 may consist of a very bright area. Fig. 2 illustrates diagrammatically an x-ray image 1 containing a region of interest 2, a masked region 3, and annotations 4. The background 7 has been masked using a dark color to obtain the masked region 3. Also, the annotations 4 have been made darker, although they are still readable. Because of the sharp transition between the region of interest 2 and the masked region 3, it is difficult to assess the image in the region of interest 2 near the masked background area 3.

Fig. 3 illustrates diagrammatically an x-ray image 1 similar to the x-ray image of Fig. 1. The x-ray image 1 of Fig. 3 contains a region of interest 2, a masked region 3, annotations 4, and a transition region 5 at the side of the masked region 3 towards the region of interest 2. Although not visible in the drawing, the transition region 5 provides a smooth transition in the masked region 3 towards the region of interest 2. The intensities in the transition region 5 are relatively bright close to the region of interest 2 and are relatively dark close to the background region 3. This way, a high contrast edge between the background region 3 and the region of interest 2 does not occur.

It was found that a transition region with a width of about 0.5 to 1 cm may be particularly effective for mammography, although other widths are equally possible.

Fig. 4 illustrates an example of a transition function 10 that describes the transition from foreground to background. In the graph shown in Fig. 4, the horizontal axis (x) indicates the location along an arbitrary line in an x-ray image, the vertical axis (v) indicates the value of a parameter which is used in a masking function, and the dashed vertical line at b indicates the boundary of the region of interest. To the right of the boundary b, it can be seen that the transition function 10 gradually becomes smaller and converges to a fixed background value at 11. In this case, the transition region extends from the boundary b of the region of interest to the point 11 where the transition function has converged to a fixed value. The value of the transition function 10 at a pixel p outside the region of interest may be determined by the shortest distance of the pixel p to the boundary of the region of interest. This function may be defined such that the aforementioned interpretation problems do not occur. To that end, the function 10 is preferably a continuous function with preferably a continuous first derivative. For example, a function resembling an arccosine function may be used.

An advantage of the method set forth is that interpretation of small changes in grey value up to the edge of the region of interest is not hindered by sudden changes introduced by the masking. Also, it is possible to enable the user to detect a potential damaging of grey values at the edge of the region of interest by making the transition more gradual, such that information is not lost at the boundary of the region of interest.

The result of applying any of the described methods is that small changes in grey value at the outer part of the region of interest are suitable for inspection, allowing maximum benefit of improved interpretation by manipulation of data values.

Fig. 5 illustrates a process of masking part of an image for display. The process starts at step 501 of identifying a region of interest in the image. This may be realized by means of a segmentation algorithm. Such a segmentation algorithm is known by itself. It is also possible to let a user outline the extent of the region of interest, as is described in EP 0 523 771 Bl.

In mammography, the region of interest contains tissue and the background region contains air. An example way of segmenting the region of interest comprises applying a pixel value threshold in combination with a connectivity constraint. However, more advanced segmentation algorithms are known in the art. Alternatively, step 501 may comprise receiving the segmentation from an external source such as a PACS database.

The process then proceeds to step 502 of identifying a pixel for processing. The pixels may be processed in a particular order (for example from top left to bottom right, row by row); in step 502 a yet unprocessed pixel is selected.

In step 503, it is tested, based on the segmentation, whether the pixel is part of the region of interest, the transition region, or the remainder of the background region. The pixel is part of the transition region if it is outside the region of interest and the distance d of the pixel to the region of interest is smaller than a predetermined threshold. If the pixel is part of the region of interest, the process proceeds to step 504. In step 504, no action is taken. Optionally, some image processing operation may take place in step 504, such as image enhancement or window width/window level setting adjustment. If the pixel is part of the background region but not of the transition region, the process proceeds to step 505. In step 505, the pixel value is modified. For example, the pixel value is set to a fixed, predetermined background grey value V. Alternatively, the pixel value is multiplied by a fixed, predetermined multiplication factor F which reduces the intensity of the pixel. If it is detected in step 505 that the pixel is part of an image region containing annotations, some special processing may be performed to ensure that the annotation remains readable, while not being overly bright.

If the pixel is part of the transition region, the process proceeds to step 506. In step 506, the pixel is modified based on the distance d of the pixel to the region of interest. For example, the pixel is assigned a value according to a mask function (V - d + W ^• (w - d)) I w , wherein V is the predetermined background grey value, d is the distance to the nearest pixel in the region of interest, W is the pixel value of the nearest pixel in the region of interest, and w is the width of the transition region. For example, the pixel value is a grey value or a color value of the pixel. Alternatively, the pixel value is multiplied by a multiplication factor (w - d + F ^■ d) I w , wherein F is the predetermined multiplication factor F which reduces the intensity of the pixel. The mask function then becomes m(p, d) = p ^■ (w — d + F ^■ d) / w , wherein m denotes the mask function and p denotes the original pixel value of the pixel. Other transition functions and mask functions may be apparent to the skilled person. The example transformations are linear in the distance d; to use a non-linear transition, d can be replaced with a function f(d), wherein f(d) may be defined according to the graph of transition function 10 in Fig. 4.

Instead of using different process steps for the transition region and the remainder of the masked region, it is also possible to use a single formula for the entire masked region including the transition region, which formula converges to a particular gray value (or multiplication factor) for pixels further away from the region of interest. In step 507, it is checked whether all pixels have been processed. If not, the process continues from step 502. Thus, pixel values are assigned to respective pixels outside the region of interest according to a mask function, the mask function being dependent on a distance of a pixel to the region of interest, for obtaining a smooth transition between the masked region and the non-masked region. If in step 507, it is determined that all pixels have been processed, the process may proceed to step 508 in which the resulting image is displayed.

The mask function may be a darkening filter which is also dependent on an existing pixel value of the respective pixel. This can be applied to the transition region, but also the remainder of the mask region, to remove the dazzling effect, while preserving image information in the mask region. This may also be applied to a region of the image containing annotations.

The pixels in the transition region (5) may be assigned brighter pixel values at pixel locations closer to the region of interest and darker pixel values at pixel locations further away from the region o f interest.

In mammography image visualization, the region of interest may comprise a breast and the boundary of the region of interest may correspond to the skin surface of the breast.

The step of assigning results in for example darkening a bright region of the image. This helps to reduce the dazzling effect. Preferably, the majority of the pixel values in the mask region are assigned a medium brightness, causing the average gray level of the mask to differ not too much from the average gray level of the region of interest.

Fig. 6 illustrates a system for masking part of an image for display. An image is received via input 601, for example via a digital network connection, and stored in storage means 602 (random access memory or hard disk, for example). A means 603 is arranged for identifying a region of interest in the image, a definition of the region of interest is stored in the storage means 602. An assignment unit 604 is provided for assigning pixel values to respective pixels outside the region of interest according to a mask function, the mask function being dependent on a distance of a pixel to the region of interest, for obtaining a smooth transition at an edge of the masked region towards a non-masked region. Thus, the assignment unit 604 effectively creates a masked image and stores the masked image in the storage means 602. An output unit 605 is provided for sending a signal to an external monitor for displaying the masked image on the external display.

It will be understood that all or part of the blocks 603, 604, 605 may be implemented by means of software modules, or by means of dedicated electronic circuitry. If any of the blocks 603,604,605 are implemented as software, the software code may be stored in a memory of a computer, and be executed by means of a processor (not shown).

The system for masking part of an image may be included in a medical imaging apparatus, for example an x-ray mammography system or a general x-ray system, or a CT scanner. Such a medical imaging apparatus comprises a sensor for obtaining medical image data. These medical image data are subsequently processed by the system for masking part of the image.

Although in the above embodiments, the emphasis has been on mammography, it will be understood that the techniques disclosed may also be applied to other kinds of images. Accordingly, the region of interest is not limited to a breast, but can correspond to any depicted object. For example any kind of x-ray images, such as for example chest x-ray images, may be processed and displayed using the techniques described herein to advantage. Medical images originating from other modalities (such as MR, CT, and Ultrasound) may also benefit from the invention. For example, CT slices showing a tissue portion and an air portion can be processed as set forth, masking the air portion with a smooth transition between tissue portion and masked portion. The techniques can be applied to 2D images as well as to 3D images. It is also possible that the region of interest depicts a particular organ, and the remaining portions of any depicted tissues are part of the background.

The techniques can also be applied in the field of non-destructive testing. For example, an x-ray image of an object may be used when looking for small cracks in the object. If the x-ray image has a very bright background, it may be difficult to view any small cracks which may be present in the image near the edge of the object (close to the bright background). In such a case, the object may correspond to the region of interest. The bright background may be masked using a mask function which is dependent on a distance of a position of a respective pixel to the object. The mask function is chosen such that a smooth transition region is obtained between the object and the masked region. The use of x-ray is not a limitation. Window width and window level may be adjusted to compare a current exam to one or more prior exams. This helps to facilitate the inspection of current medical images with prior medical images, for example in a follow-up study.

Normally, when reading current images in association with prior images, the reader has to manually adjust window width and window level settings to be able to compare the contents of images. This is necessary because images may be obtained using different acquisition devices and an image is stored using local viewing conditions (pixel- values may not be calibrated, for example). US 5305204 describes automatic window width and window level adjustments to display a single image optimally. In mammography screening, current mammograms are viewed in association with prior mammograms to obtain a more accurate prognosis and decrease the false negative rate. It may be a problem that prior and current images may be obtained using different acquisition devices or different imaging settings and window width and window level settings have to be adjusted manually to best compare them. This problem is particularly pertinent in the case where tissue densities are not calibrated such as with CT: fatty tissue and glandular tissue do not have unique pixel values. The image pixel values typically appear bright for glandular tissue (more dense) and dark for fatty tissue (less dense).

The problem with the manual adjustments of window width and window level is that it may not lead to qualitatively comparable images. In addition, in mammography screening, the reader does not have the time to manually adjust window width and window level settings.

One way to deal with this is to automatically compare average grey values of all pixels in the image. However, advantageously only object-pixels should be analyzed, separately from the background (dark pixels and tags that indicate the type of image). The object (or region of interest) may be detected in the way set forth.

Advantageously, window width and window level settings of prior images are set automatically to match with current images. An advantage thereof is that prior and current images are qualitatively comparable. Moreover, the reader can focus on the diagnostic task without the time-consuming manual labor associated with setting window width and window level.

The following steps provide an implementation-example to automatically display current and prior exams (of matching series: CC, MLO) with comparable window width and window level settings. First, the object of interest (or region of interest) is detected in the prior and current images. In the mammography application, this refers to the glandular and fatty tissue of the breast in an image. Second, considering only the selected pixels of the object of interest, pixel values of current and prior exams are described using the average grey value and variance. Finally, the window width and window level of displayed pixel values of the prior image are set such that the displayed average grey value and variance of the selected pixels in the prior and current images are identical. Advantageously, the current image is leading, because there may be just one current image and possibly more than one prior image. In an embodiment, a region of interest is identified in a first image and in a second image. A first mean and a first variance of pixel values in the region of interest are established. A second mean and a second variance of pixel values are established in the region of interest in the second image. A window level and a window width of the first image are set for adapting the first mean to the second mean and the first variance to the second variance. The window level may be adapted to match the means. Next, the window width may be adapted to match the variances. Next, the first image is displayed using the window level and/or window width; also, the second image is displayed. Alternatively, the window level and window width of the second image may be adapted to match the mean and the variance of the first image in the region of interest.

It will be appreciated that the invention also extends to computer programs, particularly computer programs on or in a carrier, adapted for putting the invention into practice. The program may be in the form of source code, object code, a code intermediate source and object code such as a partially compiled form, or in any other form suitable for use in the implementation of the method according to the invention. It will also be appreciated that such a program may have many different architectural designs. For example, a program code implementing the functionality of the method or system according to the invention may be subdivided into one or more subroutines. Many different ways to distribute the functionality among these subroutines will be apparent to the skilled person. The subroutines may be stored together in one executable file to form a self-contained program. Such an executable file may comprise computer executable instructions, for example processor instructions and/or interpreter instructions (e.g. Java interpreter instructions). Alternatively, one or more or all of the subroutines may be stored in at least one external library file and linked with a main program either statically or dynamically, e.g. at run-time. The main program contains at least one call to at least one of the subroutines. Also, the subroutines may comprise function calls to each other. An embodiment relating to a computer program product comprises computer executable instructions corresponding to each of the processing steps of at least one of the methods set forth. These instructions may be subdivided into subroutines and/or be stored in one or more files that may be linked statically or dynamically. Another embodiment relating to a computer program product comprises computer executable instructions corresponding to each of the means of at least one of the systems and/or products set forth. These instructions may be subdivided into subroutines and/or be stored in one or more files that may be linked statically or dynamically. The carrier of a computer program may be any entity or device capable of carrying the program. For example, the carrier may include a storage medium, such as a ROM, for example a CD ROM or a semiconductor ROM, or a magnetic recording medium, for example a floppy disc or hard disk. Further, the carrier may be a transmissible carrier such as an electrical or optical signal, which may be conveyed via electrical or optical cable or by radio or other means. When the program is embodied in such a signal, the carrier may be constituted by such a cable or other device or means. Alternatively, the carrier may be an integrated circuit in which the program is embedded, the integrated circuit being adapted for performing, or for use in the performance of, the relevant method. It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb "comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

CLAIMS:

1. A method of masking part of an image for display, comprising identifying a region of interest (2) in the image, and assigning (506) pixel values to respective pixels outside the region of interest (2) to obtain a masked region (3), the pixel values being determined according to a mask function, the mask function being dependent on a distance of a position of a respective pixel to the region of interest (2), for obtaining a smooth transition region (5) in the masked region (3) at a boundary (6) of the masked region towards the region of interest (2).

2. The method according to claim 1, wherein the mask function is a darkening filter which is also dependent on an existing pixel value of the respective pixel.

3. The method according to claim 1, wherein the pixels in the transition region (5) are assigned brighter pixel values at pixel locations closer to the region of interest and darker pixel values at pixel locations further away from the region of interest.

4. The method according to claim 1, wherein the transition region extends from the boundary of the masked region to approximately 0.5 to 1 cm from the boundary of the masked region.

5. The method according to claim 2, further comprising preserving an annotation in the masked region of the image.

6. The method according to claim 1, wherein the image comprises a mammogram and the region of interest comprises a depiction of a breast.

7. The method according to claim 1, wherein a boundary of the region of interest corresponds to a skin surface.

8. The method according to claim 1, wherein the step of assigning comprises darkening a bright region of the image.

9. The method according to claim 8, wherein the step of assigning comprises assigning pixel values corresponding to a medium brightness to a majority of the pixels in the bright region.

10. The method according to claim 1, comprising establishing a first mean and a first variance of pixel values in the region of interest; establishing a second mean and a second variance of pixel values in a corresponding region of interest in another image; setting a window level and a window width of the image for adapting the first mean to the second mean and adapting the first variance to the second variance; displaying the image taking into account the window level and/or window width; and displaying the second image.

11. A system for masking part of an image for display, comprising identifying means (603) for identifying a region of interest (2) in the image, and assignment means (604) for assigning (506) pixel values to respective pixels outside the region of interest (2) to obtain a masked region (3), the pixel values being determined according to a mask function, the mask function being dependent on a distance of a position of a respective pixel to the region of interest (2), for obtaining a smooth transition region (5) in the masked region (3) at a boundary (6) of the masked region towards the region of interest (2).

12. A medical imaging workstation comprising a system according to claim 11.

13. A medical image acquisition apparatus comprising a sensor for obtaining medical image data; and a system according to claim 11 for masking part of an image acquired by means of the sensor.

14. A computer program product comprising instructions for causing a processor system to perform the method according to claim 1.