WO2014050044A1

WO2014050044A1 - Image processing device, method, and program

Info

Publication number: WO2014050044A1
Application number: PCT/JP2013/005556
Authority: WO
Inventors: 元中李
Original assignee: 富士フイルム株式会社
Priority date: 2012-09-27
Filing date: 2013-09-20
Publication date: 2014-04-03
Also published as: EP2901933A1; BR112015006523A2; EP2901933A4; JP5836908B2; CA2886092A1; US20150193943A1; CN104717925A; JP2014064835A; IN2015DN02613A

Abstract

[Problem] To shorten the processing time required when extracting a region from an image using a graph cut method, and reduce the amount of memory used for computation. [Solution] An image acquisition unit (10) acquires a plurality of CT images from an X-ray CT device (2) and generates a three-dimensional image (M0). A low-resolution image generation unit (12) converts the resolution of the three-dimensional image (M0) into multiple resolutions and generates a low-resolution image (ML). A first extraction unit (14) uses a graph cut method to extract a specific region such as a liver region from the low-resolution image (ML). A contour region setting unit (16) sets the contour of the liver region extracted from the low-resolution image (ML) in the three-dimensional image (M0), and sets a contour region including said contour in the three-dimensional image (M0). A second extraction unit (18) extracts the contour of the liver region from the contour region, and extracts the liver region from the three-dimensional image (M0).

Description

Image processing apparatus and method, and program

The present invention relates to an image processing apparatus and method for extracting a region from an image using a graph cut method, and a program.

In recent years, high-quality, high-resolution three-dimensional images have come to be used for image diagnosis due to advances in medical equipment (for example, multi-detector CT). Here, since a three-dimensional image is composed of a large number of two-dimensional images and has a large amount of information, it may take time for a doctor to find and diagnose a desired observation site. Therefore, the organ of interest is recognized, and the organ of interest is extracted from a three-dimensional image including the organ of interest using a method such as a maximum value projection method (MIP method) and a minimum value projection method (MinIP method). By performing MIP display, volume rendering (VR) display of 3D images, and CPR (Curved Planer Reconstruction) display, the visibility of the entire organ and lesions can be improved and diagnosis can be made more efficient. Things are going on.

In addition, various methods for extracting an organ from a three-dimensional image have been proposed. For example, Patent Document 1 proposes a method of extracting a specific region of an organ from a number of two-dimensional images constituting a three-dimensional image, and stacking the extracted regions to generate a three-dimensional image of the specific region. Yes.

However, since the method described in Patent Document 1 extracts a region from a two-dimensional image, the extraction range may be slightly different in each two-dimensional image. The extraction accuracy of the area is not so good.

On the other hand, a graph cut method is known as a method for extracting a desired region from an image (see Non-Patent Document 1). The graph cut method includes a node Nij representing each pixel in the image, nodes S and T representing whether each pixel is a target region or a background region, an n-link which is a link connecting nodes of adjacent pixels, and each A graph composed of s-link and t-link, which is a link connecting a node Nij representing a pixel, a node S representing a target region, and a node T representing a background region, is created, and each pixel is a pixel of the target region Or a pixel in the background area is represented by the thickness (value size) of s-link, t-link and n-link, respectively, and the target area and the background area depending on the link thickness obtained as a result of the operation This is a technique for extracting a target region from an image. By using such a graph cut method, it is possible to accurately extract regions such as the heart, lungs, and liver included in the medical three-dimensional image.

JP 2003-10172 A

However, when extracting a region from an image using the graph cut method, the larger the size of the image to be calculated, the greater the number of pixels and the number of links, and thus the memory and processing time required for processing increase. In particular, when the graph cut method is applied to a three-dimensional image, the number of pixels and the number of links increase exponentially compared to the two-dimensional image. For this reason, it takes a very long time to extract a desired region using the graph cut method. In addition, when a low-spec computer with a small memory capacity is used, it may be impossible to extract a region using the graph cut method.

The present invention has been made in view of the above circumstances, and an object thereof is to shorten the processing time required for extracting an area from an image using the graph cut method and to reduce the amount of memory for calculation. .

An image processing apparatus according to the present invention is an image processing apparatus that extracts a specific area from a processing target image by a graph cut method,
First extraction means for generating a low-resolution image of the processing target image and extracting a specific region from the low-resolution image by a graph cut method;
A contour region setting means for setting a contour region including the contour of the specific region in the processing target image to the processing target image based on the extraction result of the specific region;
And a second extraction means for extracting a region corresponding to the specific region from the contour region by a graph cut method.

In the image processing apparatus according to the present invention, the contour region setting unit may be a unit that determines the size of the contour region based on the difference in resolution between the low-resolution image and the processing target image.

In the image processing apparatus according to the present invention, the contour region setting means may be a means for setting a contour region by erosion processing and dilation processing of morphology processing.

In the image processing apparatus according to the present invention, the second extraction means increases the value of s-link in the graph cut method for the pixels inside the contour of the specific region in the contour region, and Means for increasing the value of t-link in the graph cut method for pixels outside the contour may be used.

An image processing method according to the present invention is an image processing method for extracting a specific region from a processing target image by a graph cut method,
Generate a low-resolution image of the image to be processed, extract a specific area from the low-resolution image by the graph cut method,
Based on the extraction result of the specific region, the contour region including the contour of the specific region in the processing target image is set as the processing target image,
A region corresponding to the specific region is extracted from the contour region by a graph cut method.

The image processing method according to the present invention may be provided as a program for causing a computer to execute the image processing method.

According to the present invention, a low-resolution image of the processing target image is generated, and a specific region is extracted from the low-resolution image by the graph cut method. Here, since the number of pixels of the low resolution image is smaller than that of the processing target image, the amount of calculation and the memory used can be reduced, but the accuracy of the region extraction is not so good. For this reason, according to the present invention, based on the extraction result of the specific area, a contour area that includes a contour in the specific area in the processing target image is set as the processing target image, and the contour area corresponds to the specific area by the graph cut method. Regions are extracted. As described above, in the present invention, the specific area is extracted by applying the graph cut method only to the contour region of the low-resolution image and the processing target image, so the graph cutting method is applied to the entire processing target image. Compared to the case, the processing time can be greatly shortened, and the amount of memory for calculation can be greatly reduced.

1 is a schematic block diagram showing the configuration of an image processing apparatus according to an embodiment of the present invention. Diagram for explaining multi-resolution conversion Diagram for explaining the graph cut method Diagram for explaining area division by graph cut method Diagram showing the outline of the liver region extracted from the low-resolution image The figure which shows the state which set the outline of the liver area | region extracted from the low-resolution image to the three-dimensional image Diagram showing the structural elements of morphology processing The figure which shows the outline area | region set to the three-dimensional image M0 The figure which shows the outline of the liver area | region extracted from the three-dimensional image A flowchart showing processing performed in the present embodiment Diagram showing extracted liver region

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of an image processing apparatus according to an embodiment of the present invention. The configuration of the image processing apparatus 1 shown in FIG. 1 is realized by executing a program read in an auxiliary storage device (not shown) on a computer (for example, a personal computer). Further, this program is stored in an information storage medium such as a CD-ROM or distributed via a network such as the Internet and installed in a computer.

The image processing apparatus 1 generates a three-dimensional image M0 from a plurality of two-dimensional images captured by, for example, the X-ray CT apparatus 2, and automatically uses a graph cut method to identify a specific area included in the three-dimensional image M0. An image acquisition unit 10, a low-resolution image generation unit 12, a first extraction unit 14, a contour region setting unit 16, a second extraction unit 18, and a display control unit 20 are included. Note that an input unit 22 and a display unit 24 are connected to the image processing apparatus 1. In the present embodiment, the three-dimensional image M0 represents the thorax of the human body, and the specific area is the liver area.

The image acquisition unit 10 acquires, for example, a plurality of CT images (two-dimensional images) captured by the X-ray CT apparatus 2, and generates a three-dimensional image M0 from the plurality of two-dimensional images. The image acquisition unit 10 may acquire not only CT images but also two-dimensional images such as so-called MRI images, RI images, PET images, X-ray images, and the like. Alternatively, the X-ray CT apparatus 2 may generate a three-dimensional image M0, and the image acquisition unit 10 may perform only the process of acquiring the three-dimensional image M0.

The low-resolution image generation unit 12 multi-resolution-converts the three-dimensional image M0 to generate a plurality of three-dimensional multi-resolution images Msi (i = 0 to n) having different resolutions as shown in FIG. Note that i = 0 represents the same resolution as the three-dimensional image M0, and i = n represents the lowest resolution. In the present embodiment, as described later, a specific region is extracted from a low-resolution image, and the resolution of the low-resolution image is determined according to the actual size per pixel of the three-dimensional image M0. For example, when the actual size per pixel of the three-dimensional image M0 is 0.5 mm, the low resolution image generation unit 12 converts the three-dimensional multiresolution image Ms2 having a resolution of 1/4 of the three-dimensional image M0 to the low resolution image ML. Generate as

The first extraction unit 14 divides a region other than the liver region and the liver region of the low resolution image ML using the graph cut method, and extracts the liver region from the low resolution image ML. Specifically, a liver region is set as a target region, and a region other than the liver region is set as a background region, a determination region having a predetermined pixel size is set at all pixel positions in the low resolution image ML, and determination is performed using a graph cut method. The area is divided into a target area and a background area.

In the graph cut method, first, as shown in FIG. 3, nodes Sij, representing nodes Nij representing pixels in the discrimination area, and labels (target areas or background areas in the present embodiment) that each pixel can take, From n-link, which is a link connecting nodes of adjacent pixels, and from s-link and t-link, which is a link connecting node Nij representing each pixel, node S representing a target region, and node T representing a background region Create a composed graph. In order to simplify the description, in FIG. 3, the discrimination area is a 3 × 3 two-dimensional area.

Here, n-link represents the probability that adjacent pixels are pixels in the same region by the thickness of the link, and the value of the probability is the distance between the adjacent pixels and the difference between the pixel values. Can be calculated based on

The s-link connecting the node Nij representing each pixel and the node S representing the target area represents the probability that each pixel is a pixel included in the target area, and the node representing each pixel and the background area The t-link connecting the node T representing the likelihood represents the probability that each pixel is a pixel included in the background area. If the information on whether the pixel is a pixel indicating the target region or the background region is already given, the probability values can be set according to the given information. Is not given, for s-link, the target area is estimated, and a probability value is set based on the histogram of the density distribution of the estimated target area. For t-link, the background area is set. The probability value can be set based on the estimated density distribution histogram of the background region.

Here, in FIG. 3, if the pixels represented by the nodes N11, N12, N21, N22, and N31 are pixels set in the target area, the nodes N11, N12, N21, N22, N31, and the node S The s-link connecting the two nodes becomes thick, and the n-link connecting the nodes N11, N12, N21, N22, and N31 becomes thicker. On the other hand, if the pixels represented by the nodes N13, N23, N32, and N33 are pixels set in the background area, the t-link that connects the nodes N13, N23, N32, and N33 and the node T becomes thicker. The n-link connecting the nodes N13, N23, N32, and N33 becomes thicker.

Since the target area and the background area are mutually exclusive areas, for example, as shown by a broken line in FIG. 4, an appropriate link among s-link, t-link, and n-link is cut to obtain a node S Is separated from the node T, the discrimination area can be divided into the target area and the background area. Here, by performing the cutting so that the sum of the probability values of all the s-links, t-links, and n-links to be cut is minimized, the optimum region division can be performed.

The first extraction unit 14 divides the region of the low resolution image ML as described above, and extracts the liver region that is the target region from the low resolution image ML. FIG. 5 is a diagram showing the outline of the liver region extracted from the low-resolution image. In the present embodiment, the liver region is extracted from the three-dimensional low-resolution image ML, but here, for the sake of explanation, it is extracted from the low-resolution image of one two-dimensional image constituting the three-dimensional image M0. The outline of the liver region is indicated by a solid line.

The contour region setting unit 16 sets a contour region including the contour of the liver region extracted by the first extraction unit 14 in the three-dimensional image M0. FIG. 6 is a diagram illustrating a state in which the outline of the liver region extracted from the low resolution image ML by the first extraction unit 14 is set in the three-dimensional image M0. Here, for the sake of explanation, the outline of the liver region set in one two-dimensional image constituting the three-dimensional image M0 is shown by a solid line. In the present embodiment, the low resolution image ML has a resolution that is ¼ that of the three-dimensional image M0. Therefore, the outline of the liver region extracted from the low resolution image ML is enlarged four times and set to the three-dimensional image M0. Will be. For this reason, the set outline does not completely match the outline of the liver region included in the three-dimensional image M0, and includes irregularities based on the difference in resolution.

The contour region setting unit 16 contracts the contour set in the three-dimensional image M0 inward and expands outward, and sets a region surrounded by the expanded contour and the contracted contour as the contour region E0. Note that the size of the contour region E0 in the width direction (ie, the direction perpendicular to the contour) is based on the size of the low-resolution image ML and the three-dimensional image M0 (the size of the three-dimensional image M0 / the size of the low-resolution image ML). +1) × 2 is determined by calculation. In the present embodiment, since the size of the three-dimensional image M0 / the size of the low resolution image ML = 4, the range size of the contour region E0 is determined to be 10 pixels. The method for determining the size of the contour region E0 is not limited to the above method, and any method can be applied as appropriate.

Here, in the present embodiment, the contraction and expansion of the contour are performed by morphology processing. Specifically, an erosion process for searching for the minimum value in a predetermined width centered on the target pixel on the contour set in the three-dimensional image M0 is performed using a structural element as shown in FIG. Thus, the contour is contracted by one pixel, and the contour is further contracted by performing erosion processing on the contracted contour. Then, by performing such erosion processing four times, the contour set in the three-dimensional image M0 is contracted four pixels inward.

Further, by using a structural element as shown in FIG. 7, the contour is obtained by performing a dilation process for searching for the maximum value in a predetermined width centered on the pixel of interest on the contour set in the three-dimensional image M0. Is expanded by one pixel, and the contour is further expanded by performing dilation processing on the expanded contour. Then, by performing such dilation processing five times, the outline set in the three-dimensional image M0 is expanded outward by five pixels.

Then, the contour region setting unit 16 sets a region surrounded by the expanded contour and the contracted contour as the contour region E0. FIG. 8 is a diagram showing a contour region set in the three-dimensional image M0. Since the contour is shrunk 4 pixels inward by erosion processing and expanded 5 pixels outward by dilation processing, the size in the width direction of the contour region E0 is 10 pixels together with 1 pixel of the contour. ing.

The second extraction unit 18 divides the contour region E0 set in the three-dimensional image M0 into a liver region and a region other than the liver region using a graph cut method, and further, based on the result of the region division, 3 The entire liver region is extracted from the dimension image M0. The region inside the contour in the contour region E0 is likely to be a liver region, and the region outside the contour in the contour region E0 is highly likely to be a background region. For this reason, when the graph cut method is applied to the contour region E0, the value of t-link is increased toward the outside of the set contour in the contour region E0, and s-link is increased toward the inside of the set contour in the contour region E0. Increase the value of. As a result, the contour region E0 can be efficiently and accurately divided into the liver region and the other regions.

The second extraction unit 18 divides the contour region E0 as described above, and extracts the liver region that is the target region from the contour region E0. FIG. 9 is a diagram showing the contour of the liver region extracted from the contour region E0. In the present embodiment, the liver region is extracted from the three-dimensional image M0. However, for the sake of explanation, the outline of the liver region extracted in one two-dimensional image constituting the three-dimensional image M0 is illustrated by a solid line. Is shown. As shown in FIG. 9, the contour of the liver region obtained by dividing the contour region E0 of the three-dimensional image M0 smoothly connects the surfaces of the liver.

And the 2nd extraction part 18 extracts the area | region inside the outline extracted from the outline area | region E0 of the three-dimensional image M0 as a liver area | region.

The display control unit 20 displays the extracted liver region and the like on the display unit 24.

The input unit 22 includes, for example, a keyboard and a mouse, and inputs various instructions from a user such as a radiologist to the image processing apparatus 1.

The display unit 24 is composed of, for example, a liquid crystal display, a CRT display, or the like, and displays an image of the extracted liver region or the like as necessary.

Next, processing performed in the present embodiment will be described. FIG. 10 is a flowchart showing processing performed in the present embodiment. First, the image acquisition unit 10 acquires a plurality of CT images from the X-ray CT apparatus 2 and generates a three-dimensional image M0 (step ST1). Next, the low resolution image generation unit 12 multi-resolution converts the three-dimensional image M0 to generate a low resolution image ML (step ST2), and the first extraction unit 14 extracts a liver region from the low resolution image ML. (Step ST3).

Next, the contour region setting unit 16 sets the contour of the liver region extracted from the low-resolution image ML to the three-dimensional image M0, and performs the erosion processing and the dilation processing as described above, thereby converting the contour region E0 into the three-dimensional image. Set to M0 (step ST4). And the 2nd extraction part 18 extracts the outline of a liver area | region from the outline area | region E0, and extracts a liver area | region from the three-dimensional image M0 (step ST5). Further, the liver region extracted by the display control unit 20 is displayed on the display unit 24 (step ST6), and the process ends.

FIG. 11 is a diagram showing the displayed liver region. As shown in FIG. 11, according to the present embodiment, it can be seen that the liver region is accurately extracted.

As described above, according to the present embodiment, the low-resolution image ML of the three-dimensional image M0 is generated, and a specific region such as a liver region is extracted from the low-resolution image ML by the graph cut method. Here, the low-resolution image ML has a smaller number of pixels than the three-dimensional image M0. For example, when the resolution of the low resolution image ML is 1/4 of the three-dimensional image M0, the number of pixels is 1/64. For this reason, by using the low-resolution image ML, the amount of calculation and the memory used can be reduced, but the accuracy of region extraction is not so good. For this reason, in this embodiment, the contour region E0 including the contour of the liver region extracted from the low-resolution image ML is set as the three-dimensional image M0, and the liver region is extracted from the contour region E0 by the graph cut method. Is. Here, as shown in FIG. 7, the size of the contour region E0 is significantly smaller than that of the three-dimensional image M0. As described above, in the present embodiment, the liver region is extracted by applying the graph cut method only to the low-resolution image ML of the three-dimensional image M0 and the contour region E0 of the three-dimensional image M0. Compared with the case where the graph cut method is applied to the entire image M0, the processing time can be greatly shortened, and the amount of memory for calculation can be greatly reduced.
In the above embodiment, the liver region is extracted from the medical three-dimensional image M0, but the region to be extracted is not limited to this, and the brain, heart, lung field, pancreas, spleen, kidney, By applying the present invention when extracting regions of various structures included in a medical three-dimensional image such as blood vessels, it is possible to reduce the processing amount and processing time of computation.

In the above embodiment, the liver region is extracted by applying the result of region extraction in the low-resolution image ML having a resolution of 1/4 of the three-dimensional image M0 to the three-dimensional image M0, but the low-resolution image ML is extracted. Is used to extract a liver region by setting a contour region in a low-resolution image having a resolution of 1/2 that of the three-dimensional image M0, and further to a low resolution having a resolution of 1/2 that of the three-dimensional image M0. The region extraction result in the image may be applied to the three-dimensional image M0. That is, processing of setting a contour region for a resolution image having a higher one-step resolution than that of the low-resolution image and extracting a liver region from the resolution image having a higher one-step resolution using the region extraction result in the low-resolution image The liver region may be extracted from the three-dimensional image M0 by repeating up to the target three-dimensional image M0.

In the above embodiment, a medical three-dimensional image is a processing target, but a medical two-dimensional image may be a processing target. Of course, the present invention can be applied not only to extracting a medical image but also to extracting an area such as a person from an image acquired using a digital camera or the like. In particular, since recent digital cameras have a large number of pixels in an acquired image, a large amount of computation is required when extracting a region such as a person using the graph cut method. The processing amount and processing time can be greatly reduced.

Also, the present invention can be applied when extracting a region from a moving image. Since a moving image is composed of a plurality of frames, it is conceivable to extract a region from each frame. However, the image included in each frame has poor image quality, and the region cannot be extracted with high accuracy. Here, the moving image can be regarded as a three-dimensional image in which a plurality of frames are arranged along the time axis. In this way, when extracting a region from a moving image regarded as a three-dimensional image using the graph cut method, by applying the present invention, it is possible to greatly reduce the processing amount and processing time of calculation, In addition, the region can be extracted from the moving image with high accuracy.

Claims

An image processing apparatus that extracts a specific area from a processing target image by a graph cut method,
First extraction means for generating a low resolution image of the processing target image and extracting the specific region from the low resolution image by the graph cut method;
A contour region setting means for setting a contour region including the contour of the specific region in the processing target image in the processing target image based on the extraction result of the specific region;
An image processing apparatus comprising: a second extraction unit configured to extract a region corresponding to the specific region from the contour region by the graph cut method.
2. The image processing apparatus according to claim 1, wherein the contour region setting unit is a unit that determines the size of the contour region based on a resolution difference between the low-resolution image and the processing target image.
3. The image processing apparatus according to claim 1, wherein the contour region setting means is a means for setting the contour region by erosion processing and dilation processing of morphology processing.
The second extraction means increases the value of s-link in the graph cut method for pixels inside the contour of the specific region in the contour region, and is outside the contour of the specific region in the contour region. 4. The image processing apparatus according to claim 1, wherein the image processing apparatus is a means for increasing a t-link value in the graph cut method for a pixel.
An image processing method for extracting a specific area from a processing target image by a graph cut method,
Generating a low-resolution image of the processing target image, extracting the specific region from the low-resolution image by the graph cut method,
Based on the extraction result of the specific region, a contour region including the contour of the specific region in the processing target image is set in the processing target image,
An image processing method, wherein an area corresponding to the specific area is extracted from the outline area by the graph cut method.
A program for causing a computer to execute an image processing method for extracting a specific area from a processing target image by a graph cut method,
Generating a low-resolution image of the processing target image, and extracting the specific region from the low-resolution image by the graph cut method;
Based on the extraction result of the specific region, a procedure for setting a contour region including the contour of the specific region in the processing target image to the processing target image;
A program for causing a computer to execute a procedure for extracting an area corresponding to the specific area from the outline area by the graph cut method.