JP6643821B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing device that performs image processing on a medical image, an image processing system, an image processing method, and an image processing program.

  In recent years, many medical images have been used in medical sites. Along with this, the workload of so-called image reading, in which a doctor makes a diagnosis based on medical images and examines a treatment policy, is increasing. Therefore, there is an increasing expectation for a technology related to computer aided diagnosis (CAD) in which information obtained by analyzing a medical image with a computer is used to assist diagnosis. In such technology, the accuracy of identifying and extracting an analysis target region such as a tumor (extracting an object) is important in order to analyze the characteristics of a lesion or distinguish between benign and malignant.

  Patent Literature 1 discloses that a radius (expected radius) of a circle approximating the shape of a massive object is obtained from an input image, and the object is extracted based on an evaluation function using the expected radius.

Japanese Patent No. 5231007

  The shape of the lesion area to be analyzed, such as a tumor, shows a regular mass (spherical or circular), and depending on the type of lesion and the state of contact with surrounding organs, it may be ellipsoidal, lobulated, filamentous, etc. There exists a thing which shows the shape of. Further, there is an object having an irregular shape having the features of a plurality of shapes as described above. When trying to extract an object by approximating the shape of the object with a circle, when the object has an irregular shape, it is difficult to accurately determine the outer shape, and the extraction accuracy may be reduced.

An image processing apparatus according to an embodiment of the present invention includes an image acquisition unit configured to acquire a medical image, and an image acquisition unit that performs image processing on a plurality of different regions in the object included in the medical image to include the medical image in the object. A plurality of partial shapes that approximate the shape of the region to be estimated, and a rough shape obtaining unit that obtains a shape obtained by combining the plurality of partial shapes obtained by the estimation as a rough shape that approximates the shape of the target object; Extracting means for extracting a region of the object using a shape as a condition for a process of extracting the region of the object .

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to extract the external shape of an object more accurately.

FIG. 2 is a diagram illustrating a device configuration of the image processing system according to the first embodiment. FIG. 2 is a diagram illustrating a functional configuration of the image processing system according to the first embodiment. FIG. 2 is a diagram illustrating a processing procedure of the image processing apparatus 100 according to the first embodiment. FIG. 5 is a diagram illustrating estimation of a partial shape according to the first embodiment. FIG. 5 is a diagram illustrating estimation of a partial shape according to the first embodiment. It is a figure explaining combination of a partial shape concerning a 1st embodiment. It is a figure explaining extraction of an object concerning a 1st embodiment. It is a figure explaining combination of partial shape concerning modification 4 of a 1st embodiment. It is a figure showing the functional composition of the image processing system concerning a 2nd embodiment. It is a figure showing the processing procedure of the image processing device concerning a 2nd embodiment. It is a figure explaining correction of a schematic shape concerning a 2nd embodiment. It is a figure explaining correction of a schematic shape concerning a 2nd embodiment. It is a figure showing the functional composition of the image processing system concerning a 3rd embodiment. FIG. 13 is a diagram illustrating a processing procedure of the image processing apparatus according to the third embodiment. It is a figure explaining correction of a seed point concerning a 3rd embodiment.

  An image processing apparatus 100 according to the first embodiment of the present invention and an image processing system 190 including each device connected to the image processing apparatus 100 will be described in detail with reference to FIG. In the image processing system 190, a medical image is acquired, and a region of the target included in the medical image is extracted by image processing. The image processing system 190 displays an image capturing apparatus 110 that captures an image, a data server 120 that stores the captured image, an image processing apparatus 100 that performs image processing, and an acquired medical image and the result of the image processing. And an operation unit 170 for inputting operations. The medical image is an image obtained by performing image processing or the like for converting image data acquired by the image capturing apparatus 110 into an image suitable for diagnosis. Hereinafter, each unit will be described.

  The image processing apparatus 100 is, for example, a computer, and performs image processing according to the present embodiment. The image processing apparatus 100 includes a central processing unit (CPU) 11, a main memory 12, a magnetic disk 13, and a display memory 14. The CPU 11 integrally controls the operation of each component of the image processing apparatus 100. By the processing of the CPU 11, the image processing apparatus 100 may control the operation of the image photographing apparatus 110 together. The main memory 12 stores a control program executed by the CPU 11 and provides a work area when the CPU 11 executes the program. The magnetic disk 13 stores an operating system (OS: Operating System), device drivers for peripheral devices, and various application software including a program for performing image processing according to the embodiment described below. The display memory 14 temporarily stores data to be displayed on the display unit 160. The display unit 160 is, for example, a liquid crystal monitor, and displays an image based on an output from the display memory 14. The operation unit 170 is, for example, a mouse or a keyboard, and performs a pointing input, an input of characters, and the like by the operator. Display unit 160 may be a touch panel monitor that receives an operation input, and operation unit 170 may be a stylus pen. The image processing apparatus 100 may constitute a so-called computer-aided diagnosis (CAD) apparatus for utilizing the result of analyzing the medical image for a doctor's diagnosis. The above components are communicably connected to each other by a common bus 18.

  The image capturing apparatus 110 is, for example, a computed tomography apparatus (CT: Computed Tomography), a nuclear magnetic resonance imaging apparatus (MRI: Magnetic Resonance Imaging), or a radiation imaging apparatus (DR: Digital Radiography) that captures a two-dimensional radiation image. . The image photographing device 110 transmits the acquired image to the data server 120. An imaging control unit (not shown) that controls the image imaging device 110 may be included in the image processing device 100.

  The data server 120 holds an image captured by the image capturing device 110. The data server 120 is, for example, a PACS (Picture Archiving and Communication System) server. The image processing apparatus 100 reads necessary data from the data server 120 via a network such as Ethernet (registered trademark), and acquires an image stored in the data server 120.

  Next, each functional configuration of the image processing apparatus 100 will be described based on FIG. It is realized by the CPU 11 executing a program for realizing the function of each unit stored in the main memory 12. The image processing apparatus 100 includes an image acquisition unit 1000, a seed point acquisition unit 1010, a shape estimation unit 1020, a shape combination unit 1030, an extraction unit 1040, and a display control unit 1050.

  The image acquisition unit 1000 acquires a medical image to be subjected to image processing according to the present embodiment from the data server 120 as a target image. The image acquisition unit 1000 may acquire a medical image captured by the image capturing device 110 via a communication unit (not illustrated). The seed point acquisition unit 1010 stores information (hereinafter, referred to as seed point information) regarding one or a plurality of points (hereinafter, referred to as seed points) belonging to the region of the object included in the image acquired by the image acquisition unit 1000. get. The seed point information is, for example, coordinates indicating the position of each seed point and the number of acquired seed points. The shape estimating unit 1020 determines a region included in the target object (hereinafter, referred to as a partial region) based on the information on the seed points acquired by the seed point acquiring unit 1010 and the information on the density values of the peripheral pixels of each seed point. ) Is estimated. Then, a partial shape of each of the plurality of partial regions is estimated. The shape combining unit 1030 combines the plurality of partial shapes acquired by the shape estimating unit 1020 to form a general shape of the target region. Here, the approximate shape is a shape that approximates the shape of the object. That is, the shape estimating unit 1020 and the shape-coupled partial shape 1030 are configurations for obtaining a general shape. The extraction unit 1040 extracts the region of the target object based on the information regarding the density value of the target image acquired by the image acquisition unit 1000 and the schematic shape acquired by the shape combining unit 1030. The display control unit 1050 outputs the information on the region of the object acquired by the extraction unit 1040 to the display unit 160, and causes the display unit 160 to display the extraction result of the region of the object.

  Note that at least a part of each unit included in the image processing apparatus 100 may be realized as an independent device. The image processing device 100 may be a workstation. The software for realizing the functions of the respective units may be realized, and the software for realizing the functions may operate on a server via a network such as a cloud. In the present embodiment described below, it is assumed that each unit is realized by software in a local environment.

  Subsequently, image processing according to the first embodiment of the present invention will be described. FIG. 3 is a diagram illustrating a processing procedure executed by the image processing apparatus 100 according to the present embodiment. The present embodiment is realized by the CPU 11 executing a program for realizing the function of each unit stored in the main memory 12. In the present embodiment, a description will be given assuming that the target image is a CT image. The CT image is acquired as a three-dimensional gray image. In the present embodiment, the description will be given on the assumption that the target included in the target image is a lymph node.

  In step S1100, a medical image is obtained. That is, the image acquiring unit 1000 acquires a CT image as a target image from the data server 120, and develops and holds the CT image on the main memory 12 of the image processing apparatus 100. In another example, the image acquisition unit 1000 acquires image data photographed by the image photographing apparatus 110 via a communication unit (not shown), and performs image processing or the like to make the image suitable for diagnosis. A medical image to be subjected to image processing according to the embodiment is obtained. When the image capturing apparatus 110 is, for example, a CT apparatus, a one-dimensional distribution of relative X-ray absorption coefficient values called CT values is acquired from the image capturing apparatus 110 as image data. After that, the image acquisition unit 1000 performs a process called image reconstruction to obtain a medical image represented by a three-dimensional grayscale image. The image acquisition unit 1000 may acquire a three-dimensional grayscale image obtained by the image capturing apparatus 110 performing image reconstruction.

  Here, the target image in the present embodiment is composed of a plurality of pixels that can be identified by components of three orthogonal axes (x, y, z). The pixel size acquired as the supplementary information is defined for each of the three axes. In the present embodiment, a case where the pixel size in each of the three axis directions is r_size_x = 1.0 mm, r_size_y = 1.0 mm, and r_size_z = 1.0 mm will be described as an example. The density value of the target image can be regarded as a function derived with reference to a pixel position in a three-dimensional array of pixels. In the present embodiment, the target image is represented as a function I (x, y, z). The function I (x, y, z) is a function that takes the three-dimensional real space coordinates (x, y, z) of the imaging region of the target image as an argument and outputs a pixel value at the coordinates.

In step S1110, seed point information is obtained. The seed point acquisition unit 1010 _converts the coordinates p _{seed — i} (x _{seed —} i, y _{seed —} i, z _{seed —} i) (i = 1, 2,..., N) of the n points belonging to the region of the object, that is, the lymph node region, into a seed point group. To get as Here, the seed point is preferably located near the center line of the lymph node region. The center line is a line through the p _g in arbitrary two-dimensional plane including the center of gravity (p _g) of the lymph node area.

The acquisition of the seed point will be described. First, the operator operates the operation unit 170 such as a mouse while referring to a tomographic image of a target image displayed on the display unit 160, for example, a cross section (Axial), a sagittal plane (Sagittal), or a coronal plane (Coronal). To perform operation input. Obtained by the operation input, to acquire one of the coordinates included in the lymph node area as the seed point p _{Seed_1} first one. Then, in the region near the _{p Seed_1 acquired,} a pixel having a feature value similar to _{p Seed_1,} obtains as a new seed point. In another example, a lump enhancement filter based on a Hessian matrix is applied to the target image, and, for example, the center of gravity of each of the n regions is selected as a seed point from the n regions. The number of seed points may be set in advance by the operator. The operator sets the interval between the seed points in advance, and the operator specifies the range of the object on the target image using, for example, a rectangle using a mouse, and sets the seed point set at the center line of the rectangle. The seed point may be obtained based on the interval.

In step S1120, a plurality of partial shapes are estimated based on a plurality of points belonging to the region of the target object. The shape estimating unit 1020 uses the target image I (x, y, z) acquired in step S1100 and each of the seed points p _{seed_i} acquired in step S1110 to estimate a partial shape including the seed point. I do. In order to estimate partial shapes that are feature regions of various sizes included in an image, for example, a scale space expression that expresses an image as being parameterized by a scale is effective. The scale-space representation of an image is a stack of images obtained by progressively blurring a given image. Here, assuming that the partial shape is a sphere, estimation of the partial shape using the scale space of a Laplacian of Gaussian (LoG) kernel will be described. This makes use of the fact that, when LoG filters of various scales are applied, the scale appropriate for the size of the partial shape has an extreme value.

First, for each seed point p _{seed_i} , the LoG kernel represented by Expressions 1 and 2 is applied to a local image area I ^′ (x, y, z) around the seed point with the coordinates of p _{seed_i} as the origin. .
L (x, y, z, h) = I ^′ (x, y, z) * LoG _h (r) (Equation 1)

Here, h is a parameter representing the scale of the LoG kernel. * Is an operator representing convolution. The output of L (x, y, z, h) when the scale h is changed in a predetermined range {h _min ,..., H _max } is acquired. The operator can set h _min and h _max to arbitrary values. Next, a scale h corresponding to the maximum value of the output of L (x, y, z, h) is selected. Then, a sphere having the diameter of the scale is obtained as a partial shape around the seed point.

The estimation of the partial shape of the lymph node using the LoG kernel will be described with reference to FIG. For simplicity, all shapes are shown in two dimensions in the figures. FIG. 4A shows a CT image 510 that is a target image. The CT image 510 shows a lymph node 520 that is a target object and the seed points 530, 531, and 532 acquired in step S1110. When the above-described LoG kernel is applied to a local region centered on each seed point, radii 540, 541, and 542 of spheres approximating each local region are obtained as shown in FIG. . Then, as shown in FIG. 4C, the spheres 550, 551, and 552 corresponding to the obtained three radii are estimated as the partial shape Q _i (p _seed — i | i = 1,..., N). In each of the plurality of partial regions included in the target object, when the partial shape is estimated based on Expressions 1 and 2, the partial shape is estimated as a plurality of spheres. In the processing by the LoG filter, each diameter may be multiplied by a coefficient set in advance, and a sphere corresponding to the diameter multiplied by the coefficient may be estimated as a partial shape.

In another example of step S1120, estimation is performed assuming that the partial shape is either a sphere or an ellipsoid. For example, in three cross sections orthogonal to each other including the seed point, the diameter of a partial sphere or ellipsoid is estimated. Specifically, an arbitrary section is defined as a first section, and in the first section, for each seed point p _{seed_i} , a local image area I ^′ (x, y) around the seed point with the coordinates of p _{seed_i} as the origin. Is assumed to be a circle. Then, the LoG kernel represented by Expressions 3 and 4 is applied.
L (x, y, h) = I ^′ (x, y) * LoG _h (r _xy ) (Equation 3)

Then, a scale h corresponding to the maximum value of the output of L (x, y, h) is selected. The radius r _xy corresponding to the scale is set as the first diameter of the partial shape. The same processing is performed on the second section and the third section to obtain a second diameter _ryz and a third diameter _rzx . Thus, in each of the three axes, two diameters are obtained as candidates representing the diameter of the partial shape. For example, in the x-axis direction, r _xy and r _zx are acquired as candidates for the diameter of the partial shape in the x-axis direction. As a method of estimating the diameter in the x-axis direction from the two diameters, for example, an average value of the two diameters is employed. Alternatively, the larger one of the two diameters may be adopted. Similarly, the diameter of each of the three axes can be estimated. The shape estimating unit 1020 estimates an ellipsoid having the estimated three-axis direction diameter as the partial shape. When the first diameter, the second diameter, and the third diameter are respectively equal, the partial shape is estimated as a sphere. A rectangular parallelepiped having the first diameter, the second diameter, and the third diameter in three axial directions may be used as the partial shape. An ellipsoid inscribed in the rectangular parallelepiped may be the partial shape.

  In another example of step S1120, a partial shape is estimated by an ellipsoid model having degrees of freedom for three diameters. As shown in FIG. 5, the ellipsoid model is fitted to a region around each seed point of the lymph node, and the obtained ellipsoid is estimated as the partial shape.

  In still another example of step S1120, a plurality of hexahedrons are estimated as partial shapes by estimating the smallest bounding box surrounding the object from each seed point. The shape may be another polyhedron such as a rectangular parallelepiped. A plurality of methods may be applied simultaneously, and the user may manually select a set of partial shapes that are considered to be closest to the actual shape.

  In still another example of step S1120, a partial figure is set in advance as a template. The operator designates a template figure while referring to the target image displayed on the display unit 160, and designates a point included in a partial area whose shape is to be approximated using the designated figure as a seed point. For example, when a sphere is specified as a template figure and a point included in a certain partial area is specified, the diameter of the sphere centered on the specified point is estimated using Expressions 1 and 2, and the partial area is approximated. Partial shape. Examples of the template figure include a sphere, an ellipsoid, a rectangular solid, and a polyhedron.

  That is, in step S1120, in a partial region including one of a plurality of points belonging to the region of the target object, any of a sphere, an ellipsoid, or a polyhedron including one point is defined as a partial shape of the partial region. Estimate each of the partial shapes.

In step S1130, the plurality of estimated partial shapes are combined to obtain a rough shape. Shaped coupling unit 1030 combines the partial shape _{Q i} obtained in Step S1120, obtains the general shape of the lymph node. In the present embodiment, the partial shapes (Q ₁ , Q ₂ ,..., Q _n ) acquired in step 1120 are combined by ORing them. The shape Q indicated by the connected region is set as a schematic shape of the lymph node. FIG. 6A shows the spheres 550, 551, and 552 that are the partial shapes acquired in step S1120. FIG. 6B shows a schematic shape 600 obtained by combining these spheres 550, 551, and 552 by logical OR.

  In another example of step S1130, for example, a partial shape obtained by multiplying a partial shape estimated in step S1120 by a certain coefficient is combined to obtain a general shape. It is preferable that the approximate shape obtained by this is a shape larger than the lymph node that is the object. The coefficient is experimentally obtained so as to be larger than the shape to be extracted. The setting may be made by the operator. That is, in step S1130, the approximate shape may be obtained by multiplying the diameter of the partial region by a coefficient for making the region of the approximate shape larger than the target object.

In step S1140, a region of the target object is extracted based on the schematic shape. The extracting unit 1040 extracts a region of a lymph node as a target using the target image I (x, y, z) obtained in step S1100 and the schematic shape Q obtained in step S1130. The seed point information of the seed point group p _{seed_i} acquired in step S1110 may be used. In the present embodiment, a process using a variable shape model is performed as a process for extracting a specific area from a grayscale image. The variable shape model is a method of sequentially deforming a contour shape according to a predetermined evaluation function and extracting an optimized contour as a target region. Here, the level set method which is one of the processes using the variable shape model will be described as an example. The level set method is a method capable of coping with a change in the topology of a contour of an object and occurrence of a singular point. In the level set method, in an implicit function φ defined in a space one-dimensionally higher than the space of the object, a cross section where φ = 0 (hereinafter referred to as a zero isosurface) is defined as a contour of the object to be extracted. Catch. By changing the shape of φ, the shape of the zero isosurface is changed, and the contour of the object is changed. The position of the contour to be extracted is controlled by an evaluation function for designing the shape of φ. By designing the shape of φ using an evaluation function corresponding to the approximate shape Q, the region of the target object is controlled so as not to deviate from the approximate shape Q. That is, a tentative outline of the target object area is created based on the schematic shape, and the deformation of the tentative outline is controlled by the evaluation function to acquire the outline of the target object area.

  Hereinafter, extraction using the level set method will be described. A provisional outline is created for the lymph node region to be extracted. In order to deform the tentative contour, that is, the zero iso-surface, a velocity function for giving a velocity to a point belonging to the zero iso-surface is constructed. The speed function is an evaluation function for designing the shape of φ. The velocity function has a term that takes into account the density value, shape characteristic, etc. of the target object to be extracted.The velocity increases as the contour deviates from such characteristics, and the velocity increases as the contour has such characteristics. Constructed to be small.

The first characteristic is a difference in concentration value between the inside and outside of the lymph node region. Therefore, for example, a velocity function f _E (i, j, k) based on the edge strength as shown in Expression 5 is used.

Here, E (i, j, k) represents the edge strength at the pixel (i, j, k), and α is a weight coefficient. E (i, j, k) is represented by a concentration gradient. In another example, an edge detection filter such as a Sobel filter or a Laplacian filter is applied to a target image and their output values are used. Due to f _E (i, j, k), the deformation speed of the contour shape of the lymph node is large in the area where the density value change is small, and is small near the edge area where the density value change is large. Thus, the position of the contour is controlled so that the change in density value is large in the region of the contour of the extracted object.

The second characteristic is that the contour of the lymph node is smooth. Therefore, for example, a velocity function f _κ (i, j, k) based on the curvature as shown in Expression 6 is used.
f _κ (i, j, k) = 1−β · κ (i, j, k) (Equation 6)
Here, κ (i, j, k) represents the curvature at the pixel (i, j, k), and β is a weight coefficient. Due to f _κ (i, j, k), the deformation speed of the contour shape of the lymph node increases at a location where the curvature is small, and decreases at a location where the curvature is large. Thereby, the position of the contour is controlled such that the curvature of the contour of the extracted object becomes large.

Further, in the image processing according to the first embodiment of the present invention, a characteristic is added that the contour of the object to be extracted does not deviate from the general shape Q acquired in step S1130. Therefore, a velocity function f _d (i, j, k) based on the distance between a certain pixel and the outline of the general shape Q as shown in Expression 7 is used.
f _d (i, j, k) = 1−γ · d (i, j, k) (Equation 7)
Here, d (i, j, k) is the distance between the pixel (i, j, k) and the outline of the general shape Q. γ is a weight coefficient. The distance between the pixel (i, j, k) and the approximate shape Q can be calculated by, for example, signed distance conversion. Specifically, description will be given based on FIG. First, signed distance conversion is performed on the schematic shape 601 acquired in step S1130. As shown in FIG. 7, the distance value is set so that the distance value is 0 at the position on the outline of the general shape 601, the distance value is negative inside the general shape 601, and the distance value is positive outside the general shape 601. The absolute value of the distance value is represented by, for example, six neighboring distances between the pixel (i, j, k) and the contour pixel of Q. The distance may be represented by 18 neighborhood distances or 26 neighborhood distances. By using such a signed distance field, f _d (i, j, k), that is, the deformation speed of the contour shape of the lymph node is small near the contour of the general shape, and increases as the distance from the contour of the general shape increases. . Furthermore, the temporary contour can be set to be deformed in the expanding direction because the speed becomes positive inside the general shape, and the temporary outline is deformed in the direction to be reduced because the speed becomes negative outside the general shape. Thereby, the position of the extracted object is controlled so that the outline of the object does not deviate from the general shape.

The three types of speed functions described above are combined as shown in Expression 8 and used as a speed function of the level set method.
F _{i, j, k} = f _E (i, j, k) · f _κ (i, j, k) · f _d (i, j, k) (Equation 8)
Thereby, the contour of the region of the target object can be extracted as a curve that is smooth and does not deviate from the approximate shape at the edge portion. An outline in which the deformation controlled by the speed function shown in Expression 8 stops or the deformation amount becomes equal to or less than a preset threshold is defined as an object extraction result R.

When a general shape larger than the target object is acquired in step S1130, the contour of the target object falls within the range of the general shape obtained in step S1120 because the velocity function has a term of f _d (i, j, k). The position of the contour is controlled so that When the density difference between the target object and other structures is small, for example, when the lymph node is in a position close to other organs, a contour with stronger edge strength than the contour of the lymph node to be extracted is used. May be extracted. By having the term of f _d (i, j, k), the contour may be deformed within the range of the approximate shape estimated by the shape larger than the object, so that the extraction accuracy of the object is improved.

  As another example of step S1140, the rough shape Q is used as a restriction when extracting an object. In this example, the extraction of the lymph node is performed in a region satisfying Expression 9.

Here, d (P _i ) represents a value of the pixel P _{i in} the signed distance field. _dt is a threshold value defined in advance by the operator. According to the setting of _dt , the extraction processing of the lymph node can be performed in a desired area inside and outside the schematic shape. In this case, the term relating to the distance between the approximate shape Q and the pixel (i, j, k) may not be included in the velocity function.

As still another example of step S1140, the extraction unit 1040 may use a technique such as Graph cut, Water flow, or Dynamic Programming for the extraction processing. For example, in the method of extracting using the graph cut, a foreground region R _obj for extracting the outline shape Q is defined, and in the difference region between the target image and R _obj , the region obtained by the threshold processing is set to the background region R _obj. Defined as _bkg . Then, energy defined by a data term using a characteristic such as an expected density value of the target object and a smoothing term corresponding to a value of an adjacent pixel is obtained. The energy decreases when the target pixel is close to the set feature and the density value greatly differs at the boundary. A boundary that minimizes this energy is extracted as a contour of the region of the target object.

  In step S1140, in addition to the above-described extraction method, information on the density value and shape of the lymph node to be extracted or statistical information may be used together.

  FIG. 6C shows the lymph node region 710 extracted in step S1140.

  In step S1150, a process of displaying the result of the extraction is performed. The display control unit 1050 performs control to transmit information about the shape extracted in step S1140 to the display unit 160 connected to the image processing apparatus 100 and display the information. The display control unit 1050 causes the display unit 160 to display, for example, an image obtained by superimposing the extracted region and the original three-dimensional CT image and synthesizing the image (hereinafter, referred to as a synthesized image). In another example, a composite image with a two-dimensional cross-sectional image cut on a predetermined surface is displayed. In still another example, a three-dimensional composite image is volume-rendered and displayed. As shown in FIG. 6C, the seed points used in the processing up to the extraction may be displayed so as to be superimposed on the extracted area 710.

  The display control unit 1050 controls a display mode of the result of the extraction. For example, as shown in FIG. 15A, the outline of the region of the extracted target object is displayed as a curve. The color inside the outline may be changed and displayed. At this time, the seed points used for the extraction processing may be displayed together. In another example, a graphic such as an arrow is displayed so as to be superimposed on the target image, and the tip of the arrow is displayed so as to indicate the extracted region. The above-described display form may be changed according to the size of the extracted area. For example, when the size of the extracted region is larger than a predetermined threshold, the outline may be displayed as a curve, and when the size of the extracted region is smaller than the threshold, the position of the extracted region may be indicated by the tip of an arrow. These display modes facilitate observation of the target image.

  In the image processing apparatus according to the first embodiment of the present invention, the approximate shape approximate to the shape of the target object is obtained by combining a plurality of partial regions, thereby improving the accuracy of the approximate approximate shape. Therefore, the accuracy of target object extraction performed based on the schematic shape is improved.

  Subsequently, a second embodiment of the present invention will be described. In the present embodiment, the method further includes a step of correcting the shape of the acquired general shape. A process of adding a necessary area or deleting an unnecessary area is performed on the acquired schematic shape. Then, an area of the target object is extracted by an acquisition method according to the corrected schematic shape. The details will be described below.

  Based on FIG. 9, each functional configuration of the image processing apparatus 200 according to the second embodiment of the present invention will be described. The image processing device 200 executes the image processing according to the second embodiment of the present invention. The image processing device 200 includes an image acquisition unit 1000, a seed point acquisition unit 1010, a shape estimation unit 1020, a shape combination unit 1030, a shape modification unit 1035, an extraction unit 1040, and a display control unit 1050. The components having the same functions as those of the components constituting the image processing apparatus 100 according to the first embodiment are denoted by the same reference numerals as those in FIG. 2 and their description is omitted. The shape correcting unit 1035 adds a necessary area or deletes an unnecessary area to the schematic shape acquired by the shape combining unit 1030 by combining the partial shapes estimated by the shape estimating unit 1020, and thereby more appropriately. Perform processing to correct the shape. The extraction unit 1040 extracts a region of the target based on the schematic shape corrected by the shape correction unit 1035.

  Subsequently, image processing according to the second embodiment of the present invention will be described. FIG. 10 is a diagram illustrating a processing procedure executed by the image processing apparatus 200 according to the present embodiment. The present embodiment is realized by the CPU 11 executing a program for realizing the function of each unit stored in the main memory 12. The processing in steps S2000, S2010, S2020, and S2030 is the same as the processing in steps S1100, S1110, S1120, and S1130 shown in FIG.

  In step S2035, the acquired general shape is corrected. The shape correction unit 1035 corrects the schematic shape obtained in step S2030 using the density value information of the image obtained in step S2000 and the seed point information obtained in step S2010. In step S2030, for example, the schematic shape acquired based on the partial shape estimated based on Expression 1 is discontinuous as in the general shape 610 in FIG. 11 or inappropriate as in the general shape 611 in FIG. Or If the target object is extracted based on a discontinuous or inappropriate outline shape, the extraction accuracy may be reduced. In step S2035, a general shape correction process is performed to obtain a more appropriate general shape.

  As a correction method, for example, the shape is approximated by an implicit polynomial (IP: Implicit Polynomial) based on the approximate shape, and the shape is corrected to a continuous and smoother shape. Here, the implicit polynomial is an implicit function defined in the form of a polynomial. The approximation of the shape by IP is performed by expressing the implicit function in the form of a polynomial, and acquiring the coefficients of the polynomial. The approximation of the shape by IP is performed using a method disclosed in, for example, Japanese Patent No. 5335280.

The correction of the schematic shape using the IP will be described with reference to FIG. A function is obtained such that the value of IP becomes zero at the contour point 800 belonging to the surface of the general shape 610 acquired in step S2030, and the function is approximated by a curved surface represented by the function to model the surface shape of the target object. The approximate shape obtained by correcting the shape represented by the acquired model is defined as Q ^′ .

Furthermore, by taking into account the constraint conditions such as the position of the contour point and the order of the function, it is possible to acquire various schematic shapes Q ^′ as shapes having continuous and smooth contours. FIG. 12 shows examples of various modified schematic shapes. Here, the constraint conditions may be set in advance by the operator, or shapes under various conditions may be created and presented, and the operator may select the most appropriate one.

  The shape correction unit 1035 determines whether or not the schematic shape obtained in step S2030 is a continuous shape, and performs the correction as described above. Alternatively, the information may be output to the display control unit 1050 so as to display a screen for notifying the operator that the schematic shapes are not continuous.

  Also, the shape correcting unit 1035 deletes, from the schematic shape acquired in step S2030, an area that is not clearly an area belonging to the target object. The shape correction unit 1035 determines whether or not there is a large density value change in the peripheral region of the outline of the outline shape based on the density value information acquired in step S2000. If there is a large change in the density value at a position inside the schematic shape that is equal to or longer than the predetermined distance, it is determined that the general shape includes an area that does not clearly belong to the target object. Alternatively, the outline of the schematic shape may be displayed via the display control unit 1050, and the operator may appropriately input a correction instruction. The display control unit 1050 may indicate a location that can be corrected by the operator and accept an operation input to the location.

  In another example of step S2035, the approximate shape is corrected by correcting the estimated partial shape. For example, when the partial shape is estimated as a sphere or an ellipsoid in step S2020, the partial shape is corrected by changing the size of the estimated diameter. Based on the density value information acquired in step 2000, the shape correction unit 1035 determines whether or not there is a large density value change in the peripheral area of the outline of the outline shape. If there is a large change in the density value at a position outside the schematic shape that is longer than the predetermined distance, it is determined that the general shape does not clearly include a region to which the target object belongs. In that case, the diameter estimated in the corresponding step S2020 is corrected to be large, and the partial shape is corrected to be large. Based on the same determination, it is determined that an area that is not clearly the area to which the object belongs is included. In that case, the diameter estimated in the corresponding step S2020 is corrected to be small, and the partial shape is corrected to be small. The shape correction unit 1035 combines the corrected partial shapes and acquires the corrected partial shape as a corrected schematic shape. Alternatively, the partial shape may be changed by changing the position of the center point of a sphere or ellipsoid, which is a partial shape constituting the schematic shape, and the schematic shape may be corrected based on the changed partial shape.

  The shape correction unit 1035 may display the process of the correction process via the display control unit 1050. For example, a figure indicating a partial shape before correction is displayed so as to overlap the target image. If the partial shape is a sphere or an ellipsoid, a straight line indicating the diameter or the like is also displayed. The operator can input an operation for correcting the partial shape by changing the position and the size of the diameter of the graphic representing the partial shape.

In step S2040, the contour of the object is extracted. The extraction unit 1040 uses the image I (x, y, z) acquired in step S2000, the seed point group p _{seed_i} acquired in step S2010, and the corrected general shape Q ′ acquired in step S2035 to obtain the target object. Is extracted. The processing in step S2040 is the same as step S1140 in the first embodiment. However, when the level set method is used, for example, the extraction unit 1040 adds the distance d (i, j, k) in the velocity function f _d (i, j, k) to the pixel p (i , j, k). This speeds up the extraction process. Further, as in step S1140, a signed distance conversion may be performed on the corrected general shape Q ′, and the calculation may be performed based on the acquired distance map.

  The process in step S2050 is the same as the process in step S1150 of the first embodiment, and thus the description is omitted.

  According to the image processing device according to the second embodiment of the present invention, it is possible to correct the schematic shape of the target object area. By performing the extraction using the corrected schematic shape, the region of the target object can be accurately extracted.

  Subsequently, a third embodiment of the present invention will be described. This embodiment further includes a step of determining whether or not to repeat the process based on a comparison between the schematic shape and the extraction result. It is determined whether or not to repeat the processing, and if it is determined to be repeated, the processing from acquisition of the seed point to extraction of the region is performed again. The details will be described below.

  FIG. 13 illustrates each functional configuration of the image processing apparatus 300 according to the third embodiment of the present invention. The image processing device 300 executes the image processing according to the third embodiment of the present invention. The image processing device 300 includes an image acquisition unit 1000, a seed point acquisition unit 1010, a shape estimation unit 1020, a shape combination unit 1030, an extraction unit 1040, a continuation determination unit 1045, and a display control unit 1050. The components having the same functions as those of the components constituting the image processing apparatus 100 according to the first embodiment are denoted by the same reference numerals as those in FIG. 2 and their description is omitted. The continuation determining unit 1045 determines whether or not to perform the repetitive processing based on the schematic shape acquired by the shape combining unit 1030 and the extraction result acquired by the extracting unit 1040. When it is determined that it is necessary to perform the repetitive processing, the processing of each unit from the acquisition of the seed point to the extraction of the area is performed again. The shape coupling unit 1030 may include a shape correction unit 1035.

  Subsequently, image processing according to the third embodiment of the present invention will be described. FIG. 14 is a diagram illustrating a processing procedure executed by the image processing apparatus 300 according to the present embodiment. The present embodiment is realized by the CPU 11 executing a program for realizing the function of each unit stored in the main memory 12. The processing in steps S3000, S3010, S3020, S3030, and S3040 is the same as the processing in steps S1100, S1110, S1120, S1130, and S1140 shown in FIG.

In step S3045, the continuation determination unit 1045 compares the schematic shape Q acquired in step S3030 with the extraction result R acquired in step S3040, and determines whether to perform the repetitive processing. For example, a difference V between the extraction result R and the general shape Q is obtained. The difference V can be calculated by, for example, Expression 10.
V = Q∪R−Q∩R (Equation 10)
When the volume of the difference V is greater than the threshold value _{V T,} continuation determination unit 1045 determines that the seed point obtaining unit 1010 repeats the processing up to the extraction unit 1040. Here, the threshold value V _T may be calculated as Equation 11.
V _T = ω · V _Q (Equation 11)
Here, ω is a coefficient, and V _Q is the volume of the approximate shape Q. V _T is the operator may be set in advance. That is, it is determined whether or not to continue the process of acquiring and extracting the general shape based on the similarity between the general shape and the extracted region of the target object.

  When it is determined that the repetitive processing is to be performed, the seed point acquiring unit 1010, the shape estimating unit 1020, the shape combining unit 1030, and the extracting unit 1040 execute each processing again.

The processing in step S3010 is performed again, and the seed point acquiring unit 1010 acquires the target image I (x, y, z), the previously acquired seed point p _{seeded_i} (i = 1, 2,..., N), and the previously acquired A seed point is obtained based on the partial shape Q _i , the general shape Q, and the extraction result R. The acquisition of the seed point may be performed by the operator via the operation unit 170, or may be automatically performed by the image processing apparatus 300. When manually acquiring the seed points, the user may visually add the necessary seed points or delete the unnecessary seed points while visually comparing the previously acquired schematic shape Q and the extraction result R. .

When automatically acquiring seed points, for example, as shown in FIG. 15A, the seed point acquiring unit 1010 selects some points as new seed points from the center line 900 of the previously acquired extraction result 700. _And add it to p _{seed_i} . For the selection of points, for example, all points on the center line 900 are selected at regular intervals. Alternatively, a point whose characteristic such as a density value is similar to the seed point p _{seed_i} obtained up to the previous time is selected. In another example, as shown in FIG. 15B, a new seed point may be obtained from the difference area 800 between the extraction result 701 and the schematic shape 631. In this case, a point group is sampled from the area 800 by, for example, the Markov chain Monte Carlo (MCMC) method. From the sampled point group, points having characteristics such as density values similar to the previously obtained seed point 533 are obtained as new seed points 534 and 535. That is, the seed point acquisition unit 1010 further acquires a seed point based on the extraction result of the schematic shape based on the previously acquired seed point. The seed point obtaining unit 1010 may display the previously obtained seed points and the extraction result via the display control unit 1050, and may display the newly obtained seed points so as to be distinguishable from the previously obtained seed points. . As another example of automatically acquiring a seed point, a new seed point may be acquired based on a previous extraction result or an operation input for a schematic shape or a partial shape used for extraction. For example, the operator selects a point on the curve indicating the previous extraction result, and acquires a new seed point based on an operation input for moving in a certain direction. A new seed point is acquired in the area of the difference between the previous extraction result and the area indicated by the curve deformed by the operation input.

Further, unnecessary seed points are deleted. For example, as shown in FIG. 15C, the seed point 538 corresponding to the partial shape 553 that largely deviates from the extraction result 702 is deleted.
The seed point group after _{performing the} above-described seed point correction processing is referred to as p ^j _{seeded_i} (i = 1, 2,..., N), and j represents the number of repetitions. The seed point acquisition unit 1010 may display the seed point over the target image via the display control unit 1050, and the operator may delete the seed point while referring to the display unit.

  The third embodiment of the present invention includes a mode in which the number of seed points acquired in step S3010 is increased according to the number of repetitions. For example, it is assumed that one seed point is acquired in step S3010, the contour of the object is extracted by the processing in steps S3020, S3030, and S3040, and the continuation determination unit 1045 determines that the repetition processing is performed. Returning to step S3010, the seed point acquisition unit 1010 acquires at least two seed points.

In step S3020, the shape estimating unit 1020 estimates a plurality of partial shapes using the target image I (x, y, z) and the seed point group p ^j _{seed_i} , as in step S1120. At this time, the estimation process can be performed with reference to the previously obtained extraction result R. For example, when the shape of the extraction result R is an ellipsoid, the shape estimating unit 1020 estimates the partial shape of the target object as an ellipsoid. When the partial shape is estimated by the same method as the previous time, the previously estimated partial shape is used as the partial shape corresponding to the seed point where the previous seed point group and the corrected seed point group overlap. Thereby, the processing can be executed more efficiently. The newly acquired partial shape and Q ^j _i.

In step S3030, the shape coupling unit 1030 combines the partial shape ^Q _{j i} obtained in step S3020, acquires a general shape ^{Q j.} This processing is the same as the processing in step S1130 described in FIG. 3 of the first embodiment, and a description thereof will not be repeated.

In step S3040, the extraction unit 1040, using the target image I obtained in step S3020 (x, y, z) and the obtained seed point group _{^p j} ^seed_i in step S3010, and the general shape ^{Q j} obtained in step S1130 To extract the object. This process is the same as step S1140 described in FIG. 3 of the first embodiment, and a description thereof will be omitted.

  If it is determined that the repetition processing is not necessary, the display control unit in step S3050 displays the extraction result. The processing in step S3050 is the same as the processing in step S1150 described with reference to FIG. 3 of the first embodiment, and a description thereof will not be repeated.

  According to the image processing apparatus according to the third embodiment of the present invention, the difference between the extraction result of the target object region and the schematic shape is reduced, so that a more appropriate schematic shape and the extraction result can be obtained. .

  The present invention supplies a program for realizing one or more functions of the above-described embodiments to a system or an apparatus via a network or a storage medium, and one or more processors in a computer of the system or the apparatus read and execute the program. This process can be realized. Further, it can also be realized by a circuit (for example, an ASIC) that realizes one or more functions.

  The image processing apparatus and the image processing system in each of the above-described embodiments may be realized as a single apparatus, or may be a form in which apparatuses including a plurality of information processing apparatuses are communicably combined with each other to execute the above-described processing. Often, both are included in the embodiments of the present invention. The above processing may be executed by a common server device or server group. In this case, the common server device corresponds to the image processing device according to the embodiment, and the server group corresponds to the image processing system according to the embodiment. The image processing apparatus and the plurality of apparatuses constituting the image processing system need only be able to communicate at a predetermined communication rate, and do not need to be in the same facility or in the same country.

  According to an embodiment of the present invention, a software program for realizing the functions of the above-described embodiment is supplied to a system or an apparatus, and a computer of the system or apparatus reads and executes the code of the supplied program. Including form.

  Therefore, the program code itself installed in the computer in order to realize the processing according to the embodiment by the computer is also one of the embodiments of the present invention. Also, based on instructions included in the program read by the computer, the OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can also be realized by the processing. .

  An embodiment in which the above embodiments are appropriately combined is also included in the embodiments of the present invention.

REFERENCE SIGNS LIST 100 Image processing device 1000 Image acquisition unit 1010 Seed point acquisition unit 1020 Shape estimation unit 1030 Shape combination unit 1040 Extraction unit 1050 Display control unit 1035 Shape correction unit 1045 Continuation determination unit

Claims (23)

Image acquisition means for acquiring a medical image,
By performing image processing on a plurality of mutually different regions in the object included in the medical image, a plurality of partial shapes approximating the shape of the region included in the object are estimated, and a plurality of partial shapes obtained by the estimation are obtained. Schematic shape obtaining means for obtaining a shape obtained by combining the partial shapes as a schematic shape approximating the shape of the object,
An extraction unit configured to extract the region of the object by using the schematic shape as a condition for a process of extracting the region of the object.   The image processing apparatus according to claim 1, wherein the rough shape obtaining unit is configured to estimate the plurality of partial shapes based on a plurality of points belonging to a region of the object.   The general shape obtaining means includes, in a region including one of the plurality of points, a sphere or an ellipsoid including the one point as a partial shape in the region, thereby obtaining the plurality of partial shapes. The image processing apparatus according to claim 2, wherein the image processing apparatus is configured to estimate each of the following.   The general shape obtaining means includes, in a region including one of the plurality of points, a sphere or an ellipsoid centered on the one point as a partial shape in the region, thereby obtaining the plurality of points. The image processing apparatus according to claim 2, wherein each of the partial shapes is configured to be estimated.   The image processing apparatus according to any one of claims 1 to 4, wherein the rough shape obtaining unit is configured to estimate the plurality of partial shapes as a plurality of spheres. .   The said general | schematic shape acquisition means is comprised so that the said several partial shape may be combined by a logical OR and may acquire the said general | schematic shape, The Claim 1 characterized by the above-mentioned. Image processing device.   The said rough shape acquisition means is comprised so that the curved surface represented by the polynomial approximated based on the shape which combined the said several partial shape may be the said rough shape. Item 7. The image processing device according to any one of Items 6.   The general shape obtaining means is configured to obtain the general shape by multiplying a diameter of the partial shape by a coefficient for making the general shape region larger than the object. The image processing device according to claim 7.   The image processing apparatus according to claim 8, wherein the extraction unit is configured to extract a region of the target object within a range of a rough shape estimated larger than the target object.   The extraction means creates a temporary outline of the region of the object based on the general shape, and deforms the temporary outline at a deformation speed according to a positional relationship with the general shape to obtain an area of the object. The image processing apparatus according to any one of claims 1 to 9, wherein the image processing apparatus is configured to extract an outline of the image. The extraction means is a means for creating a temporary outline of the region of the object based on the schematic shape, and extracting the outline of the region of the object by deforming the temporary outline,
The extraction means creates a temporary outline of the region of the object based on the general shape, and uses the evaluation function having a term based on a distance between the outline of the general shape and the temporary outline, The apparatus according to claim 1, wherein the object is extracted by controlling the deformation so as to reduce a distance from a contour of the shape. Image processing device.   The evaluation function is a term for controlling the position of the contour so that a change in density value is large in the contour of the extracted area of the object, and the curvature of the contour of the area of the extracted object is large. The image processing apparatus according to claim 11, further comprising: a section for controlling a position of the contour so that the contour is formed. Further comprising shape correction means for correcting the shape of the general shape,
The image processing apparatus according to claim 1, wherein the extraction unit is configured to extract a region of the object based on the corrected general shape. apparatus.   The shape correcting means corrects the general shape to a continuous shape based on the area represented by the logical sum when the general shape obtained by combining the partial shapes by logical sum is discontinuous. The image processing apparatus according to claim 13, wherein the image processing apparatus is configured as follows.   14. The apparatus according to claim 13, wherein the shape correcting unit is configured to change a partial shape constituting the general shape and to correct the general shape based on the changed partial shape. Image processing device.   The shape correcting means changes the partial shape by changing a position of a center point of a sphere or an ellipsoid which is a partial shape constituting the general shape, and changes the general shape based on the changed partial shape. The image processing apparatus according to claim 15, wherein the image processing apparatus is configured to perform correction.   The shape correcting means changes the partial shape by changing the diameter of a sphere or an ellipsoid which is a partial shape constituting the general shape, and corrects the general shape based on the changed partial shape. The image processing apparatus according to claim 15, wherein the image processing apparatus is configured to:   The apparatus further comprising: a determination unit configured to determine whether or not to continue processing by the general shape acquisition unit and the extraction unit based on the similarity between the general shape and the extracted region of the target object. The image processing apparatus according to claim 13. Image acquisition means for acquiring a medical image,
By performing image processing on a plurality of mutually different regions in the object included in the medical image, a plurality of partial shapes approximating the shape of the region included in the object are estimated, and a plurality of partial shapes obtained by the estimation are obtained. Schematic shape obtaining means for obtaining a shape obtained by combining the partial shapes as a schematic shape approximating the shape of the object ,
An image processing system comprising: an extracting unit configured to extract the region of the object by using the schematic shape as a condition for a process of extracting the region of the object.   20. The image processing system according to claim 19, wherein said rough shape obtaining means is configured to estimate each of said plurality of partial shapes as a sphere or an ellipsoid. An image acquisition step of acquiring a medical image;
By performing image processing on a plurality of mutually different regions in the object included in the medical image, a plurality of partial shapes approximating the shape of the region included in the object are estimated, and a plurality of partial shapes obtained by the estimation are obtained. A schematic shape obtaining step of obtaining a shape obtained by combining the partial shapes as a schematic shape approximating the shape of the object ,
An extraction step of extracting the region of the object by using the schematic shape as a condition for the process of extracting the region of the object.   22. The image processing method according to claim 21, wherein the plurality of partial shapes are each estimated as a sphere or an ellipsoid in the rough shape obtaining step.   A program for causing a computer to execute the image processing method according to claim 21.