WO2016027840A1 - Image processing device, method, and program - Google Patents

Abstract

A reception unit 102 receives an input image. In an eigenspace such that an eigenvector calculated beforehand serves as a basis and such that points in the eigenspace indicate the shape parameters of a statistical shape model which indicates the statistical fluctuations of the shape of a specific object, in order to optimize a predetermined objective function which indicates a value corresponding to the likelihood of the shape of the specific object indicated by the shape parameters indicated by the points in the eigenspace and corresponding to the difference in pixel values between adjacent pixels in the input image, a segmentation unit 114 estimates the shape parameters which indicate the shape of a subject indicated by the input image, and extracts from the input image a region in which the subject is present, with the shape of the specific object indicated by the estimated shape parameters serving as prior knowledge. Due to this configuration, an increase in the amount of computation can be suppressed and a region in which the subject is present can be extracted with good accuracy.

Description

Image processing apparatus, method, and program

The present invention relates to an image processing apparatus, method, and program, and more particularly, to an image processing apparatus, method, and program for extracting a region of a subject.

The process of recognizing a figure from a digital image using a computer is called segmentation. However, if the SN of the target figure is low and the shape fluctuates statistically, accurate segmentation becomes very difficult.

Graph cuts (eg Boykov, Y., Veksler, O., Zabih, R., “Fast approximate energy minimization via graph cuts.”, Pattern Analysis and Machine Intelligence, IEEE Transactions on 23 (11), 2001, p.122 Segmentation algorithms based on optimization theories such as -1239.) Are superior to conventional algorithms in that they can truly optimize the objective function, and often exhibit very high performance even when the SN is low. It was. However, even when the shape fluctuated, it was often not recognized correctly.

Therefore, recently, methods that use shape information of graphics in algorithms based on optimization theory have become mainstream, but they can be roughly divided into the following three types.

The first method uses a single shape template. For example, as a general shape template, an elliptical template (eg, Slabaugh, G., Unal, G., Sept, “Graph cuts segmentation using an elliptical shape prior.”, In: IEEE International Conference on Image Processing, 2005, Vol. 2. See p.II-1222-5.) And templates of block shapes (eg Funka-Lea, G., Boykov, Y., Florin, C., Jolly, M.- P., Moreau-Gobard, R., Ramaraj, R., Rinck, D., 2006. “Automatic heart isolation for CT coronary visualization using graph-cuts.”, In: Biomedical Imaging: Nano to Macro, 3rd IEEE International Symposium on. IEEE, 2006, p.614-617.) is known.
In addition, as a specific shape template, user-defined arbitrary shapes (for example, Freedman, D., Zhang, T., “Interactive graph cut based segmentation with shape priors.”, In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Vol. 1. See IEEE, 2005, p.755-762.) And shapes selected from statistical models (eg Grosgeorge, D., Petitjean, C., Dacher, J.-N. , Ruan, S., “Graph cut segmentation with a statistical shape model in cardiac mri.”, Computer Vision and Image Understanding 117 (9), 2013, p.1027-1035., Akinobu Shimizu, Keita Nakagomi, Takuya Narihira, Hidefumi Kobatake, Shigeru Nawano, Kenji Shinozaki, Koich Ishizu, and Kaori Togashi, “Automated Segmentation of 3D CT Images based on Statistical Atlas and Graph Cuts”, Proc. Of MICCAI workshop MCV, 2010, p.129-138., Malcolm, J ., Rathi, Y., Tannenbaum, A., “Graph cut segmentation with nonlinear shape priors.”, In: Image Processing, 2007. ICIP 2007. IEEE International Conference on. Vol. 4. See IEEE, 2007, p.IV-365.).

The second method uses a plurality of (several) shape templates or probabilistic representations of shapes. For example, multiple shapes selected from statistical models (eg, Nakagomi, K., Shimizu, A., Kobatake, H., Yakami, M., Fujimoto, K., Togashi, K., “Multi-shape graph cuts with neighbor Prior constraints and its application to lung segmentation from a chest CT volume. ”, Medical image analysis 17 (1), 2013, p.62-77.) and probabilistic representation of shapes (eg Linguraru) , M. G., Pura, J. A., Pamulapati, V., Summers, R. M., “Statistical 4d graphs for multi-organ abdominal segmentation from multiphase CT.”, Medical image analysis 16 (4), 2012 , Pp.904-914.) Is known.

As the third method, a large amount (several tens or more) of shape templates is used. For example, for specific problems (eg, Kohli, P., Rihan, J., Bray, M., Torr, PH, “Simultaneous segmentation and pose estimation of humans using dynamic graph cuts.”, International Journal of Computer Vision 79 ( 3), 2008, p.285-298.) And a method using an arbitrary large number of graphic shapes (for example, see US Pat. No. 8,249,349) are known. .

The conventional technique described above can handle figures that fluctuate statistically in order from the first method to the third method. The most advanced method is an arbitrarily large number of figure shapes (the above-mentioned US Pat. No. 8,249,349) of the third method. However, all methods including this method need to prepare a shape set in advance by selecting a shape in advance. Therefore, unless the optimal shape is included in the shape set selected in advance, the optimality of the segmentation is not guaranteed and the performance deteriorates.

On the other hand, since a digital image is a set of a finite number of pixels, a method of preparing all possible shape patterns can be considered in principle, and in that case, optimality is guaranteed. However, the number of shapes that must be prepared for this purpose is enormous, and it is not realistic from the viewpoint of the memory required for the entire process including preprocessing and the calculation time. For example, even the algorithm of the above-mentioned U.S. Pat. No. 8,249,349, which has been most excellent in the past, is limited in handling about 10 ⁷ shape templates in a two-dimensional image. However, the number of possible shapes pattern statistical model may be generated in many cases there 10 ⁹ or more, it is very difficult in terms of computational cost in the conventional algorithm. Furthermore, 3D images are the mainstream of medical images, but in that case, the required memory increases by more than two orders of magnitude, and it is impossible to execute processing in a realistic time and memory size using a conventional algorithm. It was.

One embodiment of the present invention has been made in view of the above circumstances.

To achieve the above object, an image processing apparatus according to a first aspect is an image processing apparatus that extracts a region of a subject from an input image representing a subject that is a specific object, and accepts the input image. And (A) a plurality of learning images representing the subject, wherein the subject area is an eigenspace based on eigenvectors calculated in advance based on the plurality of images obtained in advance. And (B) in an eigenspace where a point on the eigenspace indicates a shape parameter of a statistical shape model representing a statistical variation in the shape of the specific object, on the eigenspace based on the input image A predetermined objective function representing a value corresponding to the likelihood of the shape of the specific object represented by the shape parameter indicated by the point and the difference in pixel value between adjacent pixels in the input image is optimized. In addition, the shape parameter representing the shape of the subject represented by the input image is estimated, the shape of the specific object represented by the estimated shape parameter is used as prior knowledge, and the region of the subject is determined from the input image. And a segmentation means for extracting.

The image processing method according to the second aspect includes an accepting unit and a segmentation unit, and is an image processing method in an image processing apparatus that extracts a region of the subject from an input image representing the subject that is a specific object. The step of receiving the input image; and the segmentation unit: (A) a plurality of learning images representing the subject, wherein the regions of the subject are obtained in advance. And (B) a point on the eigenspace indicates a shape parameter of a statistical shape model representing a statistical variation of the shape of the specific object. In the eigenspace, based on the input image, the likelihood of the shape of the specific object represented by the shape parameter indicated by the point on the eigenspace and the previous Estimating the shape parameter representing the shape of the subject represented by the input image so as to optimize a predetermined objective function representing a value corresponding to a pixel value difference between adjacent pixels in the input image; Extracting the region of the subject from the input image using the shape of the specific object represented by the estimated shape parameter as prior knowledge.

Further, the segmentation means of the third aspect estimates the shape parameter representing the shape of the subject represented by the input image so as to optimize the objective function based on the input image in the eigenspace. At the same time, the region of the subject can be extracted from the input image using the shape of the specific object represented by the estimated shape parameter as prior knowledge.

Further, the segmentation means according to the fourth aspect is configured to repeatedly divide the convex multivesicle on the eigenspace representing the shape set including the shape of the specific object represented by the point indicating the optimum shape parameter to obtain the optimum shape parameter. The shape parameter representing the shape of the subject represented by the input image is estimated so as to optimize the objective function according to a search algorithm for searching for convex multivesicles on the eigenspace including a point indicating The region of the subject can be extracted from the input image using the shape of the specific object represented by the shape parameter as a prior knowledge.

Further, the segmentation means of the fifth aspect calculates the lower bound of the objective function for the shape set included in the convex multivesicle by examining each vertex of the convex multivesicle in the search algorithm, The convex multivesicle on the eigenspace including the point indicating the optimal shape parameter is searched to estimate the shape parameter representing the shape of the subject represented by the input image, and the specific shape represented by the estimated shape parameter The region of the subject can be extracted from the input image using the shape of the object as prior knowledge.

Further, the segmentation means of the sixth aspect divides the convex multivesicle on the eigenspace so that the volumes of the two convex multivesicles after the division correspond to each other, and indicates a point indicating an optimum shape parameter. Repeatedly including a search for a convex multivesicle in the eigenspace to estimate the shape parameter representing the shape of the subject represented by the input image, and the shape of the specific object represented by the estimated shape parameter Can be extracted from the input image.

Further, the segmentation means of the seventh aspect sets sampling points on the eigenspace, and divides the convex multi-vesicle using a hyperplane determined from the sampling points set on the eigenspace. By repeatedly searching for convex multivesicles on the eigenspace including points indicating optimal shape parameters, the shape parameters representing the shape of the subject represented by the input image are estimated, and the estimated shape parameters The area of the subject can be extracted from the input image using the shape of the specific object represented by

Also, the segmentation means of the eighth aspect can arbitrarily set sampling points on the eigenspace.

Further, the objective function of the ninth aspect includes a monotone function that monotonously changes with respect to the value of the shape label in the pixel for the shape of the specific object represented by the shape parameter as the likelihood of the shape of the specific object. Can be.

Also, the search algorithm of the tenth aspect can use the Branch-and-bound method and the graph cut method.

The program according to the eleventh aspect is a program for causing a computer to function as each means of the image processing apparatus.

According to the image processing apparatus, the method, and the program according to the embodiment, (A) the subject area is an eigenspace based on eigenvectors calculated in advance based on a plurality of images obtained in advance. B) In the eigenspace where the point on the eigenspace indicates the shape parameter of the statistical shape model representing the shape of the specific object, the likelihood and input of the shape of the specific object represented by the shape parameter indicated by the point on the eigenspace A shape parameter representing the shape of the subject represented by the input image is estimated so as to optimize a predetermined objective function that represents a value corresponding to a pixel value difference between adjacent pixels in the image. The area of the subject is extracted from the input image using the shape of the specific object represented by the estimated shape parameter as prior knowledge. As a result, an effect of suppressing an increase in the amount of calculation and extracting a subject area with high accuracy is obtained.

It is a block diagram which shows the structure of the statistical shape model production | generation apparatus which concerns on embodiment. It is a block diagram which shows the computer structural example of the statistical shape model production | generation apparatus and image processing apparatus which concern on embodiment. It is explanatory drawing which shows the example of a three-dimensional image of a liver. It is a figure which shows the statistical dispersion | variation in the shape of the liver along a 1st main component at the time of performing a main component analysis on the image of the shape of a liver. It is explanatory drawing which shows the example of a statistical shape model. It is explanatory drawing which shows the relationship between eigenspace and shape space. It is a block diagram which shows the structure of the image processing apparatus which concerns on embodiment. It is explanatory drawing for demonstrating the division | segmentation method of an eigenspace. It is a flowchart which shows the learning process routine in embodiment. 3 is a flowchart illustrating an image processing routine in the embodiment. It is a flowchart which shows the segmentation process routine in embodiment. It is a figure for demonstrating the approximate solution in the case of setting a sampling point in a grid | lattice form. It is a figure which shows the recognition result of 140 examples of pancreas in 2nd Embodiment. It is a figure which shows the comparison result of the calculation time in 2nd Embodiment. It is a figure for demonstrating the approximate solution method in the case of setting a sampling point at random. It is a figure which shows the relationship where the lower bound of an objective function satisfy | fills monotonicity. It is a diagram illustrating an example of a lower bound of the objective function satisfies the monotonicity ^h _F p and (y) and ^h _B p (y). It is a figure which shows the example in the case of using LogOdds as a statistical shape model. It is explanatory drawing which shows a prior art.

Hereinafter, embodiments will be described in detail with reference to the drawings. In the embodiment, the statistical shape model generation device 10 that generates a statistical model of the shape of a graphic representing the statistical variation of a specific object (hereinafter referred to as a statistical shape model), and the specific object. The image processing apparatus 100 that extracts a subject area from an input image representing the subject will be described. In the embodiment, a case where a three-dimensional abdominal CT image is used as an input image and a pancreas region is extracted from the input image will be described as an example.

<Overview>
In the present embodiment, a segmentation algorithm using a statistical shape model is proposed. This algorithm has the following five features.

1. In the segmentation process, all three-dimensional shapes (10 ⁹ or more) that can be generated using the statistical shape model can be considered, and an optimum shape can be selected from the viewpoint of the segmentation objective function.

2. It is indispensable in the conventional method using a large amount of shape templates, and does not require preprocessing (shape generation / selection and clustering) that is particularly expensive.

3. The algorithm is not limited to the combination of the Branch and bound method and the graph cut method, and any algorithm based on optimization theory can be applied.

4). It can also be applied to different statistical shape models and eigenspace partitioning methods.

5. The world's highest accuracy for pancreas segmentation in 3D abdominal CT images is obtained.

FIG. 19 shows an outline of processing in the prior art. FIG. 19 shows processing when the method of the above-mentioned US Pat. No. 8,249,349 is applied to pancreas segmentation using a statistical shape model. As shown in FIG. 19, the conventional method requires generation processing of a shape template set T (⊂S) and clustering processing for the set T. In this embodiment, as described above, a large number of shape templates are used. This eliminates the need for pre-processing required in the conventional method using the. Incidentally, the conventional method image which has been treated with (aforementioned U.S. Pat. No. 8,249,349) a two-dimensional, while the maximum number of shapes is ^107, the algorithm used in this embodiment is a three-dimensional shape (10 ⁹ or more) can be considered.

In the embodiment, a level set distribution model (Level set distribution model; LSDM, sometimes called a signed distance model) is used as the statistical shape model. In the following, first, after the eigenvector and eigenvalue of the statistical shape model (LSDM) are calculated by the statistical shape model generation device 10, the shape parameters of the statistical shape model are estimated and estimated by the image processing device 100. An optimal segmentation algorithm based on a statistical shape model with shape parameters is presented.

[First Embodiment]
<Configuration of Statistical Shape Model Generation Device 10 according to First Embodiment>
The statistical shape model generation apparatus 10 according to the first embodiment can be represented by the functional blocks shown in FIG. Further, these functional blocks can be realized by the hardware configuration of the computer shown in FIG. The configuration of the computer will be described with reference to FIG.

The statistical shape model generation apparatus 10 according to the first embodiment illustrated in FIG. 2 is a CPU (Central Processing Unit) that performs processing according to the present embodiment of the statistical shape model generation apparatus 10 based on a program. ) 21, a RAM (Random Access Memory) 22 used as a work area when the CPU 21 executes various programs, and a ROM (Read Only Memory) 23, which is a recording medium in which various control programs, various parameters, and the like are stored in advance. A hard disk 24 (described as “HDD” in the figure) used for storing various types of information, an input device 25 such as a keyboard and a mouse, a display device 26 composed of a display, etc., and a LAN (Local Area Network) Various kinds of information are exchanged between the communication device 27 that performs communication using the image information providing device 30 connected to the outside. And an input / output interface unit (described as “external IF” in the figure) 28, which are connected to each other via a system bus BUS 29.

The CPU 21 accesses the RAM 22, ROM 23, and hard disk 24, acquires various information via the input device 25, displays various information on the display device 26, communication processing of various information using the communication device 27, and input / output interface unit Input of information from an external device including the image information providing device 30 connected to 28 can be performed.

In the statistical shape model generation apparatus 10 according to the present embodiment shown in FIG. 1, the CPU 21 reads the program for controlling the processing according to the present embodiment stored in the hard disk 24 into the RAM 22 and executes it. The functions of the processing units shown in FIG.

The statistical shape model generation apparatus 10 according to the present embodiment shown in FIG. 1 is configured by such a computer configuration. Note that FIG. 1 shows a configuration as a functional block, while FIG. 2 shows a connection state of devices and the like. As described above, since the functional block and the device constitute the statistical shape model generation apparatus 10 in an organic and interrelated manner, FIG. 1 will be described in detail with FIG.

The statistical shape model generation apparatus 10 has a learning accepting unit 12, a learning unit 14, and a statistical shape model as functions configured by using software including computer hardware and a control program shown in FIG. And a database 16.

The learning accepting unit 12 accepts, as learning data, a plurality of images representing a subject that is a specific object and in which a subject area is obtained in advance. Since the specific object that is the subject in the present embodiment is the pancreas, the learning accepting unit 12 accepts a plurality of images in which the pancreas region is obtained in advance.

The learning unit 14 calculates eigenvectors and eigenvalues based on a plurality of images in which the pancreas region received by the learning receiving unit 12 is obtained in advance. For example, the learning unit 14 calculates eigenvectors and eigenvalues by principal component analysis based on a plurality of images in which pancreatic regions are obtained in advance. By calculating the eigenvector, an eigenspace based on the eigenvector is generated. The learning unit 14 stores the calculated eigenvector and eigenvalue in the statistical shape model database 16.

Here, FIG. 3 shows an example of the shape of the liver (note that the example shown in FIG. 3 is the liver, but the essence of the problem is the same as in the case of the pancreas). As shown in FIG. 3, the shape of the liver is various. Therefore, in the embodiment, the variation in shape is expressed by a small number of shape parameters using a statistical shape model representing the shape of a specific object. FIG. 4 shows the eigenvector of the shape of the liver corresponding to the first principal component obtained by principal component analysis. As shown in FIG. 4, it can be seen that the shape of the expressed liver changes according to the value of the shape parameter α corresponding to the eigenvector. Note that σ is a parameter representing statistical variation.

Fig. 5 shows an example of a statistical shape model. As shown in FIG. 5, there are various statistical shape models. In the present embodiment, LSDM (for example, references (Cremers, D., Rousson, M., Deriche, R., “A review of statistical approaches to level set segmentation: integrating color, texture, motion and shape.”, International journal of computer vision 72 (2), 2007, p.195-215.)) Is used.

LSDM is a representative statistical shape model, which creates a signed distance image for a graphic label of learning data and performs linear statistical analysis (for example, PCA or ICA) on the signed distance image. This is a method of expressing an arbitrary shape by the following formula (1).

Here, the above equation (1) is an equation relating to the pixel p (εP). P represents a pixel set, φ ^p (α) represents a level set function corresponding to an arbitrary shape related to the pixel p, μ ^p represents an average level set function related to the pixel p, and λ _i represents the i th {U ₁ ^p ,..., U _d ^p } represents the component for the pixel p in d eigenvectors, and α = [α ₁ ,..., Α _d ] ^T is on the eigenspace R _α = {rεR ^d ||| r || _∞ ≦ w}, which is a shape parameter defined in a region (usually in a range of ± 3σ). W represents a positive constant. The shape in the digital image is obtained by mapping the parameter α with a function g shown in the following equation (2). In the embodiment, a case where the dimension of the eigenspace is d = 2 will be described as an example.

Here, L = {0, 1} is a label set, 0 indicates a background, and 1 indicates a figure. y represents one shape in the image. In the case of LSDM, as shown in the following formula (3), the function g for mapping the parameter α to the shape is Heavisideｖfunction H (•).

FIG. 6 shows the relationship between the eigenspace of LSDM (a space spanned by eigenvectors) and the shape set S in the digital image when d = 2. The function g is a mapping from the eigenspace onto the shape set S of the digital image. Further, the shape set S⊂L ^{| P |} is an image of g. As shown in FIG. 6, the set of shape labels forms a polygon in the eigenspace. Each shape yεS has an original image in the eigenspace.

The statistical shape model database 16 stores eigenvectors and eigenvalues calculated by the learning unit 14.

<Configuration of Image Processing Device 100>
The image processing apparatus 100 according to the present embodiment can be represented by functional blocks shown in FIG. These functional blocks can be realized by the hardware configuration of the computer shown in FIG.

As shown in FIG. 7, the image processing apparatus 100 includes a reception unit 102, a calculation unit 104, and an output as functions configured using software including the computer hardware and the control program illustrated in FIG. 2. Part 120.

The accepting unit 102 accepts a three-dimensional three-dimensional abdominal CT image as an input image. The receiving unit 102 receives the early phase image, the portal phase image, and the late phase image as a three-dimensional three-dimensional abdominal CT image.

The calculation unit 104 extracts a pancreas region from the three-dimensional abdominal CT image received by the receiving unit 102. The calculation unit 104 includes a statistical shape model database 106 and an image processing unit 108.

The statistical shape model database 106 stores the same eigenvectors and eigenvalues as the statistical shape model database 16.

The image processing unit 108 includes an inter-image registration unit 110, a spatial standardization unit 112, and a segmentation unit 114.

The inter-image alignment unit 110 performs alignment between the early phase image, the portal phase image, and the late phase image received by the receiving unit 102 (for example, Shimizu, A., Kimoto, T., Kobatake, H., Nawano, S., Shinozaki, K., “Automated pancreas segmentation from three-dimensional contrast-enhanced computed tomography.”, International journal of computer assisted radiology and surgery 5 (1), 2010, p.85 98.), generate a registration image.

The spatial standardization unit 112 is based on the registration image generated by the inter-image registration unit 110, for example, using a predetermined nonlinear function, so as to match the registration image with the standard image. (E.g. Shimizu, A., Kimoto, T., Kobatake, H., Nawano, S., Shinozaki, K., “Automated pancreas segmentation from three-dimensional contrast-enhanced computed tomography.”, International journal of computer assisted radiology and surgery 5 (1), 2010, p.85-98.), generating spatially standardized images.

The segmentation unit 114 optimizes the objective function based on the input image received by the receiving unit 102 in the eigenspace based on the pre-calculated eigenvector stored in the statistical shape model database 106. The shape parameter representing the shape of the subject represented by the spatial standardized image generated by the spatial standardization unit 112 is estimated, and the region of the subject represented by the spatial standardized image is obtained. Specifically, the segmentation unit 114 estimates a shape parameter and simultaneously extracts a subject area.

As shown in FIG. 6, the points on the eigenspace based on the eigenvectors indicate the shape parameters of the statistical shape model. The objective function is determined in advance so as to represent a value corresponding to the likelihood of the shape of the specific object represented by the shape parameter indicated by the point on the eigenspace and the difference in pixel value between adjacent pixels in the input image. It has been.

In general, the size of the shape set L ^{| P |} described above (= the number of shapes on a digital image) is enormous, and cannot be handled by a conventional algorithm. The algorithm according to the present embodiment is different from the conventional method in that the optimization of the objective function is executed in the eigenspace, and this makes it possible to efficiently handle a huge number of shapes.

The segmentation unit 114 repeatedly optimizes the objective function of the following equation (4) according to a search algorithm for searching for a convex polygon including a point indicating an optimal shape parameter by repeatedly dividing the convex polygon on the eigenspace. In addition, a shape parameter representing the shape of the subject represented by the input image is estimated. Note that the convex polygon on the eigenspace to be searched represents the shape of a specific object represented by the point indicating the optimum shape parameter.
Further, the segmentation unit 114 examines each vertex of the convex polygon in the search algorithm, and calculates the lower bound of the objective function for the shape set included in the convex polygon, thereby obtaining a point indicating the optimum shape parameter. Search for a convex polygon in the eigenspace that contains it.
In this embodiment, the case where the eigenspace is two-dimensional will be described as an example. Therefore, the segmentation unit 114 searches for a convex polygon on the eigenspace, but when the eigenspace is three-dimensional or more, The segmentation unit 114 searches for convex multivesicles on the eigenspace. In addition, the straight line that divides the eigenspace during the search becomes a hyperplane when the eigenspace is three-dimensional or more.

An important idea in the present embodiment is to use the fact that the shape set on the digital image corresponds to the convex polygon on the eigenspace, as shown in FIG. In the embodiment of the present invention, it is possible to efficiently handle a large number of shapes at a time by utilizing the fact.

Based on the fact that a plurality of shape sets on a digital image correspond to convex polygons on the eigenspace, the segmentation unit 114 corresponds to a convex multiplicity on the eigenspace corresponding to a shape that minimizes a preset objective function. Search for a square. Then, the segmentation unit 114 finds a shape that minimizes the objective function from the shape set S based on the searched convex polygon in the eigenspace, and uses the found shape as prior knowledge to determine the shape as a subject area. Extract as In this embodiment, the graph cut (above Boykov, Y., Veksler, O., Zabih, R., “Fast approximate energy minimization via graph cuts.”, Pattern ”Analysis and Machine Intelligence, IEEE Transactions on 23 (11), 2001, see pp.1222-1239.) An example of using an energy function often used as a preset objective function will be described. The objective function used in this embodiment is shown in the following formula (4).

Here, N is a set of adjacent pixel pairs. x _p represents the label of the pixel p, a value of 0 if the background, taking a value of 1 if figure. Also, I _p denotes a pixel value of the pixel p, y _p represents the value of the estimated shape of the pixel p, the estimated shape takes a value of 0 if the background, taking a value of 1 if the figure .

In addition, F _p (I _p , y _p ) and B _p (I _p , y _p ) in the above formula (4) are defined by the following formulas (5) and (6). Here, the following equation (5) represents the cost of assigning the pixel p as a graphic, and the following equation (6) represents the cost of assigning the pixel p as a background.

_Ip represents the pixel value of the pixel p, and _Iq represents the pixel value of the pixel q. Further, λ ₁ and λ ₂ are predetermined positive constants. Further, P _pq (I _p , I _q ) in the above formula (4) is defined by the following formula (7), and evaluates the difference in pixel value between the adjacent pixels p and q.

In the present embodiment, a branch-and-bound search algorithm, which is one of general optimization algorithms, is used as an optimization method for the objective function shown in the above equation (4). In the present embodiment, a method for efficiently calculating the lower bound of the objective function for the shape set is proposed. An important idea for efficiently calculating the lower bound of the objective function for a shape set is that if only the vertices of the convex polygon are examined, the lower bound of the objective function for all shape sets contained in the convex polygon can be found. is there.

When the above equation (4) is expressed using a function g (α) instead of the shape y, it can be expressed by the following equation (8).

[Branching processing]
In the branching process of the branch and bound search algorithm, when a parent node H ₀ (∈R _α ) is given, the parent node H ₀ is decomposed into child nodes H ₁ and H ₂ , and the objective function shown in the above equation (8) is obtained. Is represented by the following equation (9).

Here, for the dividing process in the branching process, various dividing methods are possible. In the present embodiment, the parent node H ₀ becomes the child nodes H ₁ and H _{2 as} shown in the following expressions (10) and (11) by the sign of φ ^k (α) shown in the expression (1). Is divided into Note that the pixel k represents a pixel selected from the set Q by sampling.

Here, the set Q is a set of pixels k such that the straight line φ ^k (α) = 0 that cuts the node H ₀ in the eigenspace, as shown in the following Expression (12). If the eigenspace is three-dimensional or more, the set Q is a set of pixels k such that the hyperplane φ ^k (α) = 0 that cuts the node H ₀ in the eigenspace.

V ₀ represents a vertex set of convex polygons represented by the node H ₀ . V ₀ can be obtained by analytically solving simultaneous equations of φ ^e (α) corresponding to the edge of H ₀ .

[Bounding processing]
In branching processing of the branch and bound search algorithm, the lower bound L (H _i ) (i = {1, 2}) is given by the following equations (13) to (14).

Equation (13) above is a Jensen inequality for the minimization problem. The change between “max” and “min” in the above equation (14) is based on the minus sign in the above equation (5). Further, the maximum value and the minimum value of φ (α) at α∈H _i are obtained from the vertex set V _{i of} H _i based on the basic theory of linear programming, and the following equations (15) and (16) Can be expressed as

In addition, the segmentation unit 114 repeats dividing the convex polygon in the eigenspace so that the areas of the two convex polygons after the division correspond to each other (so that the areas are substantially equal). A convex polygon on the eigenspace including a point indicating the shape parameter is searched.

Table 1 shows the pseudo code of the segmentation algorithm by the segmentation unit 114. First, when the target image I is given, the segmentation unit 114 starts the processing by setting the entire eigenspace R _α = {rεR ^d ||| r || _∞ ≦ w} as the parent node H ₀ . That is, the segmentation unit 114 starts processing with _Rα as a root, and divides the parent node H ₀ by the pixel k selected from the digital image using the function Select (Q). The divided child nodes H ₁ and H ₂ are put in Queue. However, it is assumed that the nodes included in the Queue are arranged so that the lower bound values are in ascending order. Next, the segmentation unit 114 selects the lowest lower bound node from Queue as the parent node H _0, and repeats the above processing until Q becomes an empty set. From the result at the end of the iteration, the optimum shape parameter is obtained and the optimum shape is obtained. Then, the segmentation unit 114 obtains an optimum segmentation result using the optimum shape.

H ₀ represents a convex polygon in the eigenspace. Note that the initial value of H ₀ corresponds to R _α . H ₁ and H ₂ represent two convex polygons formed by dividing H ₀ . Q is a set of pixels on the dividable digital image H _0. Queue represents a queue (used in Branch and bound search algorithm) containing a convex polygon node. Select (Q) represents a function for selecting a pixel k for division.

Also, a new proposal for Select (Q) for efficiently operating the algorithm is an important feature in the present embodiment. Here, Select (Q) is a function for selecting pixels for dividing the convex polygon from the image. In order to efficiently perform the search, the convex corresponding to the divided child nodes H ₁ and H ₂ is selected. A segmentation method in which the polygonal areas are almost equal was proposed in this embodiment, and the efficiency was verified with actual images.

In the present embodiment, the function shown in the following formula (17) is used as Select (Q).

here,

It is.

FIG. 8 shows a conceptual diagram of the function of the above equation (17). As shown in FIG. 8, the operation of the above equation (17) intuitively selects a pixel p that divides H ₀ into two regions (H ₁ , H ₂ ) having the same area.

The segmentation unit 114 uses the shape represented by the optimum shape parameter of the searched specific object as prior knowledge, and converts the object region that is the shape of the specific object into the spatial standardization unit 112 so as to optimize the objective function. Extract from the spatial standardized image generated by.

The output unit 120 outputs the object region extracted by the segmentation unit 114 as a result.

<Operation of Statistical Shape Model Generation Device 10>
Next, the operation of the statistical shape model generation apparatus 10 will be described.

The processing of the statistical shape model generation apparatus 10 shown in FIG. 9 is performed based on a program stored in the HDD 24 or the like by the CPU 21 in FIG. When a plurality of images in which pancreatic regions are obtained in advance are input to the statistical shape model generation apparatus 10, a learning process routine shown in FIG. 9 is executed.

First, in step S100, the learning receiving unit 12 receives a plurality of images in which a pancreas region is obtained in advance.

Next, in step S102, the learning unit 14 calculates eigenvectors and eigenvalues by principal component analysis based on a plurality of images in which the pancreas region received in step S100 is obtained in advance.

In step S104, the learning unit 14 stores the eigenvector and eigenvalue calculated in step S102 in the statistical shape model database 16, and ends the learning process routine.

<Operation of Image Processing Device 100>
Next, the operation of the image processing apparatus 100 will be described.

The processing of the image processing apparatus 100 shown in FIG. 10 is performed based on a program stored in the HDD 24 or the like by the CPU 21 in FIG. First, when eigenvectors and eigenvalues stored in the statistical shape model database 16 of the statistical shape model generation apparatus 10 are input to the image processing apparatus 100, they are stored in the statistical shape model database 106. Then, when a three-phase three-dimensional abdominal CT image is input as an input image to the image processing apparatus 100, the image processing routine shown in FIG. 10 is executed.

In step S200, the receiving unit 102 receives a three-phase three-dimensional abdominal CT image as an input image.

In step S202, the inter-image registration unit 110 performs registration between the early phase image, the portal phase image, and the late phase image received as the three-phase abdominal CT image of the three phases in step S200. To generate a registration image.

In step S204, the spatial standardization unit 112 generates a spatial standardized image using, for example, a nonlinear function, based on the alignment image generated in step S202.

In step S206, the segmentation unit 114 optimizes the objective function based on the input image received in step S200 in the eigenspace based on the pre-calculated eigenvector stored in the statistical shape model database 106. As described above, the shape parameter representing the shape of the subject represented by the spatial standardized image generated in step S204 is estimated to obtain the shape of the subject represented by the spatial standardized image. Step S206 is realized by the segmentation processing routine shown in FIG.

<Segmentation processing routine>
In step S300, the segmentation unit 114 sets the entire eigenspace R _α as the parent node H ₀ .

In step S302, the segmentation unit 114, as shown in the equation (12), the set of splittable pixel k parent node _{H 0,} is set as the set Q.

In step S304, the segmentation unit 114 initializes Queue.

In step S306, the segmentation unit 114 includes a parent node _{H 0} set in the step S300, the lower bound L corresponding to the parent node _{H 0} is calculated according to the equation (14); a combination of _(H 0 I) And stored in the Queue initialized in step S304.

In step S308, the segmentation unit 114 selects the pixel k from the set Q set in step S302 or the set Q updated in step S316 described later according to the above equation (17).

In step S310, the segmentation unit 114, the above formula parent node _{H 0} is set in the step S300, the or parent node _{H 0} updated in step S314 to be described later, by using a pixel k selected in step S308 (10) and divided according to (11), sets the child nodes _{H 1} and _{H 2.}

In step S312, the segmentation unit 114, the lower bound L corresponding to the child node _{H 1} calculated in accordance with the set child nodes _{H 1} and the formula in the above step S310 (14); a combination of _(H 1 I) , Stored in Queue. Further, the combination of the child node H ₂ set in step S310 and the lower bound L (H ₂ ; I) corresponding to the child node H ₂ calculated according to the above equation (14) is stored in Queue.

In step S314, the segmentation unit 114, among the stored node in Queue, select the lowest lower bound of the node, the parent node _{H 0,} updates the selected node.

In step S316, the segmentation unit 114, based on the parent node _{H 0} updated in the step S314, the as shown in the equation (12), the set Q, the set of splittable pixel k of the parent node _{H 0} Update to If there is no pixel k that can divide the parent node H ₀ , the set Q is updated to an empty set.

In step S318, the segmentation unit 114 determines whether or not the set Q is an empty set. If the set Q is an empty set, the process returns to step S308. On the other hand, if the set Q is not an empty set, the process proceeds to step S320.

In step S320, the segmentation unit 114 is finally updated by selecting the shape parameter alpha ^* in the parent node H ₀ above step S314, the substituted a selected shape parameter alpha ^* the function g, the specific Determine the shape y ^* of the object.

In step S322, the segmentation unit 114 subjects the region x of the subject having the shape of the specific object so as to optimize the objective function based on the shape y ^* and the objective function of the specific object determined in step S320. ^* Is extracted from the spatial standardized image generated in step S204.

In step S324, the segmentation unit 114 outputs the subject region x ^* extracted in step S322 as a result.

Next, returning to the image processing routine, the output unit 120 outputs the subject region x ^* extracted in step S206 as a result in step S208, and ends the image processing routine.

As described above, in the image processing apparatus 100 according to the present embodiment, as a function by computer processing based on a program, (A) eigenvectors calculated in advance based on a plurality of images in which a subject area is obtained in advance are used as a basis. (B) in the eigenspace where the point on the eigenspace indicates the shape parameter of the statistical shape model representing the shape of the specific object, the specific parameter represented by the shape parameter indicated by the point on the eigenspace A shape parameter that represents the shape of the subject represented by the input image so as to optimize a predetermined objective function that represents a value depending on the likelihood of the shape of the object and the pixel value difference between adjacent pixels in the input image Is estimated. The area of the subject is extracted from the input image using the shape of the specific object represented by the estimated shape parameter as prior knowledge. Thereby, it is possible to accurately extract the region of the subject while suppressing an increase in the calculation amount.

Also, the image processing apparatus 100 according to the present embodiment can extract a subject area at high speed.

Also, one important difference between the proposed algorithm and the conventional algorithm is that no preprocessing is required. Conventionally, preprocessing for preparing a shape in advance and performing clustering is required, and considerable time and memory are required for the preprocessing. However, the proposed algorithm in this embodiment does not require any preprocessing. This realized cost reduction.

Moreover, in the embodiment of the present embodiment, an experiment for segmenting the pancreas from 140 three-dimensional abdominal CT images was performed to verify processing accuracy and processing time, and the following results were obtained.

The world's highest accuracy for pancreas segmentation from 1.3D abdominal CT images was obtained.
2. The value after minimization of the objective function by the proposed algorithm is theoretically guaranteed to be smaller than that of the conventional method, but this was confirmed by actual images.
3. Compared with the maximum number of shapes handled by the conventional method (the above-mentioned US Pat. No. 8,249,349) being 10 ⁷ , the proposed method can handle a number of 10 ⁹ or more. In addition, the image is conventionally limited to a two-dimensional image, but a three-dimensional image having a number of pixels nearly two orders of magnitude can be handled.

[Second Embodiment]
Next, a second embodiment will be described. In addition, since the structure of the statistical shape model production | generation apparatus and image processing apparatus which concerns on 2nd Embodiment becomes a structure similar to 1st Embodiment, it attaches | subjects the same code | symbol and abbreviate | omits description.

The second embodiment is different from the first embodiment in that a shape parameter representing the shape of a subject is estimated using an approximate solution.

FIG. 12 shows a diagram for explaining the approximate solution in the second embodiment. As shown in FIG. 12, unlike the exact solution used in the first embodiment, the shape parameter is estimated by the approximate solution in the second embodiment. In the second embodiment, as shown in FIG. 12, sampling points are set in a lattice shape on the eigenspace, and the convex polygon is divided using a straight line determined from the sampling points. Note that the sampling point is set according to a predetermined sampling parameter k. For example, if the sampling parameters k = 4, 2 ⁴ sampling points for each dimension of the eigenspace is set.

First, the segmentation unit 114 of the image processing apparatus according to the second embodiment sets sampling points in a lattice shape on the eigenspace.

Then, the segmentation unit 114 sets a straight line from the sampling points set in the eigenspace so that the areas of the two convex polygons after the division correspond to each other, and divides the convex polygon by the set straight line. Thus, the search for the convex polygon including the point indicating the optimum shape parameter is repeated, and the shape parameter representing the shape of the subject represented by the input image is estimated.
Specifically, in the pseudo code of the segmentation algorithm shown in Table 1 above, instead of the process of dividing the parent node H ₀ by the pixel k selected using the function Select (Q), a straight line determined from the sampling points using, performs processing for dividing a parent node H _0.

In the second embodiment, by using an approximate solution, the number of dimensions of the eigenspace can be increased, and the shape parameter can be estimated with high accuracy.

FIG. 13 shows the recognition results of 140 cases of pancreas by the image processing apparatus according to the second embodiment. In FIG. 13, a broken-line circle represents a pancreas recognition result by the image processing apparatus according to the first embodiment, and a solid-line circle represents a pancreas recognition result by the image processing apparatus according to the second embodiment. .
As shown in FIG. 13, it can be seen that by using the approximate solution, the segmentation accuracy higher than the exact solution is achieved by increasing the number of dimensions of the eigenspace.

FIG. 14 shows an approximate solution of the second embodiment, a solution when the method described in the above-mentioned US Pat. No. 8,249,349 is applied to pancreas segmentation, and the strictness of the first embodiment. The comparison result of the calculation time of the solution is shown. FIG. 14 shows a change in calculation time when the dimension d of the eigenspace is increased. For the exact solution, the upper limit of calculation time was 100 h. For the approximate solution, the value of the sampling parameter was set to k = 4. The CPU used was 3.1 GHz Intel (R) Xeon (R), 1 CPU was used for preprocessing, and 2 CPU was used for optimization.

As shown in FIG. 14, when the approximate solution is used, the method described in the first embodiment and the method described in US Pat. No. 8,249,349 are applied to the pancreas segmentation. It can be seen that the calculation cost can be greatly reduced when the number of dimensions of the eigenspace is increased compared to the case where

Note that other configurations and operations of the image processing apparatus according to the second embodiment are the same as those of the first embodiment, and thus description thereof is omitted.

As described above, according to the image processing apparatus according to the second embodiment, sampling points are set in a lattice shape on a convex polygon in the eigenspace, and set to a convex polygon in the eigenspace. The shape parameter representing the shape of the subject represented by the input image is estimated by repeating the search by dividing the convex polygon using the straight line. The area of the subject is extracted from the input image using the shape of the specific object represented by the estimated shape parameter as prior knowledge. Thereby, it is possible to accurately extract the region of the subject while suppressing an increase in the calculation amount.

In addition, a more advanced statistical shape model with an increased number of dimensions can be used by the approximate solution method.

[Third Embodiment]
Next, a third embodiment will be described. In addition, since the structure of the statistical shape model production | generation apparatus and image processing apparatus which concerns on 3rd Embodiment becomes a structure similar to 1st Embodiment, it attaches | subjects the same code | symbol and abbreviate | omits description.

The third embodiment is different from the second embodiment in that a shape parameter representing the shape of a subject is estimated using an approximate solution that randomly sets sampling points.

FIG. 15 shows a diagram for explaining the approximate solution in the third embodiment. As shown in FIG. 15, in the third embodiment, sampling points are set randomly on the eigenspace, and the convex polygon is divided using a straight line determined from the sampling points.

First, the segmentation unit 114 of the image processing apparatus according to the third embodiment randomly sets sampling points on the eigenspace.

Then, the segmentation unit 114 sets a straight line from the sampling points set in the eigenspace so that the areas of the two convex polygons after the division correspond to each other, and divides the convex polygon by the set straight line. Thus, the search for the convex polygon including the point indicating the optimum shape parameter is repeated, and the shape parameter representing the shape of the subject represented by the input image is estimated.

Specifically, in the pseudo code of the segmentation algorithm shown in Table 1 above, instead of the process of dividing the parent node H ₀ by the pixel k selected using the function Select (Q), a straight line determined from the sampling points It performs a process of dividing the parent node H ₀ using.

Note that other configurations and operations of the image processing apparatus according to the third embodiment are the same as those in the first or second embodiment, and thus description thereof is omitted.

As described above, according to the image processing apparatus according to the third embodiment, sampling points are set randomly on the convex polygon in the eigenspace, and set to the convex polygon in the eigenspace. The shape parameter representing the shape of the subject represented by the input image is estimated by repeating the search by dividing the convex polygon using a straight line. The area of the subject is extracted from the input image using the shape of the specific object represented by the estimated shape parameter as prior knowledge. Thereby, it is possible to accurately extract the region of the subject while suppressing an increase in the calculation amount.

[Fourth Embodiment]
Next, a fourth embodiment will be described. In addition, since the structure of the statistical shape model production | generation apparatus and image processing apparatus which concerns on 4th Embodiment becomes a structure similar to 1st Embodiment, it attaches | subjects the same code | symbol and abbreviate | omits description.

The fourth embodiment differs from the first to third embodiments in that the objective function is extended.

In the first embodiment, the above equation (4) is used as an objective function, and F _p (I _p , y _p ) and B _p (I _p , y _p ) in the above equation (4) The case defined in 5) and (6) has been described.

In the fourth embodiment, the objective function shown in the above equation (4) changes monotonously with respect to the value of the shape label in the pixel for the shape of the specific object represented by the shape parameter as the likelihood of the shape of the specific object. Define to include monotonic functions and extend the objective function.

Specifically, in the fourth embodiment, F _p (I _p , y _p ) and B _p (I _p , y _p ) in the above equation (4) are expressed by the following equations (18), (19 ) And extend the objective function.

FIG. 16 shows a diagram representing a sufficient condition for the lower bound of the objective function to satisfy monotonicity. This sufficient condition is that h ^F _p (y) and h ^B _p (y) (∀pεP) are monotone functions that monotonically decrease and monotonously increase with respect to y _q (∀qεP), respectively. As shown in FIG. 16, the label pair {y, y ′} ∈L ^{| P |}

When this is true, the relationship shown in the following equation (20) is established.

FIG. 17 shows an example of h ^F _p (y) and h ^B _p (y) that satisfy the relationship of the above formula (20). In the example shown in FIG. 17, h ^F _p (y) and h ^B _p (y) are defined by the distance function shown in the following equation (21).

By defining the monotone functions h ^F _p (y) and h ^B _p (y) by the signed distance function shown in the above equation (21), and reducing the penalty near the contour of the shape, Segmentation errors can be reduced.

As described above, according to the image processing apparatus according to the fourth embodiment, as the likelihood of the shape of the specific object in the objective function, the value of the shape label in the pixel with respect to the shape of the specific object represented by the shape parameter Is defined to include a monotonic function that changes monotonically with respect to. As a result, the region of the subject can be extracted with high accuracy.

In the fourth embodiment, the monotone functions h ^F _p (y) and h ^B _p (y) that minimize the energy at the correct contour of the shape of a specific object are introduced into the objective function. A better objective function can then be used.

[Fifth Embodiment]
Next, a fifth embodiment will be described. In addition, since the structure of the statistical shape model production | generation apparatus and image processing apparatus which concerns on 5th Embodiment becomes a structure similar to 1st Embodiment, it attaches | subjects the same code | symbol and abbreviate | omits description.

The fifth embodiment is different from the first to fourth embodiments in that the function g for mapping the parameter α on the eigenspace to the shape is expanded.

In the first embodiment, the case where the mapping function g in the equation (2) is defined by the Heaviside function F (·) in the equation (3) has been described.

In the fifth embodiment, the function g for mapping the shape parameter to the shape of a specific object is defined so as to include a predetermined function f, and the function g for mapping is extended.

Also, extended result as equation (23) below the functions g _p mapping, also made available shape representations other than the level set function (e.g., Kilian M. Pohl et al., "Logarithm Odds Maps for Shape Representation ”, Proc. Of Medical Image Computing and Computer-Assisted Intervention, Vol.4191, pp.955-963, 2006, etc.).

Here, f represents a predetermined monotonic function, and t represents a predetermined threshold value.

For example, a case where LogOdds is used as an example of a possible shape expression represented by the statistical shape model will be described. When the class representing the shape is 2, when using LogOdds as the statistical shape model, it is represented by the following equation.

FIG. 18 shows an example when LogOdds is used as a statistical shape model. As shown in FIG. 18, the shape representation is expanded by using LogOdds.

As described above, according to the statistical shape model generation apparatus according to the fifth embodiment, further advanced statistics can be obtained by expanding the function g to be mapped (introducing a predetermined function f and threshold t). A geometric shape model becomes available.

Note that the present invention is not limited to the above-described example, and various modifications and applications are possible without departing from the gist of the present invention.

Since this embodiment is basic, its application range is considered wide. Specifically, the present invention can be widely used for image recognition processing for a graphic whose shape or the like fluctuates statistically. For example, in addition to other organs and other medical images (for example, MR, PET, etc.), it can be used for face recognition from face images and processing for recognizing specific figures from general landscape images. is there. In the present embodiment, the pancreas is targeted, but the spleen can also be targeted as another organ, for example.

In the first embodiment, the case where LSDM is used as the statistical shape model has been described as an example. However, the present invention is not limited to this. The present invention can be used not only for LSDM but also for other statistical shape models. If the statistical shape model can be expressed by a linear function as in the above formula (1) (strictly speaking, the function φ for the pixel is a linear function), this embodiment is applied to other statistical shape models. Can be applied. Depending on the objective function and the optimization method, this embodiment can be applied to a statistical shape model other than the statistical shape model expressed by a linear function such as the above formula (1) shown in the embodiment. Can be applied.
Also, as shown in the fifth embodiment, various statistical shape models can be used by expanding the mapping function g.

In the first embodiment, the case where the eigenspace dimension is 2 (d = 2) has been described as an example. However, the present invention is not limited to this. For example, a higher dimensional eigenspace may be targeted. For example, as shown in the second and third embodiments, the eigenspace dimension can be increased by using an approximate solution.

In the above embodiment, when the dimension of the eigenspace is increased to 3 or more, the convex polygon corresponds to the convex multivesicle, and the area of the convex polygon corresponds to the volume of the convex multivesicle.
Therefore, when the dimension of the eigenspace is increased to 3 or more, the segmentation unit 114 repeatedly divides the convex multivesicle on the eigenspace and searches for the convex multivesicle including the point indicating the optimal shape parameter, A shape parameter representing the shape of the subject represented by the input image is estimated so as to optimize the objective function represented by the above equation (4).
Further, the segmentation unit 114 searches the convex multivesicle on the eigenspace including the point indicating the optimum shape parameter by calculating the lower bound of the objective function for the shape set included in the convex multivesicle in the search algorithm. Estimate the shape parameters.
Further, the segmentation unit 114 divides the convex multivesicle on the eigenspace so that the volumes of the two convex multivesicles after the division correspond to each other, and the convexity on the eigenspace including the point indicating the optimum shape parameter. Repeat the search for multivesicles to estimate the shape parameters.

In the embodiment, the case where the eigenspace is divided by the pixel k included in the set Q selected according to the above equation (17) has been described as an example. However, the present invention is not limited to this, and any other arbitrary It can also be applied to convex polygons created by the division method. For example, as shown in the second embodiment, the present invention can also be applied to a convex polygon that regularly divides an eigenspace, such as a lattice shape.
In this case, depending on the fineness of the division, “Tightness” (one of the three conditions for guaranteeing optimality) may not be established.

Here, Tightness will be briefly described.

The above formula (14) is shown as the following formula (24).

In this case, when the following equation for all the leaf nodes _{H i} (25) is satisfied, tightness is established.

When the fineness of the division is coarse, the above formula (25) may not be satisfied, and the Highness may not be satisfied, and only an approximate solution may be obtained. However, when the dimension d of the eigenspace is increased There is an advantage that the calculation cost can be further reduced. The point to be noted is that the user can select the balance between the calculation cost and the calculation accuracy by sacrificing the optimality, and is an important point regarding the extension of the proposed method of the present embodiment. is there. At this time, it is the same as the above-mentioned U.S. Pat. No. 8,249,349, etc. in that the optimality is not guaranteed, but it is noted that no pre-processing is required compared with the above-mentioned U.S. Pat. No. 8,249,349. I want.

For the objective function and algorithm for minimization, the objective function shown in the above embodiment, the branch and bound method and the graph cut method are used this time. It is applicable not only to the thing.

Further, after executing the approximate solution according to the second or third embodiment, the exact solution according to the first embodiment may be executed.

In the computer configuration shown in FIG. 2, a program for realizing the functions of the processing units according to the embodiment is recorded on a computer-readable recording medium, and the program recorded on the recording medium is stored in the computer system. The processing by each component may be executed by causing the program to be read and executed, or the program may be read using a communication function not shown.

The computer-readable recording medium means a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built in the computer system.

The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.

Further, the program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

In the present embodiment, each processing unit of the statistical shape model generation apparatus 10 shown in FIG. 1 and the image processing apparatus 100 shown in FIG. 7 is configured by a computer that can execute each function by a program. However, a hardware configuration including a logic element circuit may be used.

Further, regarding the computer configurations of the statistical shape model generation apparatus 10 and the image processing apparatus 100 shown in FIG. 2, the configuration contents may be changed as appropriate.

As described above, the embodiment to be implemented has been described in detail with reference to the drawings. However, the specific configuration is not limited to the embodiment, and includes a design and the like without departing from the scope of the invention.

Further, the program according to the embodiment may be provided by being stored in a storage medium.

A computer-readable medium according to an embodiment is a program for extracting a region of a subject from an input image representing a subject that is a specific object, the computer accepting means for receiving the input image, and (A) the subject And eigenspaces based on eigenvectors calculated in advance based on the plurality of images in which the area of the subject is obtained in advance, and (B) eigenspaces In the eigenspace where the upper point indicates the shape parameter of the statistical shape model representing the statistical variation of the shape of the specific object, the shape parameter indicated by the point on the eigenspace represents the input image. A predetermined objective function representing a value corresponding to the likelihood of the shape of the specific object and the difference in pixel value between adjacent pixels in the input image is optimized. The shape parameter representing the shape of the subject represented by the input image is estimated, and the region of the subject is extracted from the input image using the shape of the specific object represented by the estimated shape parameter as prior knowledge. A computer-readable medium storing a program for functioning as a segmentation means.

The entire disclosure of Japanese Application 2014-169911 is incorporated herein by reference.

All documents, patent applications, and technical standards mentioned in this specification are to the same extent as if each individual document, patent application, and technical standard were specifically and individually described to be incorporated by reference, Incorporated herein by reference.

Claims (19)

1. An image processing apparatus that extracts a region of a subject from an input image representing a subject that is a specific object,
   Receiving means for receiving the input image;
   (A) a plurality of learning images representing the subject, wherein the subject area is an eigenspace based on an eigenvector calculated in advance based on the plurality of images obtained in advance; (B) In an eigenspace in which a point on the eigenspace indicates a shape parameter of a statistical shape model representing a statistical variation of the shape of the specific object,
   Based on the input image, a value corresponding to the likelihood of the shape of the specific object represented by the shape parameter indicated by the point on the eigenspace and the difference in pixel value between adjacent pixels in the input image is represented. Estimating the shape parameter representing the shape of the subject represented by the input image so as to optimize a predetermined objective function;
   Segmentation means for extracting the region of the subject from the input image, using the shape of the specific object represented by the estimated shape parameter as prior knowledge;
   An image processing apparatus.
2. The segmentation means is in the eigenspace,
   While estimating the shape parameter representing the shape of the subject represented by the input image so as to optimize the objective function based on the input image,
   The image processing apparatus according to claim 1, wherein a region of the subject is extracted from the input image by using, as prior knowledge, the shape of the specific object represented by the estimated shape parameter.
3. The segmentation means includes:
   The convex multiplicity on the eigenspace including the point indicating the optimum shape parameter by repeatedly dividing the convex multivesicle on the eigenspace representing the shape set including the shape of the specific object represented by the point representing the optimum shape parameter Estimating the shape parameter representing the shape of the subject represented by the input image so as to optimize the objective function according to a search algorithm for searching for a cell;
   The image processing apparatus according to claim 1, wherein the region of the subject is extracted from the input image using the shape of the specific object represented by the estimated shape parameter as prior knowledge.
4. The segmentation means searches the convex multivesicle on the eigenspace including a point indicating an optimal shape parameter by calculating a lower bound of the objective function for the shape set included in the convex multivesicle in the search algorithm. Then, the shape parameter representing the shape of the subject represented by the input image is estimated, the shape of the specific object represented by the estimated shape parameter is used as prior knowledge, and the region of the subject is defined as the input image. The image processing apparatus according to claim 3, extracted from
5. The segmentation means divides the convex multivesicle on the eigenspace so that the volumes of the two convex multivesicles after the division correspond to each other, and includes a convex on the eigenspace including a point indicating an optimum shape parameter. By repeatedly searching for multivesicles, estimating the shape parameter representing the shape of the subject represented by the input image, and using the shape of the specific object represented by the estimated shape parameter as prior knowledge, the subject The image processing apparatus according to claim 4, wherein the region is extracted from the input image.
6. The segmentation means sets sampling points on the eigenspace, and divides the convex multivesicle using a hyperplane determined from the sampling points set on the eigenspace to indicate optimal shape parameters Repetitively searching for convex multivesicles in the eigenspace containing the parameter, estimating the shape parameter representing the shape of the subject represented by the input image,
   The image processing apparatus according to claim 4, wherein a region of the subject is extracted from the input image using the shape of the specific object represented by the estimated shape parameter as prior knowledge.
7. 6. The image processing apparatus according to claim 4, wherein the segmentation means arbitrarily sets a sampling point on the eigenspace.
8. The objective function includes a monotone function that monotonously changes as a likelihood of the shape of the specific object as a likelihood of a shape label in a pixel with respect to the shape of the specific object represented by the shape parameter. The image processing apparatus according to claim 1.
9. The image processing apparatus according to any one of claims 3 to 8, wherein the search algorithm is a branch-and-bound method and a graph cut method.
10. An image processing method in an image processing apparatus that includes a reception unit and a segmentation unit and extracts an area of the subject from an input image representing the subject that is a specific object,
    The accepting means accepting the input image;
    The segmentation means is (A) a plurality of learning images representing the subject, and a region of the subject is an eigenspace based on eigenvectors calculated in advance based on the plurality of images obtained in advance. And (B) in the eigenspace where a point on the eigenspace indicates a shape parameter of a statistical shape model representing a statistical variation of the shape of the specific object, the eigenspace is based on the input image. Optimize a predetermined objective function that represents a value corresponding to the likelihood of the shape of the specific object represented by the shape parameter indicated by the upper point and the difference in pixel value between adjacent pixels in the input image As described above, the shape parameter representing the shape of the subject represented by the input image is estimated, and the shape of the specific object represented by the estimated shape parameter is used as prior knowledge. The area of the object, extracting from the input image,
    An image processing method including:
11. The step of the segmentation means extracting the region of the subject from the input image;
    In the eigenspace,
    While estimating the shape parameter representing the shape of the subject represented by the input image so as to optimize the objective function based on the input image,
    The image processing method according to claim 10, wherein a region of the subject is extracted from the input image using the shape of the specific object represented by the estimated shape parameter as prior knowledge.
12. The step of the segmentation means extracting the region of the subject from the input image;
    The convex multiplicity on the eigenspace including the point indicating the optimum shape parameter by repeatedly dividing the convex multivesicle on the eigenspace representing the shape set including the shape of the specific object represented by the point representing the optimum shape parameter The shape parameter representing the shape of the subject represented by the input image is estimated so as to optimize the objective function according to a search algorithm for searching for a cell, and the specific object represented by the estimated shape parameter is estimated. The image processing method according to claim 11, wherein the region of the subject is extracted from the input image using the shape as prior knowledge.
13. The step of extracting the region of the subject from the input image by the segmentation means calculates an optimum shape parameter by calculating a lower bound of the objective function for the shape set included in the convex multivesicle in the search algorithm. The shape parameter of the specific object represented by the estimated shape parameter is estimated by searching for the convex multivesicle on the eigenspace including the point indicating the shape and estimating the shape parameter representing the shape of the subject represented by the input image. The image processing method according to claim 12, wherein a region of the subject is extracted from the input image with prior knowledge.
14. The step of the segmentation means extracting the region of the subject from the input image divides the convex multivesicle on the eigenspace so that the volumes of the two convex multivesicles after the division correspond, and By repeatedly searching for convex multivesicles on the eigenspace including points indicating optimal shape parameters, the shape parameters representing the shape of the subject represented by the input image are estimated, and the estimated shape parameters The image processing method according to claim 13, wherein the region of the subject is extracted from the input image using the shape of the specific object represented by the prior knowledge as the prior knowledge.
15. The step of the segmentation means extracting the subject area from the input image sets a sampling point on the eigenspace, and uses a hyperplane determined from the sampling point set on the eigenspace. The shape parameter representing the shape of the subject represented by the input image is estimated by repeatedly searching for the convex multivesicle on the eigenspace including the points indicating the optimum shape parameter by dividing the convex multivesicle. ,
    The image processing method according to claim 13 or 14, wherein a region of the subject is extracted from the input image by using the shape of the specific object represented by the estimated shape parameter as prior knowledge.
16. 15. The image processing method according to claim 13, wherein the step of the segmentation means extracting the object region from the input image arbitrarily sets a sampling point on the eigenspace.
17. The objective function includes a monotone function that monotonously changes as a likelihood of the shape of the specific object as a likelihood of a shape label in a pixel with respect to the shape of the specific object represented by the shape parameter. The image processing method according to claim 1.
18. The image processing method according to any one of claims 12 to 17, wherein the search algorithm is a branch and bound method and a graph cut method.
19. A program for causing a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 9.

Images