JP6827707B2 - Information processing equipment and information processing system - Google Patents

Description

The disclosed technology relates to information processing devices and information processing systems.

In recent years, many medical images such as computed tomography (CT) images and nuclear magnetic resonance (MR) images have come to be used in medical practice. Along with this, the load of work called image interpretation, in which a doctor makes a diagnosis based on hundreds of medical images and examines a treatment policy, is increasing. Therefore, there are increasing expectations for a technology related to computer-aided diagnosis (CAD) that uses information obtained by analyzing a medical image with a computer as an aid in diagnosis. In such a technique, in order to analyze the characteristics of organs / lesions and distinguish between benign and malignant lesions, it is necessary to identify and extract a region of interest such as a tumor. However, the region of interest in the medical image is characterized in that there are large variations in shape, pixel value (density) distribution, relationship with the peripheral region, and the like. Therefore, in order to extract a region of interest having such a large variation, it is necessary to prepare a plurality of extraction algorithms or a plurality of extraction parameters.

Patent Document 1 Discloses a technique for allowing a user to manually select at least one result from the results of a plurality of image analysis processes obtained based on a plurality of parameter candidates.

Japanese Unexamined Patent Publication No. 2008-104798

However, it is a burden on the user to manually select the processing result.

The disclosed information processing apparatus uses a plurality of different parameters to perform processing for extracting a region of interest for a medical image for each of the parameters, and whether or not each pixel of the medical image belongs to the extracted region of interest. a result obtaining unit that acquire a plurality of extraction results showing, before the first obtaining means for obtaining a first feature amount obtained based on the respective Ki抽out results, a method different from the method to obtain the extraction result using the second acquisition means for acquiring second characteristic amount corresponding to the region of interest for medical images, it makes a comparison between the first feature amount of more than the second feature amount, based on the comparison result The present invention includes a selection means for selecting one extraction result from the plurality of extraction results and specifying a region of interest in the medical image based on the selected extraction result .

According to the disclosed technique, it is possible to reduce the burden on the user when selecting an appropriate extraction result from a plurality of extraction results of the region of interest obtained from the medical image.

It is a figure which shows an example of the apparatus configuration of the image processing system which concerns on 1st Embodiment. It is a figure which shows an example of the functional structure of the image processing system which concerns on 1st Embodiment. It is a figure which shows an example of the processing procedure of the image processing of the image processing apparatus 100 which concerns on 1st Embodiment. It is a figure explaining an example of the setting method of the parameter information which concerns on 1st Embodiment. It is a figure explaining an example of the acquisition method of the extraction result group which concerns on 1st Embodiment. FIG. 1 is a first diagram illustrating an example of a method for calculating consistency according to the first embodiment. FIG. 2 is a second diagram illustrating an example of a method for calculating consistency according to the first embodiment. FIG. 3 is a third diagram illustrating an example of a method for calculating consistency according to the first embodiment. FIG. 4 is a fourth diagram illustrating an example of a method for calculating consistency according to the first embodiment. It is a figure which shows an example of the functional structure of the image processing system which concerns on 2nd Embodiment. It is a figure which shows an example of the processing procedure of the image processing apparatus 200 which concerns on 2nd Embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the components described in this embodiment are merely examples, and the technical scope of the present invention is determined by the scope of claims and is not limited by the following individual embodiments. Absent.

(First Embodiment)
(Overview)
The image processing apparatus according to the present embodiment extracts a region (area of interest) in which a predetermined anatomical site of a subject such as an organ or a tumor exists from a medical image (input image) of a target case. At this time, the present apparatus sets a plurality of extraction parameters used for extracting the region of interest, and performs extraction processing of the region of interest from the input image using each parameter. After that, the attributes of the region of interest (second attribute) in the input image before the extraction process and the attributes of the plurality of extraction results acquired by the extraction process (first attribute) are calculated. Based on the degree of coincidence (consistency) between the second attribute 1 and the plurality of first attributes, the extraction result most consistent with the actual region of interest is selected. By such a method, the optimum region extraction result can be automatically selected even in the region of interest where the variation in shape and density distribution is large. The first attribute corresponds to an example of the first feature amount, and the second attribute corresponds to an example of the second feature amount.

In the following description, the case where the lung nodule shown in the CT image is to be processed will be described, but the scope of application of the present invention is not limited to a specific target organ, a type of abnormal shadow, and a type of modality. Absent. Hereinafter, specific device configurations, functional configurations, and processing procedures will be described.

(Device configuration)
An image processing device 100 according to the first embodiment of the present invention and an image processing system 190 composed of each device connected to the image processing device 100 will be described in detail with reference to FIG. The image processing device 100 corresponds to an example of an information processing device. The image processing system 190 corresponds to an example of an information processing system.

In the image processing system 190, the image processing device 100 acquires an input image and extracts a region of interest reflected in the input image by image processing. The image processing system 190 includes an image capturing device 110 that captures an image of a subject, a data server 120 that stores the captured image, an image processing device 100 that performs image processing, and acquired input images and image processing results. It has a display unit 160 for displaying the above and an operation unit 170 for inputting an instruction from the user. The input image is an image obtained by subjecting the image data acquired by the image capturing apparatus 110 to, for example, image processing for making an image suitable for diagnosis. Further, the image processing device 100, the image capturing device 110, the data server 120, the display unit 160, and the operation unit 170 are connected to each other so as to be able to communicate with each other via wired or wireless. The display unit 160 may be separate from the image processing device 100, or may be provided integrally with the image processing device 100. Each part will be described below.

The image processing device 100 is, for example, a computer (information processing device), and performs image processing according to the present embodiment. The image processing device 100 includes a central processing unit (CPU: Central Processing Unit) 11, a main memory 12, a magnetic disk 13, and a display memory 14. The CPU 11 integrally controls the operation of each component of the image processing device 100.

The image processing device 100 may also control the operation of the image capturing device 110 by controlling the CPU 11. The main memory 12 stores a control program executed by the CPU 11 and provides a work area when the program is executed by the CPU 11. The magnetic disk 13 stores various application software including an operating system (OS: Operating System), device drivers for peripheral devices, and a program for performing image processing according to the present embodiment described later. The display memory 14 temporarily stores data to be displayed on the display unit 160. The display unit 160 is, for example, a liquid crystal monitor, and displays an image based on the output from the display memory 14. The operation unit 170 is, for example, a mouse or a keyboard, and the operator inputs position information and characters. The display unit 160 may be a touch panel monitor that accepts operation input, and the operation unit 170 may be a stylus pen. Each of the above components is communicably connected to each other by a common bus 18.

The imaging device 110 is, for example, a computer tomography device (CT: Computed Tomography), a nuclear magnetic resonance imaging device (MRI), a radiography device (DR) for capturing a two-dimensional radiograph, or the like. Is. The image capturing device 110 transmits the acquired image to the data server 120. An imaging control unit (not shown) that controls the image capturing apparatus 110 may be included in the image processing apparatus 100. The imaging control unit controls imaging by the image capturing device 110 based on the imaging order, and transmits the captured image to the data server 120, for example.

The data server 120 holds an image captured by the image capturing device 110. The data server 120 is, for example, a PACS (Picture Archiving and Communication System) server. The image processing device 100 acquires an image from the data server 120 via a network such as a LAN (Local Area Network).

(Functional configuration)
Next, an example of the functional configuration of the image processing apparatus 100 will be described with reference to FIG. When the CPU 11 executes the program stored in the main memory 12, the functions of each part shown in FIG. 2 are realized. The image processing device 100 includes an image acquisition unit 1000, a parameter acquisition unit 1010, an area extraction unit 1020, a consistency calculation unit 1030, an extraction result determination unit 1040, and a display control unit 1050. The image processing device 100 may have a plurality of CPUs, and the programs may be distributed and stored in a plurality of memories. Further, the main body that executes the program is not limited to the CPU. That is, each function shown in FIG. 2 is realized by executing a program stored in at least one memory by at least one or more processors.

The image acquisition unit 1000 acquires a medical image (input image) to be image-processed according to the present embodiment from the data server 120.

The parameter acquisition unit 1010 acquires a plurality of parameters for extracting the region of interest. For example, the parameter acquisition unit 1010 acquires parameters obtained in advance by machine learning or the like from the main memory 12. Further, the parameter acquisition unit 1010 may acquire the parameters input by the user via the operation unit 170.

The area extraction unit 1020 performs a process of extracting a region of interest from an input image using a plurality of parameters acquired by the parameter acquisition unit 1010, and acquires a plurality of extraction results (label images indicating the region of interest). More specifically, the region extraction unit 1020 acquires a plurality of extraction results corresponding to each parameter by performing the extraction process a plurality of times while changing the parameters for the same region of interest. That is, the region extraction unit 1020 corresponds to an example of a result acquisition means for acquiring a plurality of extraction results of the region of interest included in the medical image.

The consistency calculation unit 1030 first obtains the characteristics (first attribute) of the plurality of extraction results acquired by the region extraction unit 1020 and the characteristics (second attribute) of the region of interest in the input image acquired by the image acquisition unit 1000. calculate. The consistency calculation unit 1030 calculates the characteristics of each of the plurality of extraction results of the region of interest. That is, the consistency calculation unit 1030 calculates a plurality of first attributes. The consistency calculation unit 1030 corresponds to an example of a first acquisition means for acquiring a plurality of first feature quantities representing a plurality of extraction results. Further, the consistency calculation unit 1030 calculates a second attribute indicating the approximate shape (size) of the region of interest from the medical image using, for example, the scale space of the Laplacian of Gaussian (LoG) kernel. That is, the consistency calculation unit 1030 corresponds to an example of the second acquisition means for acquiring the second feature amount representing the region of interest by using a method different from the method for obtaining the extraction result.

In addition, the consistency calculation unit 1030 calculates the degree of coincidence (consistency) with the second attribute for each of the plurality of first attributes. That is, the consistency calculation unit 1030 acquires a plurality of consistencys.

The extraction result determination unit 1040 determines, for example, the extraction result having the highest consistency among the plurality of consistency as the extraction result of the region of interest, based on the plurality of consistency calculated by the consistency calculation unit 1030. That is, the extraction result determination unit 1040 corresponds to an example of a selection means for selecting one extraction result from a plurality of extraction results based on a comparison between the plurality of first feature amounts and the second feature amount.

The display control unit 1050 outputs the extraction result of the region of interest determined by the extraction result determination unit 1040 to the display unit 160, and causes the display unit 160 to display the extraction result. The display control unit 1050 corresponds to an example of the display control means for displaying the selection result by the selection means on the display unit.

At least a part of each part of the image processing device 100 may be realized as an independent device. The image processing device 100 may be a workstation. The functions of each part may be realized as software that operates on a computer, and the software that realizes the functions of each part may be realized on a server via a network such as a cloud. In the present embodiment described below, it is assumed that each part is realized by software running on a computer installed in a local environment.

(Processing flow)
Subsequently, the operation of the image processing configured as described above will be described. FIG. 3 is a diagram showing an example of an image processing processing procedure executed by the image processing apparatus 100 of the present embodiment. In the present embodiment, the input image is a CT image. The CT image is acquired as a three-dimensional shade image. Further, in the present embodiment, the region of interest shown in the input image is a lung nodule. The input image and the region of interest are not limited to the above example.

(Step S1100)
In step S1100, the image acquisition unit 1000 acquires an input image. That is, the image acquisition unit 1000 acquires the CT image from the data server 120 and stores it in the main memory 12 of the image processing device 100. In another example, the image acquisition unit 1000 directly acquires the image data captured by the image capturing device 110 via a communication means, and performs the present medical image subjected to image processing or the like to obtain an image suitable for diagnosis. It is acquired as an input image related to the form. When the image capturing device 110 is, for example, a CT device, image data composed of pixel values called HU (Hounsfield Unit) values is acquired from the image capturing device 110.

Here, the input image in the present embodiment is composed of a plurality of pixels whose positions can be specified by three-dimensional Cartesian coordinates (x, y, z). The pixel size, which is one of the attributes of an image, is defined for each of the three coordinate directions. In the present embodiment, the pixel sizes in each of the x, y, and z directions are described as r_size_x, r_size_y, and r_size_z, and an example in which all the values are 1.0 mm will be described. Since each pixel value of the input image is determined for each value of the three-dimensional coordinates (x, y, z), the input image is defined by the function I (x, y, z) having the three-dimensional coordinate values as arguments. Can be regarded as data.

(Step S1110)
In step S1110, the parameter acquisition unit 1010 acquires information (parameter information) regarding the extraction parameter of the region of interest. When the parameter for the parameter obtaining unit 1010 extracts a region of interest from the input image _{N P} pieces to obtain, each parameter _{P i (i = 1,2, ...} , N p) and.

As a method for extracting a region of interest from a medical image, a region expansion method, a level-set method, a Graph cut method, or a method such as Water flow or Dynamic programming can be used. Thus, the parameter P _i in this case can be expressed, for example the threshold (one factor) in the case of utilizing a region growing technique. In the case of using the Graph cut method as the extraction method, since the energy function E for use in Graph cut method includes two types of coefficients of the coefficient _{P S} of the coefficient _{P d} and smoothing term data section, each parameter P _i may be defined as P _i = (P _id , P _iS ) (i = 1, 2, ..., N _p ).

As described above, according to the extraction method, the one parameter, because it may contain multiple types of coefficients, referred to as the parameter set P _i in the following. Thus, the parameter information acquired in this step, a specific value of the number of parameter sets N _P and the respective parameter set P _i to be used in the extraction of the region of interest.

Specific values for each parameter set may be determined according to clinical diagnostic rules. In clinical imaging, lung nodules are classified into solid nodules (SN, Solid nodules) and ground-glass nodules (GGN, ground-glass nodules) according to the distribution of pixel values in the lung nodules. In addition, partial regions within lung nodules may be classified into soft tissue regions, calcified regions, air regions, and the like. The average concentration value in the partial region used to identify these various types of lung nodule regions can be used as a parameter set (threshold value) in the extraction method based on region expansion. The number of parameter sets N _P can be determined according to the user's experience or the task to be tackled. For example, if you want to extract two types of lung nodules between SN and GGN in separate parameter set, and N P _{= 2.}

Further, the parameter obtaining unit 1010, the value of N _P and each parameter set, using a machine learning, may be determined in advance based on the extraction accuracy of the lung nodule region. That is, a plurality of different parameters are obtained by machine learning using the extraction accuracy of the region of interest. In this embodiment, the region extraction of the lung nodule will be described as an example by performing the Graph cut method. For T learning cases in which the correct region of the lung nodule is known in advance (the doctor manually gives the correct label), the parameter acquisition unit 1010 uses the Graph cut method parameter set P = (P _d ,). varied within a certain range the value of the coefficient _{P d} and _{P S} of P _S), performing the extracting process by the region extracting unit 1020. The value of the coefficient P _d and P _S is varied manually or automatically. At that time, the total number of parameter sets is n, representing each of the parameter set _{P I (I = 1,2, ...} , n) in. Since each parameter set has an extraction target area that is good at such as shape and density value distribution, by using only one of the n parameter sets, the target area can be extracted accurately in all cases. Is difficult. Therefore, n number of the parameter set _{P I (I = 1,2, ...} , n) a plurality of _{(N P)} parameter set _{P i (i = 1,2, ...} , N p) Select Must be combined. In addition, N _P ≦ n.

When the parameter obtaining unit 1010 selecting and combining N _P pieces of n parameter set, it is necessary to select one optimal combination from all combinations. First, the region extraction unit 1020 applies each combination to all T cases to perform extraction processing, and the parameter acquisition unit 1010 calculates the average degree of agreement of extraction accuracy. At that time, for example, when the parameter set combination _Pi (i = 1, 2, ..., N _p ) is applied to all T cases, the average degree of agreement obtained is calculated.

とする。ここで、ｔはＴ症例中の任意の症例Ｃａｓｅ_ｔを表し、Ｒ_ｔは症例Ｃａｓｅ_ｔにおける抽出精度の評価値である。ここで、抽出精度の評価値Ｒ_ｔとは、例えば、症例Ｃａｓｅ_ｔで得た最も高い抽出精度である。Ｐ_ｉに属するそれぞれのパラメータセットによって得た症例Ｃａｓｅ_ｔの抽出結果の精度（症例Ｃａｓｅ_ｔの正解領域との一致度）をＲ_ｔｉとする際、Ｒ_ｔは以下の式

And. Here, t represents an arbitrary case Case _t in the T case, and R _t is an evaluation value of the extraction accuracy in the case Case _t . Here, the evaluation value R _t of the extraction accuracy is, for example, the highest extraction accuracy obtained in the case case _t . When each of the parameter sets of extraction results of cases Case _t obtained by the precision belonging to P _i (the degree of coincidence between correct region of cases Case _t) and _{R ti, R t} is the following formula

によって計算され、Ｎ_Ｐ個の抽出結果の中から抽出精度が最も高いものが選択される。

Is calculated by, with the highest extraction precision from the N _P-number of the extraction result is selected.

Parameter acquisition unit 1010, a combination P _i of when the highest average degree of coincidence represented by Formula 1 parameter set number is the optimum combination of time is N _p. Then, the parameter acquisition unit 1010 sets the average degree of agreement by the optimum combination as the evaluation value E. The parameter acquisition unit 1010 may determine the evaluation value E based on the maximum or minimum degree of agreement over all T cases, in addition to the average degree of agreement.

In general, the larger the value of the number of parameter sets N _P , the higher the evaluation value E. However, overall the consideration factors such as computational time and processing complexity, it is necessary to determine the N _P to an appropriate value. For example, with respect to all the learning cases, the region extraction unit 1020 while increasing the number of N _P 1 is the extraction process. Parameter acquisition unit 1010, the N _P when the increase amount of the evaluation value E caused by the increase in N _P is equal to or lower than a predetermined value (converged), it may be determined as the final parameter set number.

The setting of the extraction parameter set by machine learning may be performed based not only on the degree of agreement of the extraction results but also on the characteristics of the region of interest. For example, the case where the parameter acquisition unit 1010 sets the parameter based on the shape feature of the region of interest will be described.

As shown in FIG. 4, the learning case 500 is classified into three data sets according to the shape. This classification may be automatic or manual. Further, the number of classifications is not limited to 3, and may be 2 or 4 or more. 501 is a spherical or similar spherical data set that is close to a spherical shape, 505 is a polygonal data set consisting of a polyhedron such as a hexahedron, and 510 is an irregular data set in which various other shapes are mixed in one region of interest. In each learning data set, the parameter acquisition unit 1010 searches for the extraction parameter set, and sets the obtained parameter sets as _PS , P _P , and P _I , respectively. That is, a plurality of different parameters are obtained for each of the plurality of shape features in the region of interest. Here, as the search parameter set for each data set, the parameter obtaining unit 1010, for example, when a 1 parameter number N _P, the average extraction degree of coincidence may select the highest parameter set.

The following processing of the present embodiment will be described by using the optimum parameter set selected based on the average degree of matching of the extracts shown in Equation 1 when _NP = 3. The three selected parameter sets are P ₁ , P ₂ , and P ₃ . Further, in the present embodiment is set to N _{P =} 3, the N _P may be a value other than this number.

(Step S1120)
In step S1120, the region extraction unit 1020 acquires a plurality of lung nodule extraction results. Specifically, the region extraction unit 1020 extracts the input image I (x, y, z) acquired in step S1100 and the extraction parameter set acquired in step S1110 using P ₁ , P ₂ , and P _3. And obtain the results of extracting three lung nodules.

In the present embodiment, the case where the Graph cut method is used as an example of the region extraction process in the region extraction unit 1020 will be described. The parameter sets for Graph cut acquired in step S1110 are P ₁ (P _1d , P _1S ), P ₂ (P _2d , P _2S ), and P ₃ (P _3d , P _3S ), respectively. The area extraction unit 1020 sets the foreground area _Robj (pixels belonging to the extraction target), the background area R _bkg (pixels not the extraction target) preset by the user, and the data term and smoothing term to which the above parameter set is assigned. Based on this, the energy function of the Graph cut method is constructed. The region extraction unit 1020 applies the energy function to each pixel of the input image, and sets a set of pixels belonging to the boundary where the energy is minimized as the outline of the region of interest. At this time, the region extraction unit 1020 sets the region surrounded by the contour as the extraction result of the target lung nodule.

As shown in FIG. 5, extraction processing was performed using the parameter sets P ₁ (P _1d , P _1S ), P ₂ (P _2d , P _2S ), and P ₃ (P _3d , P _3S ), and the extraction results of each were performed. 601, 602 and 603 are obtained. The plurality of extraction results are the results of extracting the region of interest using a plurality of different parameters in one extraction method. The extraction result 601 is the extraction result by the region extraction unit 1020 when the parameter P ₁ is used, the extraction result 602 is the extraction result by the region extraction unit 1020 when the parameter P ₂ is used, and the extraction result 603 uses the parameter P ₃ . The extraction result by the region extraction unit 1020 is shown. In the extraction results 601-603, the black-painted portion indicates the region extracted as the region of interest, and different regions are extracted as the region of interest according to the difference in the parameters.

(Step S1130)
Next, in step S1130, the consistency calculation unit 1030 calculates the consistency. The consistency calculation unit 1030 uses the input image I (x, y, z) acquired in step S1100 and the three extraction results acquired in step S1120 to obtain the region of interest in the input image and the respective extraction results. Calculate the consistency.

In the present embodiment, the consistency is calculated as the degree of coincidence between the attribute of the region of interest estimated from the input image (second attribute) and the attribute calculated from the extraction result (first attribute). First, a process of estimating the second attribute from the input image by the consistency calculation unit 1030 will be described.

Although the exact contour of the region of interest in the input image is unknown, it is possible for the consistency calculation unit 1030 to estimate the approximate shape of the region of interest by analyzing the characteristics of the density distribution. Here, in the present embodiment, the approximate shape of the region of interest estimated from the input image is set as the second attribute. As an example of a specific method for estimating the approximate shape, the consistency calculation unit 1030 can use the scale space of the Laplacian of Gaussian (LoG) kernel.

First, assuming that the concentration value distribution in the region of interest follows a Gaussian distribution, the consistency calculation unit 1030 applies the LoG kernel to the pixels belonging to the region of interest while changing the standard deviation σ of the Gaussian distribution to multiple values (scales). To do. At that time, if the LoG kernel on the optimum scale (corresponding to the diameter of the circle that approximates the region of interest) is applied to the region of interest, the output value of the LoG kernel becomes maximum at the center of the region of interest. Therefore, the size (rough shape) of the region of interest can be estimated based on the scale of the LoG kernel at that time.

The estimation of the approximate shape of the region of interest by the LoG kernel can be performed on a three-dimensional or two-dimensional image. In this embodiment, a two-dimensional case will be described. As shown in FIG. 6, when the lung nodule 620 is present in the right lung field 615 of the input image 610, the consistency calculation unit 1030 first obtains an axial cross section 630, a sagittal cross section 631, and a coronal cross section 632 through the lung nodule 620. get. In this method, for example, the user inputs a seed point (S _point ) belonging to the lung nodule 620 via the operation unit 170, the consistency calculation unit 1030 acquires the coordinates of the seed point, and the axial, sagittal, including the S _point , Three cross sections of the coronal can be obtained from the input image. The seed point S _point may be acquired by using an automatic detection algorithm instead of manually.

Further, as shown in FIG. 7, in order to shorten the processing time, the consistency calculation unit 1030 further narrows the range including the region of interest with respect to the sagittal cross section 630, the sagittal cross section 631, and the coronal cross section 632, and the local cross section f. You may set _a , f _s , and f _c . In this embodiment, an example for estimating the approximate shape of the region of interest using a local sectional f _a 640 shown in FIG. 8 (a).

Two-dimensional orthogonal coordinates (x, y) local section _f a (x, y) is defined with respect to applying the LoG kernel with multi-scale, wherein

に示す出力曲面Ｓ（ｘ，ｙ，ｈ）を取得する（図８（ｂ）の出力曲面６４５）。なお、式（３）において、ＬｏＧ_ｈ（ｒ）はＬｏＧカーネルを表し、ｒは以下のように定義される。

The output curved surface S (x, y, h) shown in FIG. 8 is acquired (output curved surface 645 in FIG. 8B). In the equation (3), LoG _h (r) represents the LoG kernel, and r is defined as follows.

また、スケールを表すパラメータｈは、Ｓ（ｘ，ｙ，ｈ）の中央部の幅を表しており、

Further, the parameter h representing the scale represents the width of the central portion of S (x, y, h).

に設定される。また、記号「＊」は畳み込み演算を表している。

Is set to. The symbol "*" represents a convolution operation.

A specific process of rough shape estimation will be described with reference to FIG. The consistency calculation unit 1030 is obtained by varying the scale h within a certain range {h _min , ..., H _max } and acquiring the maximum output value of S (x, y, h) near the _{Point at} each scale h. Obtains the output curve C (h) (FIG. 9 (a)). The user can specify h _min and h _{max as} arbitrary values. When the scale h coincides with the size of the diameter 654 of the circle that approximates the region of interest shown in FIG. 9B, C (h) obtains the maximum value. Therefore, the consistency calculation unit 1030 selects a scale ( _hopt 652) corresponding to the maximum value 651 of C (h), and sets the value as a coefficient D ₁ representing the approximate shape (size) of the region of interest. For example, D ₁ indicates the diameter of a circle that approximates the region of interest. The radius of the circle that approximates the region of interest may be D ₁ . The consistency calculation unit 1030 can apply the same processing to the local cross sections f _c and f _s as described above, and can obtain coefficients D ₂ and D ₃ representing the approximate shape of the region of interest in each cross section. The coefficients D ₁ , D ₂ , and D ₃ representing the schematic shape are the second attributes estimated from the input image. That is, the consistency calculation unit 1030 acquires the second feature amount by applying the LoG (Laplacian of Gaussian) kernel to the medical image.

Next, the consistency calculation unit 1030 calculates the first attribute from the extraction result. In order to calculate the consistency with the second attribute, the first attribute needs to represent the approximate shape of the extraction result. Here, the approximate shape estimation from the extraction result 603 will be described as an example.

First, the matching degree calculation unit 1030, the extracted result 603, creates a local cross-section similar to the estimation of the general shape of the input image, and _{f 'a, f' c,} f 's. The consistency calculation unit 1030 calculates the approximate shape from each cross section using the extraction result of the region of interest. At that time, the consistency calculation unit 1030 can calculate the size of the region of interest using the foreground pixels (pixels belonging to the extraction result) in the local cross section. Matching degree calculation unit 1030, for example, 'to estimate the approximate circle of the foreground pixels in _a, the diameter of the circle D' f may be _{one (first} attribute). Further, the average distance between the boundary of the foreground pixel and the center of gravity may be D' ₁ . The radius of the approximate circle may be D' ₁ . That is, the consistency calculation unit 1030 acquires a feature amount indicating the shape of the extracted region of interest. Matching degree calculation unit 1030, f _'c, f' also in _s, performs the same processing as described above, to obtain the coefficient D _'2, D' ₃ representing the general shape of the extraction result of the region of interest as the first attribute be able to. As described above, the first feature amount (first attribute) and the second feature amount (second attribute) are comparable because they both show the shape of the region of interest. As an example, the first feature amount and the second feature amount indicate the diameter or radius of the region of interest.

Finally, the consistency calculation unit 1030 uses the second attribute (D ₁ , D ₂ , D ₃ ) and the first attribute (D' ₁ , D' ₂ , D' ₃ ) to determine the region of interest and the extraction result. Calculate the consistency ε with. Various methods can be used to calculate ε. For example, Equation 4

によって計算してもよい。なお、Ｅ（Ａ，Ｂ）は式５で表される。

It may be calculated by. E (A, B) is represented by Equation 5.

式４によると、εが大きいほど第２属性と第１属性との差が小さいため、整合度が高いことを示す。

According to Equation 4, the larger the ε, the smaller the difference between the second attribute and the first attribute, indicating that the consistency is higher.

In another example of consistency calculation, the consistency calculation unit 1030 calculates the consistency based on the volume of the region of interest calculated from each of the input image and the extraction result. Matching degree calculation unit 1030, for _example, on the basis of the _{D 1, D 2, D 3} , estimates the ellipsoid that approximates a region of interest from the input image, calculate the volume _{V I} of the ellipsoid. On the other hand, the volume of the foreground pixels in the extraction result 601 and _{V S1.} Then, the consistency calculation unit 1030 calculates the consistency ε by the equation 6.

整合度計算のさらに別の例では、入力画像および抽出結果それぞれから計算される注目領域の曲率に基づいて整合度算出部１０３０は整合度を計算する。例えば、整合度算出部１０３０は、抽出結果６０１における前景画素の輪郭をＳ_ｓ１として検出する。輪郭の検出は一般的な画素処理技術であるため説明を省略する。また、整合度算出部１０３０は、Ｓ_ｓ１における各画素の曲率を計算（距離変換画像に基づく曲率）し、それらの曲率の平均値をκ_ｓ１とする。ここで、κ_ｓ１の計算は例えば、抽出結果６０１に対して、前景領域における各画素と背景領域との距離値を計算する距離変換処理を実施し、その距離値に基づいて計算することができる。一方、入力画像においては、整合度算出部１０３０は、Ｓ_ｓ１と重なる部分の画素における曲率（濃度値に基づく曲率）を計算し、それらの曲率の平均値をκ_Ｉとする。

In yet another example of consistency calculation, the consistency calculation unit 1030 calculates the consistency based on the curvature of the region of interest calculated from each of the input image and the extraction result. For example, the consistency calculation unit 1030 detects the contour of the foreground pixel in the extraction result 601 as S _s1 . Since contour detection is a general pixel processing technique, the description thereof will be omitted. Further, the consistency calculation unit 1030 calculates the curvature of each pixel in S _s1 (curvature based on the distance conversion image), and sets the average value of those curvatures to κ _s1 . Here, the calculation of κ _s1 can be performed based on, for example, a distance conversion process for calculating the distance value between each pixel in the foreground region and the background region on the extraction result 601 and the distance value. .. On the other hand, in the input image, the consistency calculation unit 1030 calculates the curvature (curvature based on the density value) of the portion overlapping with S _s1 and sets the average value of those curvatures as κ _I.

Then, the consistency ε between the curvature κ _s1 and the curvature κ _I is calculated by Equation 6.

なお、上記整合度計算手法の何れかを用いて整合度を算出してもよいし、それぞれを組み合わせて整合度を算出してもよい。本実施形態では、式４によって、入力画像から計算された注目領域の概略形状と抽出結果６０１、６０２、６０３それぞれの概略形状との整合度を計算し、複数の整合度ε_１、ε_２、ε_３を取得する。

The consistency may be calculated by using any of the above-mentioned consistency calculation methods, or may be combined to calculate the consistency. In the present embodiment, the consistency between the approximate shape of the region of interest calculated from the input image and the approximate shape of each of the extraction results 601, 602, and 603 is calculated by Equation 4, and a plurality of consistency ε ₁ , ε ₂ , Get ε ₃ .

(Step S1140)
In step S1140, the extraction result determination unit 1040 determines an appropriate extraction result from a plurality of extraction results in the region of interest. The extraction result determination unit 1040 determines the final extraction result of the region of interest by using the plurality of extraction results acquired in step S1120 and the plurality of consistency acquired in step S1130.

In the present embodiment, the extraction result determination unit 1040 selects the most suitable extraction result as the final extraction result from the extraction results 601, 602, and 603. The extraction result determination unit 1040 may select, for example, the extraction result corresponding to the highest consistency from the consistency ε ₁ , ε ₂ , and ε ₃ calculated in step S1130. That is, the selection means selects the extraction result corresponding to the first feature amount having the highest consistency with the second feature amount among the plurality of first feature amounts.

Further, the consistency calculation unit 1030 calculates the feature amount (for example, sphericity, moment, elongation, etc.) of each extraction result itself according to the demand of the user, and the extraction result determination unit 1040 is based on this calculation result. The most suitable extraction result may be determined by adding the likelihood of whether or not the extraction result is the optimum extraction result to the above consistency.

The result of the region extraction selected in step S1140 is R _seg . Further, the extraction parameter set corresponding to R _seg is set as the optimum parameter set _OPT for the input image.

(Step S1150)
In step S1150, the display control unit 1050 causes the display unit 160 to display the result of the area extraction. The display control unit 1050 transmits the information regarding the extraction result R _seg selected in step S1140 to the display unit 160 connected to the image processing device 100, and causes the display unit 160 to display the information regarding the extraction result R _seg . The display control unit 1050 causes the display unit 160 to display, for example, an image (hereinafter, referred to as a composite image) synthesized by superimposing the R _seg and the input image. In another example, the display control unit 1050 causes the display unit 160 to display a composite image of a two-dimensional cross-sectional image cut on a predetermined surface and R _seg . In yet another example, the display control unit 1050 volume-renders the three-dimensional composite image and displays it on the display unit 160. That is, the display control unit 1050 superimposes the three-dimensional extraction result on the three-dimensional medical image and displays it on the display unit 160.

In addition, the display control unit 1050 controls the display form of the result of region extraction. For example, the outline of the extraction result R _seg is displayed as a curve. The display control unit 1050 may display the color inside the contour of the extraction result R _{seg on} the display unit 160 with a color different from the color outside the contour. At this time, the extraction results not selected by the extraction result determination unit 1040 among the plurality of extraction results may be displayed together in a color lighter than the display color of the selected extraction results. That is, the displayed extraction result is not limited to the extraction result of 1 selected by the extraction result determination unit 1040. For example, the display control unit 1050 may display a plurality of extraction results on the display unit 160 in a state in which the extraction results selected by the extraction result determination unit 1040 can be distinguished from other extraction results. It should be noted that a plurality of extraction results may be displayed at the same time, or the extraction results to be displayed may be switched and displayed.

In another example, the display control unit 1050 causes the display unit 160 to display a figure indicating the arrow on the input image so that the tip of the arrow indicates the extracted region. Further, the above-mentioned display form may be changed according to the size of the extracted area. For example, the display control unit 1050 displays the contour as a curve on the display unit 160 when the size of the extracted area is larger than a predetermined threshold value, and when it is smaller than the threshold value, the position of the extraction area is at the tip of the arrow. An arrow may be displayed on the display unit 160 so as to indicate. These display modes make it possible to improve the visibility of the input image and the region of interest.

In the image processing apparatus according to the first embodiment described above, the extraction process of the region of interest is performed by a plurality of parameter sets, and a plurality of extraction results are acquired. Further, the optimum extraction result is automatically selected based on the degree of coincidence between the first attribute calculated from each extraction result and the second attribute of the region of interest calculated from the input image. Therefore, it is possible to reduce the labor of users such as doctors.

(Modification example 1)
In the description of the first embodiment, the calculation of the consistency in step S1130 is the calculation of the degree of coincidence between the second attribute of the attention region calculated from the input image and the first attribute of the attention region calculated from the extraction result. The case of executing by is explained. However, the present invention is not limited to this example. For example, in step S1110, when the parameter acquisition unit 1010 sets the extraction parameter set based on the shape feature, the consistency calculation unit 1030 creates an average shape from the learning data in each shape category, and uses the average shape. Consistency may be calculated.

Specifically, for example, three types of learning data shown in FIG. 4 are used. The parameter acquisition unit 1010 searches for the optimum extraction parameter sets P _S , P _P , and P _I from each training data set. The parameter set P _S is obtained from the spherical data set 501, the parameter set P _P is obtained from the polygonal data set 505, and the parameter set P _I is obtained from the irregular data set 510. Then, the region extracting unit 1020, the optimum extraction parameters, each of the learning data sets corresponding to extract the region of the region of interest by using a (P _{S, P P,} either P _I), from the obtained extraction result mean shape _{a S, a P,} to create the _{a I.} The region extracting unit 1020, the input image, the parameter set _{P S,} P P, followed by extraction of the region of interest by _{P I,} the extraction results obtained _{R S,} R P, and _{R I.} Finally, the matching degree calculation unit 1030, _{A S} and _R S, _{A P} and _R P, _{A I} and _{R I,} calculates a matching degree of each attribute. Then, the extraction result determination unit 1040 selects the extraction result of the set having the highest degree of attribute matching as the final result. Here, the attribute calculation may use the same method as in the first embodiment.

Further, instead of the above-mentioned average shape of the training data, the extraction result determination unit 1040 may select the extraction result of the learning data that most closely resembles the input image in each training data set. In this case, for example, the region extraction unit 1020 calculates the degree of approximation between all the learning cases and the input image in the spherical data set 501, and sets the closest learning case as the reference case of the input image in the data set. select.

Here, the evaluation of the degree of approximation between the learning case and the input image is performed based on the degree of deformation when aligning the input image and the learning case using, for example, the Evaluation Deformation Differential Mapping (LDDMM) technique. You may be broken. Because of the high degree of approximation as the degree of deformation is small cases, the reference case B _S a small sample the least deformation degree. Polygon shaped data set 505, also do the same from the irregular data set 510, to the reference case in each _B P, and _{B I.} Then, as described above, the region extracting unit 1020, a parameter set P _S obtained from the training data to the input _image, P _P, followed by extraction of the region of interest using P _I, the extraction results obtained R _{Let S} , R _P , and R _I.

Finally, the matching degree calculation unit 1030 calculates _{B S} and _R S, _{B P} and _R P, _{B I} and _{R I,} matching degree of each attribute (matching degree). Then, the extraction result determination unit 1040 selects the extraction result of the set having the highest degree of attribute matching as the final result. The attribute calculation may use the same method as in the first embodiment.

(Modification 2)
In the first embodiment, the case where the region extraction unit 1020 uses a plurality of parameters in Graph-Cut to obtain a plurality of extraction results of the region of interest has been described, but the present invention is not limited to the above example. .. For example, the region extraction unit 1020 acquires a plurality of extraction results of the region of interest by the Graph-Cut method and the Level-set method. That is, the plurality of extraction results are the results of extracting the region of interest using a plurality of different extraction methods. Then, the consistency calculation unit 1030 acquires a plurality of first attributes as described above from each extraction result, and calculates the consistency with the second attribute. The extraction result determination unit 1040 selects the extraction result by the Graph-Cut method or the Level-set method based on the calculated consistency.

As described above, the extraction result determination unit 1040 may select an appropriate extraction result from a plurality of extraction results obtained by different region extraction methods.

The effect of the image processing apparatus according to the first embodiment will be described. The image processing apparatus in the present invention solves some problems existing in the known region extraction method. This problem is that, in the conventional region extraction method, when a single parameter set is used, it is not possible to accurately extract the region of interest with a large variation in shape. Moreover, even when a plurality of parameter sets are used, it is difficult to select the optimum extraction result conveniently and accurately. In other words, there is a problem that the region of interest cannot be extracted with high accuracy.

On the other hand, in the image processing apparatus according to the present embodiment, a plurality of attention region extraction results are obtained from the input image by a plurality of parameter sets suitable for the attention region having a large variation in shape. Further, the optimum extraction result can be efficiently and accurately selected based on the degree of agreement between the attribute of the region of interest obtained from the input image and the attribute obtained from the plurality of extraction results.

From the above, in the image processing apparatus according to the first embodiment of the present invention, the region of interest can be extracted with high definition.
(Second Embodiment)
Subsequently, the image processing according to the second embodiment will be described. In the present embodiment, in step S1110, the parameter acquisition unit 1015 sets the type of the parameter based on the image finding F of the region of interest, and in step S1140, the extraction result determination unit 1040 determines the extraction result of the region of interest. Then, in step S1145, the image finding determination unit 1045 determines the value of the image finding F in the region of interest. Then, the display control unit 1055 presents the determined image finding F to the user by displaying it on the display unit 160. Hereinafter, the image processing according to the second embodiment will be described in detail.

(Functional configuration)
An example of each functional configuration constituting the image processing apparatus 200 according to the second embodiment will be described with reference to FIG. The image processing device 200 executes the image processing according to the second embodiment of the present invention. The image processing device 200 includes an image acquisition unit 1000, a parameter setting unit 1015, an area extraction unit 1020, a consistency calculation unit 1030, an extraction result determination unit 1040, an image finding determination unit 1045, and a display control unit 1050. The configurations having the same functions as the respective parts constituting the image processing apparatus 100 according to the first embodiment are numbered the same as those in FIG. 2, and detailed description thereof will be omitted.

The parameter acquisition unit 1015 acquires a plurality of parameters for extracting the region of interest based on the image findings items of the region of interest. After that, the region extraction unit 1020 extracts the region of interest using a plurality of parameters, and the consistency calculation unit 1030 calculates the consistency between each extraction result and the input image. The image finding determination unit 1045 determines the extraction result of the region of interest based on the consistency, and further determines the value of the image finding item in the region of interest.

(Processing flow)
An example of image processing according to the second embodiment of the present invention will be described. FIG. 11 is a diagram showing an example of a processing procedure executed by the image processing apparatus 200 of the present embodiment. This embodiment is realized by the CPU 11 executing a program that realizes the functions of each part stored in the main memory 12. Since the processing of steps S2100, S2120, S2130, and S2140 is the same as the processing of steps S1100, S1120, S1130, and S1140 shown in FIG. 3, detailed description thereof will be omitted.

(Step S2100)
In step S2100, the image acquisition unit 1000 acquires the image to be processed (input image). Since this process is the same as step S1100, the description thereof will be omitted.

(Step S2110)
In step S2110, the parameter acquisition unit 1015 acquires the parameter information. The parameter acquisition unit 1015 sets the value of the parameter set for extracting the region of interest from the input image based on the clinical image finding item F.

For example, a case where the parameter acquisition unit 1015 acquires a parameter set by machine learning technology based on an image finding item of overall shape will be described. If there are seven items in the overall shape: spherical, spherical, lobulated, polygonal, wedge-based, linear, and irregular, the parameter acquisition unit 1015 collects the training data according to the seven items and seven subs. Classify into datasets. The number of classifications is not limited to 7, and may be any other number.

Next, the parameter acquisition unit 1015 searches for the optimum region extraction parameter set for each data set. Since the parameter set search is the same process as in step S1110, the description thereof will be omitted.

(Step S2120)
In step S2120, the region extraction unit 1020 acquires the extraction results of a plurality of lung nodules based on the input image acquired in step S2100 and the seven parameter sets acquired in step S2110. Since this process is the same as step S1120, the description thereof will be omitted.

(Step S2130)
In step S2130, the consistency calculation unit 1030 calculates the consistency based on the input image acquired in step S2100 and the plurality of extraction results acquired in step S2120. Since this process is the same as step S1130, the description thereof will be omitted.

(Step S2140)
In step S2140, the extraction result determining unit 1040, based on the obtained degree of matching calculation result in step S2130, the optimal extraction parameter _{P OPT} is a parameter when the extraction results to obtain a _{R seg} and _{R seg} of the region of interest decide. Since this process is the same as step S1140, the description thereof will be omitted.

(Step S2145)
Next, in step S2145, the image finding determination unit 1045 determines the image finding in the region of interest. The image finding determination unit 1045 determines the value of the image finding F in the region of interest based on the classification of the above-mentioned sub-dataset to which the optimum extraction parameter P _OPT acquired in step S2140 belongs. For example, when the optimum extraction parameter P _OPT is a parameter obtained from the sub-dataset of the item lobulated, the image finding determination unit 1045 determines the image finding F to be lobulated. That is, the image finding determination unit 1045 corresponds to an example of the determination means for determining the findings representing the region of interest based on the selection result by the selection means. More specifically, the image finding determination unit 1045 determines the finding representing the shape of the region of interest.

Further, instead of setting the overall shape finding to one value, for example, the likelihood of each of the seven items may be set. In this case, for example, the consistency of the extraction result acquired by the seven parameters can be used as the above-mentioned likelihood.

Further, in addition to the above-mentioned likelihood, the machine uses the input image I (x, y, z), the feature amount of the region of interest calculated from each extraction result, and the clinical information of the region of interest acquired in advance. The value of the overall shape finding may be determined by the learning technique.

(Step S2150)
In step S2150, the display control unit 1055 performs a process of displaying the result of the area extraction on the display unit 160. The display control unit 1055 transmits information about the value of finding F of the region of interest determined by the extraction result _{R seg} and step S2145 of the region of interest determined in step S2140 on the display unit 160 connected to the image processing apparatus 100 , Displayed on the display unit 160. Since the display of the extraction result R _seg is the same process as in step S1150, the description thereof will be omitted. On the other hand, the display control unit 1055, the value of findings F of the region of interest, is displayed near the internal or external R _seg. Further, the value of the finding F may be displayed when the user specifies the R _seg region via the operation unit 170 such as a mouse.

According to the image processing apparatus according to the second embodiment of the present invention, it is possible to specify the region of interest and at the same time present the attributes (findings) of the region of interest.

The image processing device and the image processing system in each of the above-described embodiments may be realized as a single device, or may be a form in which devices including a plurality of information processing devices are combined so as to be able to communicate with each other to execute the above-mentioned processing. Often, both are included in the embodiments of the present invention. The above processing may be executed by a common server device or a group of servers. In this case, the common server device corresponds to the image processing device according to the embodiment, and the server group corresponds to the image processing system according to the embodiment. The image processing device and the plurality of devices constituting the image processing system need not be present in the same facility or in the same country as long as they can communicate at a predetermined communication rate.

In the embodiment of the present invention, a software program that realizes the functions of the above-described embodiment is supplied to the system or device, and the computer of the system or device reads and executes the code of the supplied program. Including morphology.

Therefore, in order to realize the processing according to the embodiment on the computer, the program code itself installed on the computer is also one of the embodiments of the present invention. Further, based on the instruction included in the program read by the computer, the OS or the like running on the computer performs a part or all of the actual processing, and the function of the above-described embodiment can be realized by the processing. ..

An embodiment in which the above-described embodiments are appropriately combined is also included in the embodiment of the present invention.

100 Image processing device 1000 Image acquisition unit 1010 Parameter acquisition unit 1020 Area extraction unit 1030 Consistency calculation unit 1040 Extraction result determination unit 1050 Display control unit

Claims (15)

A process of extracting a region of interest for a medical image for each of the parameters using a plurality of different parameters is performed, and a plurality of extraction results indicating whether or not each pixel of the medical image belongs to the extracted region of interest are obtained. and the result acquisition means retrieve,
A first obtaining means for obtaining a first feature amount obtained based on the respective front Ki抽out results,
A second acquisition means for acquiring a second feature amount corresponding to a region of interest for the medical image by using a method different from the method for obtaining the extraction result.
The first feature amount of several to compare with the second feature amount, selects one of the extraction result from the plurality of extraction results based on the comparison result, the medical image based on the extracted result of the selected Selection means to identify the area of interest in
An information processing device characterized by being equipped with. The method according to claim 1, wherein the selection means selects the extraction result corresponding to the first feature amount having the highest consistency with the second feature amount among the plurality of first feature amounts. Information processing device. The information processing apparatus according to claim 1 or 2, wherein the first feature amount and the second feature amount indicate the shape of the region of interest. The information processing apparatus according to claim 3, wherein the first feature amount and the second feature amount indicate the diameter or radius of the region of interest. The information processing apparatus according to any one of claims 1 to 4, wherein the plurality of extraction results are the results of extracting the region of interest using a plurality of different parameters in one extraction method. The information processing apparatus according to claim 5, wherein the plurality of different parameters are parameters obtained by machine learning. The information processing apparatus according to claim 6, wherein the plurality of different parameters are parameters obtained based on the extraction accuracy of the region of interest. The information processing apparatus according to claim 5 or 7, wherein the plurality of different parameters are parameters obtained for each of the plurality of shape features in the region of interest. The information processing apparatus according to any one of claims 1 to 4, wherein the plurality of extraction results are the results of extracting the region of interest using a plurality of different extraction methods. Each of the plurality of extraction results is the result of extracting the region of interest by the region expansion method, the Level-set method, the Graph Cut method, or Dynamic programming.
The second acquisition means according to any one of claims 1 to 9, wherein the second acquisition means acquires the second feature amount by applying a LoG (Laplacian of Gaussian) kernel to the medical image. Information processing equipment. The information processing apparatus according to any one of claims 1 to 10, further comprising a determination means for determining a finding representing the region of interest based on the selection result by the selection means. The information processing apparatus according to claim 11, wherein the determination means determines a finding representing the shape of the region of interest. The information processing apparatus according to any one of claims 1 to 12, further comprising a display control means for displaying the selection result by the selection means on the display unit. The information processing apparatus according to claim 1, wherein the selection means selects at least one other extraction result in addition to the extraction result of 1. A means of imaging to obtain a medical image by photographing a subject,
A process of extracting a region of interest for the medical image for each of the parameters using a plurality of different parameters is performed, and a plurality of extraction results indicating whether or not each pixel of the medical image belongs to the extracted region of interest. and taken Tokusuru result acquisition means,
A first obtaining means for obtaining a first feature amount obtained based on the respective front Ki抽out results,
A second acquisition means for acquiring a second feature amount corresponding to a region of interest for the medical image by using a method different from the method for obtaining the extraction result.
The first feature amount of several to compare with the second feature amount, selects one of the extraction result from the plurality of extraction results based on the comparison result, the medical image based on the extracted result of the selected Selection means to identify the area of interest in
An information processing system characterized by being equipped with.

Images