CN111986216A - RSG liver CT image interactive segmentation algorithm based on neural network improvement - Google Patents

Abstract

The present invention proposes an improved region growing algorithm based on one-dimensional convolutional neural network to interactively segment liver CT images, and comprehensively considers pixel gray value, spatial information, different gradient values and other information as growth rules through the neural network , which improves the stability of the region growing method and enhances the algorithm's ability to segment complex edge structures. The specific steps are as follows: first, image preprocessing, extracting slices containing liver in CT image sequence set, and using window algorithm to convert CT image into grayscale image; then image edge detection, calculating the gradient value of pixels under different edge detection operators As the feature of the pixel, the pixel feature vector is formed; the next step is to build a network model, extract the training data set, and train the network model; the last step is segmentation, using the trained convolutional neural network model as the growth criterion of the region growing algorithm, using the mouse Click on the liver area to generate the initial segmentation results, and use morphological methods to fill the holes to obtain the final results.

Description

An Improved Interactive Segmentation Algorithm of RSG Liver CT Image Based on Neural Network

technical field

The present invention proposes an improved region growing algorithm (Region Seeds Growing, RSG) based on a one-dimensional convolutional neural network to interactively segment liver CT images, and the gray value, spatial information, different gradient values, etc. of pixels are considered as a whole through the neural network. A variety of information is used as a growth rule, which improves the stability of the region growing method and enhances the algorithm's ability to segment complex structures at the edge.

Background technique

CT is a non-invasive in vitro imaging method of organs. Because of its fast imaging speed, high resolution and good effect, it has become an indispensable and important method for clinicians to make medical diagnosis. The combination of visualization technology and medical image analysis has It dominates the diagnosis of liver disease. By segmenting liver CT images, extracting liver tissue and obtaining corresponding feature information, doctors can intuitively understand the details of the patient's liver, which plays a key role in the diagnosis and the formulation of the next treatment plan.

Current segmentation methods can be divided into three categories: manual, semi-automatic and fully automatic. Manual segmentation methods are cumbersome, time-consuming, and can suffer from inter- and intra-observer variability. Each pixel of the image needs to be manually assigned to its class, and while very accurate results can be obtained with this technique, the time required will limit the translation of some tasks into clinical practice. For some tasks, manual segmentation of a single case can take hours. Fully automated methods do not require human effort, and researchers have developed many automatic segmentation methods over the past few decades. However, fully automated segmentation methods rarely achieve sufficiently accurate and robust results to be clinically impractical. This is usually due to poor image quality (with noise, partial volume effects, artifacts and low contrast), large patient-to-patient variability, uneven appearance from pathology, and protocol differences between clinicians for a given structure Boundaries are defined differently.

To address the limitations of fully automated segmentation methods, an interactive segmentation method is feasible in clinical practice as it can provide higher accuracy and robustness in many applications, such as planning radiation therapy for brain tumors. Since providing manual annotations for segmentation is time-consuming and labor-intensive, effective interactive segmentation methods are important for practical use. A good interactive segmentation method should obtain accurate results with as little user interaction as possible, thereby improving interaction efficiency. Although a large number of interactive segmentation methods exist, most of them require a large amount of user interaction and take a long user time, or their underlying models have limited learning ability. For example, the widely used ITK-SNAP takes user-supplied seed pixels or blobs as starting points and adopts an active contour model for segmentation. It requires a lot of user interaction at the beginning, and once the initial segmentation is obtained, it is difficult to refine the base model with other user interactions. SlicSeg accepts user-provided scribbles in a single start slice to train an online random forest for 3D segmentation, but lacks flexibility for further user editing. Random Walks and Graph Cuts learn from doodles and allow users to provide other doodles for refinement. They used random walks and Gaussian mixture models (GMMs) as base models. However, they require a lot of doodles to obtain satisfactory segmentation results. In this paper, the convolutional neural network is used to improve the growth rules of the conventional region growing algorithm, and the segmentation image can be generated interactively by clicking the mouse.

SUMMARY OF THE INVENTION

The purpose of the present invention is to solve the problems of low accuracy and weak stability of liver CT image segmentation by the traditional region growing method, and proposes to use an improved region growing algorithm based on a one-dimensional convolutional neural network to interactively segment the liver CT image. The neural network comprehensively considers the gray value, spatial information, different gradient values and other information of pixels as growth rules. In order to achieve the above purpose, the steps of the present invention are as follows:

Step 1: Image preprocessing, extracting slices containing liver in the CT image sequence set, and using window algorithm (W/L) to convert CT images into grayscale images;

Step 2: Image edge detection, calculate the gradient value of the pixel under different edge detection operators as the feature of the pixel, and form a pixel feature vector;

Step 3: constructing a network model, extracting a training data set, and training a network model, the network takes a pair of pixel feature vectors as input, and takes the correlation coefficient of two pixels as output;

Step 4: Segmentation, using the trained convolutional neural network model as the growth criterion of the region growth algorithm, using the mouse to click the liver region to generate the initial segmentation result, and using the morphological method to fill the holes to obtain the final result.

The specific situation in the step 1 is as follows:

(1) Extract slices:

The dataset includes raw CT images and segmentation labels. In the label images, professionals have assigned 13 abdominal organs to numbers one-to-one, of which the liver corresponds to a number 6. Slice T satisfies: Start+5 < T <End-5. Among them, Start represents the serial number of the earliest number 6 in the label image sequence set, and End represents the serial number of the last number 6 in the label image sequence set;

(2) Image conversion:

The value g(i) of the pixel after processing using the Window-Leveling (W/L) window algorithm is:

，

，肝脏组织的CT值通常位于50～250之间，取ww=200，wl=150。in:

,

, the CT value of liver tissue is usually between 50 and 250, take ww=200, wl=150.

The specific situation in the second step is as follows:

，其中

为该像素的灰度值。 The image is filtered by the Sobel operator, Roberts operator, Canny operator, Gabor operator, Sobel_h operator, Sobel_v operator, and Robert_neg_diag operator respectively, and the obtained value is used as the eigenvalue of the pixel to form a pixel eigenvector:

,in

is the grayscale value of the pixel.

The specific situation in the third step is as follows:

(1) Extract data:

Limit the value area, within 10 pixels of the city block distance from the liver boundary:

对应的输出标签Y, The region contains two parts: the inner region of the liver and the region outside the liver with a distance of 10 pixels. Arbitrarily select two pixel combinations in the area to form an input sample X of the neural network,

The corresponding output label Y,

(2) Train the network model

归一化到（0，1）之间，表示输入两像素属于同一区域的概率：

，其中Z表示未激活前的输出值；使用二元交叉熵函数(binary cross entropy)作为网络的损失函数：The final layer of the network model uses the sigmoid activation function to output the value

Normalized to (0, 1), indicating the probability that the input two pixels belong to the same region:

, where Z represents the output value before activation; the binary cross entropy function is used as the loss function of the network:

和

相等时，loss才为0，否则loss就是个正数，且概率相差越大，loss就越大。only when

and

When they are equal, the loss is 0, otherwise the loss is a positive number, and the greater the probability difference, the greater the loss.

The specific situation in the fourth step is as follows:

(1) The trained convolutional neural network model is used as the growth criterion of the region growth algorithm. When judging the seed pixel, when the pixel f ₂ in the four neighborhoods of f ₁ is merged into the growth region represented by the seed pixel, the f ₁ , f ₂ are used as the input of the neural network, and the output result y ^' is obtained. When y ^' > 0.9, they are merged; otherwise, they are not merged. Repeat this step until all the pixels in the four fields of seed pixels do not meet the conditions; the initial seed pixels are selected by mouse click;

,其中

是孔洞填充的起始点，B是用来填充孔洞的结构元素，

是A的补集。对公式不断进行迭代计算，直到

，最终的填充结果是

和边界的并集，即最终分割结果。(2) Since the liver tissue contains blood vessels and tumors, there are holes in the liver region in the segmentation result. The basic principle of morphological filling of holes is:

,in

is the starting point for hole filling, B is the structural element used to fill the hole,

is the complement of A. Iteratively calculate the formula until

, the final filling result is

and the union of the boundaries, that is, the final segmentation result.

The present invention also includes such features:

Compared with the growth rules of the traditional region growing algorithm, only the single dimension of the gray value of adjacent pixels is compared. In this paper, the gray value, spatial information, different gradient values and other information of pixels are considered as the growth rules through the neural network. The stability of the algorithm is improved, and the processing ability of the algorithm to the complex structure of the edge is enhanced. Although only the pixels in the area near the liver are trained, the present invention can also effectively segment the untrained area.

Compared with other interactive methods, the interactive mode of the present invention is simple to operate, and the edge of the segmentation result is finer. The present invention is suitable for medical image segmentation with a single internal structure, and has less obvious effect on natural image segmentation with complex semantics.

Description of drawings

Fig. 1 is the method flow chart of the present invention

Figure 2 is a one-dimensional convolutional neural network architecture diagram

Detailed ways

It will be understood by those skilled in the art that some well-known structures and their descriptions may be omitted from the drawings. The present invention will be further described below in conjunction with the accompanying drawings and implementation steps.

The present invention proposes an improved region growing algorithm based on one-dimensional convolutional neural network to interactively segment liver CT images, and comprehensively considers pixel gray value, spatial information, different gradient values and other information as growth rules through the neural network , which improves the stability of the region growing method and enhances the algorithm's ability to segment complex edge structures.

Fig. 1 is the flow chart of the method of the present invention, first is image preprocessing, extracts slices containing liver in CT image sequence set, uses window algorithm to transform CT image into grayscale image; Then is image edge detection, calculates pixel to detect at different edges The gradient value under the operator is used as the feature of the pixel to form the pixel feature vector; the next step is to build a network model, extract the training data set, and train the network model; the last step is segmentation, using the trained convolutional neural network model as the region growing algorithm The growth criterion was used to generate the initial segmentation results by clicking the liver region with the mouse, and the morphological method was used to fill the holes to obtain the final results.

The specific implementation steps are:

Step1.1 Extract slices:

The dataset includes raw CT images and segmentation labels. In the label images, professionals have assigned 13 abdominal organs to numbers one-to-one, of which the liver corresponds to a number 6. Slice T satisfies: Start+5 < T <End-5. Among them, Start represents the serial number of the earliest number 6 in the label image sequence set, and End represents the serial number of the last number 6 in the label image sequence set;

Step1.2 Image conversion:

The value g(i) of the pixel after processing using the Window-Leveling (W/L) window algorithm is:

，

，肝脏组织的CT值通常位于50～250之间，取ww=200，wl=150。in:

,

, the CT value of liver tissue is usually between 50 and 250, take ww=200, wl=150.

，其中

为该像素的灰度值。 Step2 filter the image with Sobel operator, Roberts operator, Canny operator, Gabor operator, Sobel_h operator, Sobel_v operator, Robert_neg_diag operator respectively, and the obtained value is used as the feature value of the pixel to form the pixel feature vector:

,in

is the grayscale value of the pixel.

Step3.1 Extract data:

Limit the value area, within 10 pixels of the city block distance from the liver boundary:

对应的输出标签Y, The region contains two parts: the inner region of the liver and the region outside the liver with a distance of 10 pixels. Arbitrarily select two pixel combinations in the area to form an input sample X of the neural network,

The corresponding output label Y,

Step3.2 Train the network model:

归一化到（0，1）之间，表示输入两像素属于同一区域的概率：

，其中Z表示未激活前的输出值；使用二元交叉熵函数(binary cross entropy)作为网络的损失函数：The final layer of the network model uses the sigmoid activation function to output the value

Normalized to (0, 1), indicating the probability that the input two pixels belong to the same region:

, where Z represents the output value before activation; use the binary cross entropy function as the loss function of the network:

和

相等时，loss才为0，否则loss就是个正数，且概率相差越大，loss就越大。only when

and

When they are equal, the loss is 0, otherwise the loss is a positive number, and the greater the probability difference, the greater the loss.

Step4.1 Use the trained convolutional neural network model as the growth criterion of the region growth algorithm. When judging the seed pixel, when the pixel f ₂ in the four neighborhoods of f ₁ is merged into the growth region represented by the seed pixel, the f ₁ , f ₂ are used as the input of the neural network, and the output result y ^' is obtained. When y ^' > 0.9, they are merged; otherwise, they are not merged. Repeat this step until all the pixels in the four fields of seed pixels do not meet the conditions; the initial seed pixels are selected by mouse click;

,其中

是孔洞填充的起始点，B是用来填充孔洞的结构元素，

是A的补集。对公式不断进行迭代计算，直到

，最终的填充结果是

和边界的并集，即最终分割结果。Step4.2 Since the liver tissue contains blood vessels and tumors, there are holes in the liver area in the segmentation result. The basic principle of morphological filling of holes is:

,in

is the starting point for hole filling, B is the structural element used to fill the hole,

is the complement of A. Iteratively calculate the formula until

, the final filling result is

and the union of the boundaries, that is, the final segmentation result.

Figure 2 is a one-dimensional convolutional neural network architecture diagram. The neural network architecture of the present invention is shown in Figure 2, which is similar to the convolutional neural network. First, the convolution layer is performed, and then the two-dimensional input is one-dimensionalized by the flatten layer, and then transitioned to the fully connected layer, and finally the constant is output through the sigmoid activation function. probability value. But the convolutional layers of the network are different, using one-dimensional convolutions. The step size of the convolution kernel is 1, that is, for each convolution, the convolution kernel corresponds to an entire row of the vector, and adjacent rows are independent of each other, and no cross-merging is performed.

Claims (5)

1. an improved RSG liver CT image interactive segmentation algorithm based on neural network, is characterized in that, comprises the following steps: Step1: Image preprocessing, extract the slices containing the liver in the CT image sequence set, and use the window algorithm (W/L) to convert the CT image into a grayscale image; Step2: Image edge detection, calculate the gradient value of the pixel under different edge detection operators as the feature of the pixel, and form the pixel feature vector; Step3: Build a network model, extract the training data set, and train the network model. The network takes a pair of pixel feature vectors as input, and takes the correlation coefficient of two pixels as the output; Step4: Segmentation, using the trained convolutional neural network model as the growth criterion of the region growth algorithm, using the mouse to click the liver region to generate the initial segmentation result, and using the morphological method to fill the holes to obtain the final result. 2. a kind of RSG liver CT image interactive segmentation algorithm based on neural network improvement according to claim 1, is characterized in that, the concrete process in described Step1 is as follows: Step1.1 Extract slices: The dataset includes original CT images and segmentation labels. In the label images, professionals have mapped 13 abdominal organs to numbers, among which the number corresponding to liver is 6; slice T satisfies: Start+5 < T <End-5 , where Start represents the serial number of the earliest number 6 in the label image sequence set, and End represents the serial number of the last number 6 in the label image sequence set; Step1.2 Image conversion: The value g(i) of the pixel after processing using the Window-Leveling (W/L) window algorithm is:

in:

,

, the CT value of liver tissue is usually between 50 and 250, take ww=200, wl=150. 3. a kind of RSG liver CT image interactive segmentation algorithm based on neural network improvement according to claim 1, is characterized in that, the concrete process in described Step2 is as follows: Step2 Filter the image with Sobel operator, Roberts operator, Canny operator, Gabor operator, Sobel_h operator, Sobel_v operator, Robert_neg_diag operator respectively, and the obtained value is used as the eigenvalue of the pixel to form the pixel eigenvector:

,in

is the grayscale value of the pixel. 4. a kind of RSG liver CT image interactive segmentation algorithm based on neural network improvement according to claim 1, is characterized in that, the concrete process in described Step3 is as follows: Step3.1 Extract data: Limit the value area, within 10 pixels of the city block distance from the liver boundary:

The area consists of two parts: the inner area of the liver and the area with a distance of 10 pixels outside the liver; two pixel combinations are randomly selected in the area to form an input sample X of the neural network,

The corresponding output label Y,

Step3.2 Train the network model: The final layer of the network model uses the sigmoid activation function to output the value

Normalized to (0, 1), indicating the probability that the input two pixels belong to the same region:

, where Z represents the output value before activation; use the binary cross entropy function as the loss function of the network:

only when

and

When they are equal, the loss is 0, otherwise the loss is a positive number, and the greater the probability difference, the greater the loss. 5. a kind of RSG liver CT image interactive segmentation algorithm based on neural network improvement according to claim 1, is characterized in that, the concrete process in described Step4 is as follows: Step4.1 Use the trained convolutional neural network model as the growth criterion of the region growth algorithm. When judging the seed pixel, when the pixel f ₂ in the four neighborhoods of f ₁ is merged into the growth region represented by the seed pixel, the f ₁ , f ₂ are used as the input of the neural network, and the output result y ^' is obtained. When y ^' > 0.9, merge; otherwise, do not merge; repeat this step until all the pixels in the four fields of the seed pixels do not meet the conditions; the initial seed Pixels are selected by mouse click; Step4.2 Since the liver tissue contains blood vessels and tumors, there are holes in the liver area in the segmentation result; the basic principle of morphological filling of holes is:

,in

is the starting point for hole filling, B is the structural element used to fill the hole,

is the complement of A; iteratively computes the formula until

, the final filling result is

and the union of the boundaries, that is, the final segmentation result.

Images