WO2023221954A1

WO2023221954A1 - Pancreatic tumor image segmentation method and system based on reinforcement learning and attention

Info

Publication number: WO2023221954A1
Application number: PCT/CN2023/094394
Authority: WO
Inventors: 李劲松; 田雨; 周天舒; 董凯奇
Original assignee: 浙江大学
Priority date: 2022-05-19
Filing date: 2023-05-16
Publication date: 2023-11-23
Also published as: CN114663431B; CN114663431A

Abstract

Disclosed are a pancreatic tumor image segmentation method and system based on reinforcement learning and attention. The method comprises: extracting an ROI region by using a three-dimensional coarse segmentation model, and segmenting an ROI region image and an original image into 2D images along a z-axis; selecting two reference layers from the segmented ROI region image by using a reinforcement learning network, selecting a segmentation layer from the segmented original image, and jointly inputting the two reference layer and the segmentation layer into a two-dimensional fine segmentation model having cross-attention feature fusion modules; and using the cross-attention feature fusion modules between the layers to enable segmentation features to exchange information in the segmentation layer and the reference layers, to obtain a segmentation result of a pancreatic tumor. According to the present invention, related information of non-adjacent 2D images is learned using a cross-attention mechanism, so that not only the limitation that a 2D neural network cannot use interlayer information to accurately locate tumors, but also the problem in 3D neural network of inaccurate tumor segmentation of caused by redundancy and interference of 3D data information are avoided.

Description

Pancreatic tumor image segmentation method and system based on reinforcement learning and attention

Technical field

The present invention relates to the field of image segmentation, and in particular to a pancreatic tumor image segmentation method and system based on reinforcement learning and attention.

Background technique

The five-year survival rate after diagnosis of pancreatic cancer is about 10%, making it one of the malignant tumors with the worst prognosis. Computed Tomography (CT) has been widely used in cancer research, prevention, diagnosis and treatment, and is currently the main imaging diagnostic basis for the diagnosis and treatment of pancreatic cancer. Fully automatic segmentation technology for pancreatic tumors can realize large-scale clinical CT image processing, improve patient diagnosis and treatment, and accelerate related clinical research, which is of great significance to families, society, and the national economy.

The automatic segmentation of the pancreas and pancreatic tumors in CT images faces huge challenges. On the one hand, the differences between pancreatic tumors and the pancreas and other organs around the abdomen are small in CT images, making it difficult to have clear boundaries. On the other hand, the shape, size, and location of pancreatic tumors are not fixed and are highly complex. Moreover, the pancreas is a small abdominal organ, and pancreatic tumors are even smaller. Traditional methods and general neural network methods cannot accurately locate the target area. The existing pancreatic tumor segmentation still mainly relies on manual annotation by doctors. The annotation process is boring and inefficient. More importantly, pancreatic annotation often requires doctors' rich experience. For doctors, the annotation work is a big challenge.

The difficulties in developing segmentation algorithms for CT pancreatic tumors mainly lie in the following aspects:

1. With the widespread application of convolutional neural networks in image processing, convolutional neural networks are also widely used in medical image segmentation. The current mainstream segmentation method for three-dimensional images uses one or more layers of CT images as input, and uses a complex convolutional neural network to output predictions of the pancreatic region to achieve segmentation. Improve segmentation accuracy by learning prediction errors. Although certain results have been achieved, the neural network model segments these two-dimensional images independently, ignoring the intrinsic connections between these two-dimensional images, resulting in insufficient segmentation accuracy.

2. Although the information between adjacent layers is easier to use when directly using a three-dimensional neural network for segmentation, the three-dimensional neural network regards all slices as equally important, which will introduce a large amount of invalid information and interference information during segmentation. In addition, due to the small field of view of the convolutional neural kernel, it is difficult to effectively utilize the information between slices in non-adjacent layers.

Existing medical image segmentation methods use a cascade method for segmentation. First, a network is used for rough segmentation to obtain the ROI (region of interest) of the target area, and then a fine segmentation network is used for segmentation. The fine segmentation network often takes the probability map generated by the coarse segmentation network as input, and the fine segmentation network is only responsible for optimizing the results of the coarse segmentation. However, such a method will make the fine segmentation network unable to utilize information outside the ROI, amplify the prediction errors of the coarse segmentation network, and introduce a large number of false negatives. For small targets such as pancreatic tumors, the false negatives caused by the cascade method Sexual issues will be more prominent.

Contents of the invention

The purpose of the present invention is to propose a pancreatic tumor image segmentation method and system based on reinforcement learning and attention in view of the shortcomings of the existing technology. In view of the fact that the existing two-dimensional convolutional neural network pancreatic tumor CT cannot utilize inter-layer information, and the three-dimensional volumetric volume Accumulative neural network will learn the problem of erroneous position and shape information between layers. When labeling, clinicians often judge the approximate shape and position of the pancreas and tumor based on certain key slices, and rely on several key slices to perform other tasks. Layer segmentation, this method is efficient and accurate. In view of the problems existing in two-dimensional and three-dimensional networks, the present invention proposes a method of using reinforcement learning method to simulate the behavior pattern of clinicians in the process of labeling tumors, focusing the attention of a CT image sequence on several key CT layers. Secondly, in order to avoid the false negative problem caused by cascade networks, the inter-layer attention mechanism is used to flow information between layers to achieve accurate segmentation of pancreatic tumors.

The object of the present invention is achieved through the following technical solutions: On the one hand, the present invention provides a pancreatic tumor image segmentation method based on reinforcement learning and attention, which method includes the following steps:

(1) Collect pancreatic CT images of patients with pancreatic cancer and perform preprocessing, outline pancreatic tumor segmentation labels on CT images, and construct a pancreatic tumor segmentation training set;

(2) Construct a three-dimensional rough segmentation model for pancreatic CT rough segmentation, obtain the pancreatic ROI region of interest, and segment the image and label of the ROI region into 2D images along the z-axis;

(3) Construct a two-dimensional precise segmentation model with a cross-attention feature fusion module, and use the cross-attention feature fusion module between layers to enable segmentation features to interact with information in the segmentation layer and the reference layer;

(3.1) Split the data and labels of the training set into 2D images along the z-axis in the same way as the image of the ROI area in step (2), and randomly select the two 2D images segmented in step (2) as Reference layer, the 2D image after segmentation of the training set data is used as the segmentation layer; the reinforcement learning network is used to select the reference layer of pancreatic tumors;

(3.2) Each reference layer corresponds to a cross-attention feature fusion module, which interacts with the segmentation layer. The cross-attention feature fusion module unifies the feature dimensions of the reference layer and the segmentation layer, and then performs a splicing operation to perform the first step. For the second fusion, the first fusion result is subjected to a dot product operation with the segmentation layer features after the feature dimensions are unified to generate an information correlation matrix of the cross attention mechanism, and then the dot product operation is performed with the segmentation layer features after the feature dimensions are unified for the second time. Second fusion, and use the residual operation to fuse the second fusion result with the information of the original segmentation layer features as the segmentation result;

(4) Given a pancreatic tumor image to be segmented and preprocessed, input it into the three-dimensional rough segmentation model to obtain the ROI area and segment it, use the reinforcement learning network to select the reference layer, and segment the pancreatic tumor image to be segmented. Then select a segmentation layer, input the segmentation layer and reference layer into the two-dimensional fine segmentation model for segmentation, and obtain the tumor segmentation result.

Further, in step (1), the preprocessing process is specifically: adjust the voxel space distance of all data in the training set to 1mm; truncate the HU value of the image to between -100 and 240, and then normalize to 0 to 1.

Further, in step (2), the three-dimensional coarse segmentation model consists of two parts: encoding and decoding. The encoding part includes four encoding blocks, each encoding block is connected to a downsampling layer; the decoding part includes four decoding blocks, each of which Each decoding block is preceded by an upsampling layer; each encoding block and decoding block consists of a varying number of convolutional-activation layers.

Further, in step (2), the ROI area image is recorded as Corresponds to the pancreatic CT image of the nth pancreatic cancer patient in the training set; Divide it into a 2D image along the z-axis, so Represents the 2D image of the kth layer after segmentation. The label of the CT image after truncation is recorded as The label corresponding to the pancreatic CT image of the nth pancreatic cancer patient in the training set is also divided into 2D images along the z-axis, so Represents the 2D image label corresponding to the k-th layer, where K _RIO is the minimum layer number after truncation, and K _ROI ′ is the maximum layer number after truncation.

Further, in step (2), the loss function used in the three-dimensional coarse segmentation model is the cross-entropy loss function Loss _CE :

in, Represents the predicted rough segmentation result output by the network, Y _n is the pancreatic tumor segmentation label of the CT image, m is the number of pixels in the input image, y _j and z _j are the real label and predicted label of pixel j respectively, τ = 0 , 1 and 2 respectively represent the background, pancreas or pancreatic tumor; function I(·) is an indicative function, function log is a logarithmic function, and p(·) is the probability function predicted by the model.

Further, in step (3.1), the environment of the reinforcement learning network is the ROI area obtained from the original CT image, the state is the two-layer slice randomly selected along the z-axis, and the action is that each iteration agent moves along the last selected reference layer. The z-axis moves forward and backward, and each reference layer corresponds to an agent. The action value function is the loss function of the prediction result of the two-dimensional fine segmentation model and the real label, and the maximum reward value of the next action in the current state is calculated through the heuristic function; in During the iterative process, the negative feedback method is used to train the reinforcement learning network.

Further, after the reinforcement learning network is trained, the parameters of the reinforcement learning network are fixed, the reference layer is filtered using the reinforcement learning network, and the reference layer and segmentation layer are input into the two-dimensional fine segmentation model to complete the two-dimensional fine segmentation model training.

Further, in step (3.2), the two reference layers are recorded as and The segmentation layer is recorded as x _k'=c , and the 2D image of the k'th layer after x _k' is segmented. For the reference layer The interaction process with the segmentation layer x _k'=c and the reference layer It is consistent with the interaction process of the segmentation layer x _k'=c ; for the reference layer For the segmentation layer x _k'=c , the implementation of the cross-attention feature fusion module is as follows:

Place the reference layer and segmentation layer x _k'=c, after downsampling and multiple convolution operations, high-dimensional features F _k=a and F _k'=c are obtained respectively; F _k=a and F _k'=c are used as cross-attention feature fusion module input;

The cross-attention feature fusion module first uses two linear mapping functions g(·) and f(· ⁾ to convert the input features from three-dimensional to one-dimensional, and transform the one-dimensional features to make the dimensions of the related features consistent; The features F _k=a and F _k'=c are mapped through g(·) and f(·) to make the dimensions of the features unified:
F _k＝a ′＝g(F _k＝a )
F _k'=c ′=f(F _k'=c )

Parallel F _k=a ′ and F _k′=c ′, and then use a convolution kernel W ₁ for mapping operation, and fuse the two for the first time, and the fused features are used as reference features:
F _{k＝a||k'＝c} =W ₁ Cat(F _k＝a ′||F _k′＝c ′)

Cat(·) is the splicing operation along the channel direction;

Use F _k=a||k'=c and F _k'=c ′ to perform dot product operations to generate the information correlation matrix A of the cross-attention mechanism:
q=W _q (F _k=a||k'=c ), k=W _k (F _k'=c ′), v=W _v (F _k'=c ′)

Among them, W _q , W _k , W _v are three convolutions used to give each feature adaptive weight; sig(·) is the sigmoid function; D is the number of channels of the feature F _k'=c ';

The information correlation matrix A and v perform a dot product operation to complete the second fusion, and the residual operation is used to fuse the information of F _k'=c to v':
v'=Av+f'(F _k'=c )

Among them, f′(·) is a linear mapping function.

Further, in step (3), the two-dimensional precise segmentation model takes the segmentation layer and the reference layer as input, the prediction result of the segmentation layer as the output, and uses the loss function Dice Loss for negative feedback learning:

Among them, m' is the number of pixels in the input 2D image, y _k' represents the label of the 2D image of the k'th layer after segmentation, y _k'=c is the prediction result of the segmentation layer, y _h and z _h are respectively are the true labels and predicted labels of pixel h.

On the other hand, the present invention also provides a pancreatic tumor image segmentation system based on reinforcement learning and attention. The system includes a pancreatic tumor segmentation training set building module, a three-dimensional rough segmentation model module, a reinforcement learning network module and a two-dimensional fine segmentation module. model module;

The pancreatic tumor segmentation training set building module is used to collect and preprocess pancreatic CT images of patients with pancreatic cancer, outline pancreatic tumor segmentation labels on CT images, and construct a pancreatic tumor segmentation training set;

The three-dimensional rough segmentation model module is used to obtain the pancreas ROI region of interest, and segment the image of the ROI region and its label along the z-axis into 2D images;

The reinforcement learning network module is used to select two reference layers from the 2D image segmented by the three-dimensional coarse segmentation model module;

The two-dimensional fine segmentation model module is used to divide the data and labels of the training set into 2D images along the z-axis, and select a segmentation layer. The two-dimensional fine segmentation model module includes two cross-attention feature fusion sub-modules, Corresponding to two reference layers respectively, the two cross-attention feature fusion sub-modules interact with the segmentation layer respectively to unify the feature dimensions of the reference layer and the segmentation layer, and then perform a splicing operation to perform the first fusion. The first fusion result is subjected to a dot product operation with the segmentation layer features after the feature dimensions are unified to generate an information correlation matrix of the cross attention mechanism, and then the dot product operation is performed with the segmentation layer features after the feature dimensions are unified for the second fusion, and used The residual operation fuses the second fusion result with the information of the original segmentation layer features to obtain the tumor segmentation result.

Beneficial effects of the present invention:

1. Use the reinforcement learning network to select two layers of 2D images from the three-dimensional image as the reference layer, without involving the transfer of information between layers, and provide a reference segmentation example for the segmentation of the two-dimensional neural segmentation network.

2. Use the cross-attention mechanism to learn relevant information of non-adjacent 2D images, which not only avoids the limitation of 2D neural networks that cannot use inter-layer information to accurately locate tumors, but also avoids the redundancy and redundancy of 3D data information in 3D neural networks. Inaccurate tumor segmentation caused by interference.

3. Use a fully automated segmentation method to simulate the clinician's segmentation process. The training and verification processes do not require doctor's intervention.

Description of the drawings

Figure 1 is a flow chart of a pancreatic tumor image segmentation method based on reinforcement learning and attention provided by the present invention.

Figure 2 is a schematic diagram of the cross-attention feature fusion module of the present invention.

Figure 3 is a schematic structural diagram of the coarse segmentation model 3D UNet of the present invention.

Figure 4 is a schematic structural diagram of the precise segmentation model 2D UNet of the present invention.

Figure 5 is a flow chart of reinforcement learning training of the present invention.

Figure 6 is a schematic diagram of a pancreatic tumor image segmentation system based on reinforcement learning and attention provided by the present invention.

Detailed ways

The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

As shown in Figure 1, the present invention provides a pancreatic tumor segmentation method based on reinforcement learning and attention. The implementation steps are as follows:

(1) Creation and preprocessing of pancreatic tumor segmentation data set

(1.1) Collect CT volume data and make liver standard segmentation results of these data; collect pancreatic CT images of patients with pancreatic cancer, recorded as Outline the label for pancreatic tumor segmentation in CT images, recorded as y _j ∈ {0, 1, 2}, where |X| represents the number of all voxels in X, x _j represents the jth voxel in Represents a set of natural numbers, y _j =0, y _j =1, and y _j =2 respectively represent that voxel j belongs to the background, pancreas or pancreatic tumor. Let the pancreatic tumor segmentation data set be S={X _r , Y _r , r=1,...,N}, where N is the number of CT images, X _r is the pancreatic CT image of the r-th pancreatic cancer patient in S, Y _r is the corresponding CT image pancreatic tumor segmentation label. The segmented data set is divided into a training set T _train ={X _n , Y _n , n = 1, ..., N _train } and a test set T _val = {X _n' , Y _n' , n' = 1,. .., N _val }, N=N _train +N _val , where N _train is the number of training sets, N _val is the number of test sets, X _n is the pancreatic CT image of the nth pancreatic cancer patient in T _train , Y _n is the pancreatic tumor segmentation label of the corresponding CT image, X _n' is the pancreatic CT image of the n'th pancreatic cancer patient in T _val , and Y _n' is the pancreatic tumor segmentation label of the corresponding CT image.

(1.2) Adjust the voxel space distance of the x, y, and z axes of all data to 1mm. The HU value of the image is truncated between [-100, 240], and then normalized to between [0, 1]. The HU value is the CT value, which is a unit of measurement for measuring the density of a certain local tissue or organ in the human body. It is usually called Hounsfield unit (HU). Air is -1000, and dense bone is +1000.

(2) Use the 3D UNet network to construct a three-dimensional rough segmentation model M _c for pancreatic CT rough segmentation and conduct training.

(2.1) As shown in Figure 3, a 3D UNet network for coarse segmentation of pancreatic CT is constructed, denoted as 3D coarse segmentation model M _c . This model consists of two parts: encoding and decoding. The encoding part includes four encoding blocks, each Each coding block is followed by a downsampling layer. The decoding part consists of four decoding blocks, each of which is preceded by an upsampling layer. Each encoding block and decoding block consists of an varying number of convolutional-activation layers. The network is trained using training set samples, and the loss function used is the cross-entropy loss function Loss _CE :

in, represents the predicted rough segmentation result output by the network, m is the number of pixels in the input image, y _j and z _j are the real label and predicted label of pixel j respectively, τ = 0, 1, 2 respectively represent the background, pancreas or Pancreatic tumor; function I(·) is an indicator function, function log is a logarithmic function, and p(·) is a probability function predicted by the network.

(2.2) Obtain the pancreatic ROI (region of interest) area through the M _c model.

Obtain the prediction probability map of 3D CT images X _n in the training set T _train through the model M _c According to conditions The data is truncated on the 3D CT _image Will Divided into a 2D image along the z-axis, then Represents the 2D image of the kth layer after segmentation. The label of the CT image after truncation is recorded as It is also divided into a 2D image along the z-axis, so Represents the 2D image label corresponding to the k-th layer, where K _ROI is the minimum layer number after truncation, and K _ROI ′ is the maximum layer number after truncation.

(3) Use the 2D UNet network with a cross-attention feature fusion module to build a two-dimensional precision segmentation model M _F for pancreatic tumor segmentation (see Figure 4) and perform pre-training.

(3.1) Train a two-dimensional fine segmentation model M _F . The main function of this model is to use the cross-attention feature fusion module between layers so that segmentation features can interact with information in the main segmentation layer and the reference layer.

The training set data X _n and Y _n are arranged according to the 3D ROI area Slice in the same way, so

X _n =(x _k′ , k'∈[1, K]), x _k′ represents the 2D image of the k′th layer after segmentation, Y _n =(y _k′ , k′∈[1, K]) , y _k' represents the label of the 2D image of the k'th layer after segmentation. From the 3D ROI region obtained in step (2) Randomly select two slices of 2D images, layer a and layer b Among them, a, b∈[K _ROI , K _ROI ′]. use and As the reference layer, the cth layer of the training set data _X is segmented.

(3.2) Implementation of cross-attention feature fusion module. In the model, the present invention designs two inter-layer information interaction modules based on the cross-attention mechanism so that inter-layer information can interact in the reference layer and segmentation layer. The two cross-attention feature fusion modules are completely consistent. For reference layer For the segmentation layer x _k'=c , the cross-attention feature fusion module (see Figure 2) is implemented as follows. For the reference layer The interaction process with the segmentation layer x _k'=c and the reference layer It is consistent with the interaction process of segmentation layer x _k'=c :

Place the reference layer and segmentation layer x _k'=c, respectively, to obtain high-dimensional features F _k=a and F _k'=c after downsampling and multiple convolution operations. F _k=a and F _k′=c serve as inputs to the cross-attention feature fusion module.

The cross-attention feature fusion module first uses two linear mapping functions g(·) and f(·) to convert the input features from three-dimensional to one-dimensional, and transform the one-dimensional features to make the dimensions of the related features consistent. The features F _k=a and F _k'=c are mapped through g(·) and f(·) to make the dimensions of the features unified:
F _k＝a ′＝g(F _k＝a )
F _k'=c ′=f(F _k'=c )

Cat(·) is the splicing operation along the channel direction.

Among them, W _q , W _k , and W _v are three convolutions used to give each feature adaptive weights. sig(·) is the sigmoid function. D is the number of channels with feature F _k'=c '.

The correlation matrix A and v perform a dot product operation to complete the second fusion, and the residual operation is used to fuse the information of F _k'=c to v':
v'=Av+f'(F _k'=c )

Among them, f′(·) is a linear mapping function.

(3.3) Pre-training of the two-dimensional fine segmentation model M _F. Will And x _k'=c is used as the input, the prediction result y k' ₌ _{c of x k'=c} is the output, Dice Loss is the loss function for negative feedback learning, and the two-dimensional fine segmentation model M _F is trained.

The definition of Dice Loss is:

Among them, m' is the number of pixels in the input 2D image, y _h and z _h are the real label and predicted label of pixel h respectively.

(4) Reinforcement learning network training.

(4.1) Use reinforcement learning network Q to select pancreatic tumor segmentation layers.

The reinforcement learning network consists of a 3D ResNet network whose output is a vector that maps to the agent's action space. The entire reinforcement learning framework can be divided into the following parts: Environment, Agents, States, Action, Reword and Loss function. This invention explains the meaning of each part and the process of reinforcement learning:

environment ROI area obtained from original CT image as the entire reinforcement learning environment.

Agent: In order to select the reference layer a-th layer and layer b This invention sets up two agents Agent ₁ and Agent ₂ .

State: s ^(t) is defined at the iteration number t from The initial state of the two reference layers layer a and layer b selected by the reinforcement learning network is from Two slices randomly selected along the z-axis.

Action: The action strategy function of Agent ₁ and Agent ₂ is π(act ^(t) |st ^(t) ). Here, the present invention chooses a greedy strategy and traverses all actions in the action space. st ^(t) and act ^{( t)} are the state and the action of the current agent respectively. is the action space of Agent ₁ and Agent ₂ , specifically {-3, -2, -1, 0, 1, 2, 3, Stop}. Each action represents the last choice of Agent ₁ and Agent ₂ in each iteration. The reference layer moves forward and backward along the z-axis. The final Stop operation indicates the termination of Q selection, indicating that Agent ₁ and Agent ₂ cannot find a reference layer that can be upgraded.

Action value function: This invention uses the set of prediction results of all 2D volume data layers of a CT image X _n in the two-dimensional fine segmentation model M _F Dice loss with the real label Y is expressed as:

Heuristic function: The heuristic function is used to calculate the maximum reward value of the next action in the current state:

Among them, γ∈[0, 1] is the attenuation coefficient. The more actions, the smaller the benefit.

Loss function: During the iteration process, the negative feedback method is used to train the reinforcement learning network, so that agents Agent ₁ and Agent ₂ can quickly and accurately find the most appropriate reference layer. The loss function of the t-th iteration can be expressed as:

Instructions for the training steps of the reinforcement learning network (see Figure 5):

In one iteration t, the reinforcement learning network makes agents Agent ₁ and Agent ₂ learn from the environment Select two reference layers, layer a and layer b Denote it as state s ^(t) . Will x _k'=c is input into the two-dimensional fine segmentation model M _F to obtain the value function R (st ^(t) , act ^(t) ) of the current action. Use the greedy algorithm to exhaustively find the current maximum reward value Q(st ^(t) , act ^(t) ), then find the loss function L _t for negative feedback, and update the weight of the reinforcement learning network Q.

(5) Fix the reinforcement learning network and update the weights of the two-dimensional fine segmentation model M _F model.

After the reinforcement learning network is trained, the parameters of the reinforcement learning network are fixed. Use reinforcement learning network to filter reference layer a and layer b Input the reference layer and segmentation layer x _k'=c into the model M _F to complete the two-dimensional fine segmentation model training.

(6) Automatic segmentation of pancreatic tumors.

(6.1) Resample and adjust the grayscale value of the test image in the given test set, and truncate the HU value of the image between [-100, 240], and then normalize it to [0, 1]. The processed test image is input into the three-dimensional rough segmentation model M _c to obtain the segmentation probability map of the pancreas and tumor. and based on Get ROI area

(6.2) Change the ROI area Input to the reinforcement learning network Q to obtain the reference volume reference layer.

(6.3) Divide the test image into 2D images layer by layer along the volume data layer, select a segmentation layer, input the segmentation layer and reference layer to _MF for segmentation, and obtain the tumor segmentation result.

On the other hand, as shown in Figure 6, the present invention also provides a pancreatic tumor image segmentation system based on reinforcement learning and attention. The system includes a pancreatic tumor segmentation training set building module, a three-dimensional coarse segmentation model module, and a reinforcement learning network. module and two-dimensional fine segmentation model module;

The two-dimensional fine segmentation model module is used to divide the data and labels of the training set into 2D images along the z-axis, and select a segmentation layer. The two-dimensional fine segmentation model module includes two cross-attention feature fusion sub-modules, Corresponding to two reference layers respectively, the two cross-attention feature fusion sub-modules interact with the segmentation layer respectively to unify the feature dimensions of the reference layer and the segmentation layer, and then perform a splicing operation to perform the first fusion. The first fusion result is subjected to a dot product operation with the segmentation layer features after the feature dimensions are unified to generate an information correlation matrix of the cross attention mechanism, and then the dot product operation is performed with the segmentation layer features after the feature dimensions are unified for the second fusion, and used The residual operation fuses the second fusion result with the information of the original segmentation layer features to obtain the segmentation result of pancreatic tumors.

The following is a specific embodiment of the present invention

This example uses the CT image data of the public dataset Medical Segmentation Decathlon (MSD) pancreatic tumor segmentation dataset for research. There are 281 cases of pancreatic tumor data in the MSD data set.

This invention divides the data into a training set of 224 cases and a test set of 57 cases. The data of the training set is used to train the three-dimensional coarse segmentation model M _c , the reinforcement learning network Q and the two-dimensional fine segmentation model M _F , and the test set is used to test the performance of the model. This invention uses DSC coefficient, Jaccard coefficient, accuracy Precision and recall rate Recall to evaluate 2D UNet and 3D UNet networks.

In addition, in order to verify the effectiveness of the cross-attention feature fusion module, the present invention adds a simulation process that removes the reinforcement learning network, randomly selects a reference layer from the ROI, and compares it with the present invention. The results are shown in Table 1.

Table 1. Comparison results between segmentation methods based on reinforcement learning and cross-attention and other methods in pancreatic tumor segmentation

It can be seen from the results that the pancreatic tumor image segmentation method based on reinforcement learning and attention achieved the best results compared with other method strategies. Compared with the 2D UNet network and the 3DUNet network, the introduction of the reference layer and cross attention can enhance the recognition and positioning of segmentation targets by the 2D network, while avoiding the 3D network from introducing too much redundant information, causing segmentation difficulties. In addition, the reinforcement learning method can better reduce the spread and accumulation of false false labels during the model training process (the accuracy rate increases by 8.67%). Compared with other methods, the present invention achieves the best results in pancreatic tumor segmentation.

The above embodiments are used to illustrate the present invention, rather than to limit the present invention. Within the spirit of the present invention and the protection scope of the claims, any modifications and changes made to the present invention fall within the protection scope of the present invention.

Claims

A pancreatic tumor image segmentation method based on reinforcement learning and attention, characterized in that the method includes the following steps:

(1) Collect pancreatic CT images of patients with pancreatic cancer and perform preprocessing, outline pancreatic tumor segmentation labels on CT images, and construct a pancreatic tumor segmentation training set;

(2) Construct a three-dimensional rough segmentation model for pancreatic CT rough segmentation, obtain the pancreatic ROI region of interest, and segment the image and label of the ROI region into 2D images along the z-axis;

(3) Construct a two-dimensional precise segmentation model with a cross-attention feature fusion module, and use the cross-attention feature fusion module between layers to enable segmentation features to interact with information in the segmentation layer and the reference layer;

(3.1) Split the data and labels of the training set into 2D images along the z-axis in the same way as the image of the ROI area in step (2), and randomly select the two 2D images segmented in step (2) as Reference layer, the 2D image after segmentation of the training set data is used as the segmentation layer; the reinforcement learning network is used to select other reference layers of pancreatic tumors; the environment of the reinforcement learning network is the ROI area obtained from the original CT image, and the state is randomly selected along the z-axis The two-layer slice, the action is that each iteration agent moves forward and backward along the z-axis in the last selected reference layer, and each reference layer corresponds to an agent;

(3.2) Each reference layer corresponds to a cross-attention feature fusion module, which interacts with the segmentation layer. The cross-attention feature fusion module unifies the feature dimensions of the reference layer and the segmentation layer, and then performs a splicing operation to perform the first step. For the second fusion, the first fusion result is subjected to a dot product operation with the segmentation layer features after the feature dimensions are unified to generate an information correlation matrix of the cross attention mechanism, and then the dot product operation is performed with the segmentation layer features after the feature dimensions are unified for the second time. Second fusion, and use the residual operation to fuse the second fusion result with the information of the original segmentation layer features as the segmentation result;

(4) Given a pancreatic tumor image to be segmented and preprocessed, input it into the three-dimensional rough segmentation model to obtain the ROI area and segment it, use the reinforcement learning network to select the reference layer, and segment the pancreatic tumor image to be segmented. Finally, a segmentation layer is selected, and the segmentation layer and reference layer are input into the two-dimensional fine segmentation model for segmentation, and the segmentation results of pancreatic tumors are obtained.
A pancreatic tumor image segmentation method based on reinforcement learning and attention according to claim 1, characterized in that in step (1), the preprocessing process is specifically: adjusting the voxel space distance of all data in the training set to 1mm; truncate the HU value of the image to between -100 and 240, and then normalize it to between 0 and 1.
A pancreatic tumor image segmentation method based on reinforcement learning and attention according to claim 1, characterized in that in step (2), the three-dimensional rough segmentation model consists of two parts: encoding and decoding, and the encoding part includes four encoding block, each coding block is connected to a downsampling layer; the decoding part includes four decoding blocks, and each decoding block is connected to an upsampling layer; each coding block and decoding block are composed of different numbers of convolution-activation layer composition.
A pancreatic tumor image segmentation method based on reinforcement learning and attention according to claim 1, characterized by That is, in step (2), the ROI area image is recorded as Corresponds to the pancreatic CT image of the nth pancreatic cancer patient in the training set; Divide it into a 2D image along the z-axis, so Represents the 2D image of the kth layer after segmentation. The label of the CT image after truncation is recorded as The label corresponding to the pancreatic CT image of the nth pancreatic cancer patient in the training set is also divided into 2D images along the z-axis, so Represents the 2D image label corresponding to the k-th layer, where K ROI is the minimum layer number after truncation, and K ROI ′ is the maximum layer number after truncation.
A pancreatic tumor image segmentation method based on reinforcement learning and attention according to claim 1, characterized in that, in step (2), the loss function used in the three-dimensional rough segmentation model is the cross-entropy loss function Loss CE :

in, Represents the predicted rough segmentation result output by the network, Y n is the pancreatic tumor segmentation label of the CT image, m is the number of pixels in the input image, y j and z j are the real label and predicted label of pixel j respectively, τ = 0 , 1 and 2 respectively represent the background, pancreas or pancreatic tumor; function I(·) is an indicative function, function log is a logarithmic function, and p(·) is the probability function predicted by the model.
A pancreatic tumor image segmentation method based on reinforcement learning and attention according to claim 1, characterized in that in step (3.1), the action value function of the reinforcement learning network is the prediction result of the two-dimensional fine segmentation model and the real label The loss function is used, and the maximum reward value of the next action in the current state is calculated through the heuristic function; in the iterative process, the reinforcement learning network is trained using the negative feedback method.
A pancreatic tumor image segmentation method based on reinforcement learning and attention according to claim 6, characterized in that after the reinforcement learning network is trained, the parameters of the reinforcement learning network are fixed, and the reinforcement learning network is used to filter the reference layer and combine the reference layer with The segmentation layer is input to the two-dimensional fine segmentation model to complete the two-dimensional fine segmentation model training.
A pancreatic tumor image segmentation method based on reinforcement learning and attention according to claim 4, characterized in that in step (3.2), the two reference layers are respectively recorded as and The segmentation layer is recorded as x k′=c . The 2D image of the k′th layer after x k′ is segmented. For the reference layer The interaction process with the segmentation layer x k′=c and the reference layer It is consistent with the interaction process of the segmentation layer x k′=c ; for the reference layer For the segmentation layer x k′=c , the implementation of the cross-attention feature fusion module is as follows:

Place the reference layer and segmentation layer x k′=c. After downsampling and multiple convolution operations, high-dimensional features F k=a and F k′=c are obtained respectively; F k=a and F k′=c are used as cross-attention feature fusion. module input;

The cross-attention feature fusion module first uses two linear mapping functions g(·) and f(·) to convert the input features from three-dimensional to one-dimensional, and transform the one-dimensional features to keep the dimensions of the related features consistent; Pair features through g(·) and f(·) F k=a and F k′=c perform mapping operations to make the dimensions of the features unified:
F k′＝a ′＝g(F k＝a )
F k′=c ′=f (F k′=c )

Parallel F k = a ′ and F k ′ = c ′, and then use a convolution kernel W 1 for mapping operation, and fuse the two for the first time. The fused features are used as reference features:
F k＝a||k′＝c =W 1 Cat(F k＝a ′||F k′＝c ′)

Cat(·) is the splicing operation along the channel direction;

Use F k=a||k′=c and F k′=c ′ to perform dot product operations to generate the information correlation matrix A of the cross-attention mechanism:
q＝W q (F k＝a||k′＝c ), k＝W k (F k′＝c ′), v＝W v (F k′＝c ′)

Among them, W q , W k , and W v are the weights of three convolutions used to adapt each feature; sig(·) is the sigmoid function; D is the number of channels of the feature F k′=c ′;

The information correlation matrix A and v perform a dot product operation to complete the second fusion, and the residual operation is used to fuse the information of F k′=c to v′:
v′=Av+f′(F k′=c )

Among them, f′(·) is a linear mapping function.
A pancreatic tumor image segmentation method based on reinforcement learning and attention according to claim 5, characterized in that, in step (3), the two-dimensional fine segmentation model takes the segmentation layer and the reference layer as input, and the prediction of the segmentation layer The result is used as the output, and the loss function Dice Loss is used for negative feedback learning:

Among them, m′ is the number of pixels in the input 2D image, y k′ represents the label of the 2D image of the k′ layer after segmentation, y k′=c is the prediction result of the segmentation layer, y h and z h are respectively are the true labels and predicted labels of pixel h.
A pancreatic tumor image segmentation system based on reinforcement learning and attention, characterized in that the system includes a pancreatic tumor segmentation training set building module, a three-dimensional coarse segmentation model module, a reinforcement learning network module and a two-dimensional fine segmentation model module;

The pancreatic tumor segmentation training set building module is used to collect and preprocess pancreatic CT images of patients with pancreatic cancer, outline pancreatic tumor segmentation labels on CT images, and construct a pancreatic tumor segmentation training set;

The three-dimensional coarse segmentation model module is used to obtain the pancreas ROI region of interest, and segment the image of the ROI region and its label along the axis z into 2D images;

The reinforcement learning network module is used to select two reference layers from the 2D image segmented by the three-dimensional coarse segmentation model module;

The two-dimensional fine segmentation model module is used to divide the data and labels of the training set into 2D images along the z-axis, and select a segmentation layer. The two-dimensional fine segmentation model module includes two cross-attention feature fusion sub-modules, Corresponding to two reference layers respectively, the two cross-attention feature fusion sub-modules interact with the segmentation layer respectively to unify the feature dimensions of the reference layer and the segmentation layer, and then perform a splicing operation to perform the first fusion. The first fusion result is subjected to a dot product operation with the segmentation layer features after the feature dimensions are unified to generate an information correlation matrix of the cross attention mechanism, and then the dot product operation is performed with the segmentation layer features after the feature dimensions are unified for the second fusion, and used The residual operation fuses the second fusion result with the information of the original segmentation layer features to obtain the segmentation result of pancreatic tumors.