CN113096766B - Three-dimensional dose prediction method and system in personalized accurate radiotherapy plan - Google Patents

Abstract

The invention discloses a three-dimensional dose prediction method and a three-dimensional dose prediction system in an individualized accurate radiotherapy plan, wherein the method comprises the following steps: step 1, acquiring radiotherapy related information such as an electronic computed tomography image, a dangerous organ structure mask image, a three-dimensional dose distribution image and the like; step 2, carrying out data preprocessing operation on the image in the step 1; step 3, inputting the acquired image data into a two-stage generator network to generate a three-dimensional dose distribution image and a confidence map; step 4, adopting a Markov discriminator to resist the three-dimensional dose distribution predicted image and the three-dimensional dose distribution real image; step 5, jointly optimizing a prediction model through a reconstruction loss function, a reconstruction loss function with confidence coefficient weight and an antagonistic loss function; and 6, generating three-dimensional dose distribution by using the trained prediction model. By the technical scheme, manual intervention in radiotherapy planning can be reduced, the accuracy of dose prediction is improved, and personalized accurate radiotherapy is realized.

Description

Three-dimensional dose prediction method and system in personalized accurate radiotherapy plan

Technical Field

The invention relates to the field of crossing artificial intelligence and medical image processing, in particular to a three-dimensional dose prediction method and a three-dimensional dose prediction system in an individualized accurate radiotherapy plan.

Background

Conventional radiation treatment planning is accomplished by repeated and coordinated discussions between clinicians and physicists with relevant expertise to ensure that the patient receives the correct treatment plan and high quality radiation dose, but this means that several days may be required to complete a treatment plan, which in turn reduces tumor control and patient survival opportunities. In order to solve the problem, an automatic radiotherapy planning method and an automatic radiotherapy planning system provide a possibility for computer-aided personalized precise radiotherapy, so that the planning time of a radiotherapy plan is shortened, in recent years, the development of an automatic radiotherapy method is promoted due to the rapid development of artificial intelligence, and the direct prediction of three-dimensional full-dose distribution by using a high-dimensional feature generated by a neural network has important application value.

The basic goal of radiotherapy is to maximize the treatment gain ratio, plan design is a key step for realizing the goal, the current clinical radiotherapy plan design process usually adopts a manual formulation method, quality audit mostly sees the plan dosimetry performance, the performance standard takes the relevant clinical specifications obtained based on patient population statistics as reference, and the consideration of individual structure difference of patients is lacked, so that the homogenization degree of the clinical radiotherapy plan accepted by the patients is not high, and the realization of the basic radiotherapy goal is restricted. Clinical experience and research show that high-quality radiotherapy schemes available for different patients have strong correlation with the geometrical anatomical structure characteristics of the patients, and the correlation is learned and modeled, so that the corresponding high-quality radiotherapy dose can be accurately predicted by a new patient before radiotherapy plan design is carried out on the new patient.

Therefore, a model considering the association of the geometric anatomical structure of the patient and the three-dimensional dose distribution of the organ needs to be designed by means of a neural network, and the influence factors such as the relationship between the volume of the organ and the spatial position of the organ are considered, so that the three-dimensional dose distribution with higher refinement and accuracy is generated, the dose information and the spatial position relationship of all voxels are effectively reserved, and more sufficient data reference information is provided for the optimization and quality control of the subsequent treatment plan.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provides a three-dimensional dose prediction method and a three-dimensional dose prediction system in an individualized and accurate radiotherapy plan, which can improve the efficiency and accuracy of radiotherapy dose prediction. The method comprises the step of conducting rough prediction on radiotherapy three-dimensional dose distribution through a two-stage generator network formed by combining a rough network and a fine network so as to generate fine prediction. Meanwhile, the Markov discriminator is used for resisting the three-dimensional dose distribution predicted image and the three-dimensional dose distribution real image, the prediction quality of the three-dimensional dose distribution image is improved, and the strong performance and theoretical advantage are shown on the aspect of solving the problem of clinical radiotherapy dose prediction.

In order to achieve the purpose, the invention adopts the following technical scheme:

in a first aspect, an embodiment of the present invention provides a method for predicting a three-dimensional dose in an individualized precise radiotherapy plan, which is characterized by including the following steps:

step 1, acquiring radiotherapy related information such as an electronic computed tomography image, a dangerous organ structure mask image, a planned target area image, a three-dimensional dose distribution image and the like;

step 2, carrying out data preprocessing operation on the image in the step 1;

step 3, inputting the acquired image data into a two-stage generator network to generate a three-dimensional dose distribution image and a confidence map;

step 4, a Markov discriminator is adopted to resist the three-dimensional dose distribution predicted image and the three-dimensional dose distribution real image, and the prediction quality of the three-dimensional dose distribution image is improved;

step 5, jointly optimizing a prediction model through a reconstruction loss function, a reconstruction loss function with confidence coefficient weight and an antagonistic loss function;

and 6, generating three-dimensional dose distribution by using the trained prediction model, namely predicting new data by using the trained two-stage generator network to generate the three-dimensional dose distribution.

As a possible implementation manner of this embodiment, the step 1 includes the following steps:

step 11, using an electronic computed tomography imaging device to scan and image the patient to obtain an electronic computed tomography image, wherein the electronic computed tomography image is defined as C e R^H×W×D. Where R represents the entire image area, H, W are the length and width of the image, respectively, and D is the depth of the image.

Step 12, before the radiotherapy plan, delineating N risk organs according to the computed tomography image of the patient, and creating a structural mask image for each risk organ to effectively protect the normal organizer when the radiotherapy plan is implementedAnd reducing the irradiated dose and volume. The mask image of the danger organ structure is defined as O_n∈R^H×W×D，n∈[1,N]Wherein R represents the whole image area, H and W are the length and width of the image respectively, D is the depth of the image, N is a positive integer and N is more than or equal to 1.

Step 13, after the positioning scanning image data of the patient is preliminarily processed, a clinician and a physicist discuss and draw a planning radiotherapy target area, and a mask image P belonging to R of the planning target area is created^H×W×DWhere R represents the entire image area, H, W are the length and width of the image, respectively, and D is the depth of the image.

Step 14, after the contour of the target area and the dangerous organ of the radiotherapy is outlined, the physicist determines the optimal radiotherapy plan according to the requirements of the clinician, so as to ensure that the irradiation dose of the important organ tissues does not exceed the tolerance dose of the important organ tissues while the tumor obtains enough radiotherapy dose, and a three-dimensional dose distribution image Y epsilon R is created^H×W×DWhere R represents the entire image area, H, W are the length and width of the image, respectively, and D is the depth of the image.

As a possible implementation manner of this embodiment, the step 2 includes the following steps:

step 21, carrying out electron computer tomography image C e R^H×W×DAnd (3) carrying out normalization treatment:

wherein C (H, W, D) represents a pixel value of the electron computed tomography image at coordinates (H, W, D), H ∈ [0, H ], W ∈ [0, W), and D ∈ [0, D). C_max，C_minRespectively representing the maximum and minimum values among all pixel values within the electron computed tomography image.

Step 22, mask images O of the N danger organ structures delineated in step 12_n∈R^H×W×D，n∈[1,N]Performing one-hot encoding, and combining the encoded N danger organ images into O along the channel dimension_one-hot∈R^N×H×W×D。

Step 23, mask image P E R of planning target area^H×W×DAnd (3) carrying out normalization treatment:

P_ptv70(h,w,d)、P_ptv63(h,w,d)、P_ptv56(h, w, d) respectively indicate pixel values at coordinates (h, w, d) in images representing planned target doses of 70Gy, 63Gy, 56Gy, and P (h, w, d) indicates pixel values at coordinates (h, w, d) after normalization and summation of the planned target images. Wherein H is from 0, H), W is from 0, W), D is from 0, D).

Step 24, the preprocessed electron computer tomography image C e R^H×W×DMask image of dangerous organ Structure O_one-hot∈R^N×H×W×DPlanning target area image P ∈ R^H×W×DCombined into X along the channel dimension₁∈R^{(N+2)×H×W×D}And used as input data for the model.

As a possible implementation manner of this embodiment, the step 3 includes the following steps:

for 3D-UNet, there is one coding path with three downsamplings and a decoding path with three upsamplings, step 31. The encoding path contains two convolution and one max-pooling operation per layer, and the decoding path contains one transposed convolution and two convolution operations per layer. The features with the same resolution in the encoding path and the decoding path are combined through jump connection, and the last layer is a convolution layer with a convolution kernel of 1 × 1 × 1, which is used for reducing output channels and generating a final output result.

And step 32, constructing a rough network model by taking the 3D-UNet as a basic model. In order to improve the calculation efficiency and the dose distribution prediction result, the residual learning idea is used for reference, and the internal structure of the model has the capability of input-output identity mapping so as to ensure that the degradation caused by continuous stacking can be avoided in the network stacking process. Assuming that the input of the residual block is defined as x, the output y can be expressed as y ═ x + F (x, { W)_i}) of the formula (I), wherein F (x, { W)_iIs a learning objective, i.e. directlyUsing identity mapping as part of the network, the problem is translated into learning a residual function F (x, { W }_iY-x, provided that F (x, { W)_i0, the identity map y x is constructed, so the residual part can be expressed as F W₃σ(W₂σ(W₁x)), where σ denotes the ReLU activation function, and W₁,W₂,W₃Refers to the weight of a three-layer network. Based on the idea, two convolution operations of the 3D-UNet are replaced by residual modules, one residual module is composed of 1 multiplied by 1, 3 multiplied by 3 and 1 multiplied by 1 and 3 convolution layers in sequence, and the input and the output of the residual modules are added through residual connection to obtain the final output result of the module. In addition, in order to effectively utilize the low-level features of the coding network, the scSE attention module is embedded into the last convolution layer of the residual module, and the attention module is composed of two modules, namely a channel attention cSE module and a space attention sSE module. cSE are compressed and excited in the channel, and its input characteristic diagram U ═ U₁,u₂,...,u_c]Wherein each channel u_c∈R^H×W×DAnd obtaining a vector z after U passes through the global pooling layer, wherein the z belongs to R^1×1×1×cEach position k has a value of

Then, the two full-connection layers are passed through, and normalized to [0,1 ] by using Sigmoid activation function](ii) a sSE are compressed and excited in space, and its input characteristic diagram U ═ U^1,1,1,u^1,1,2,...,u^{h ,w,d},...,u^H,W,D]Spatial extrusion is achieved by convolution of 1 × 1 × 1, and then normalized to [0,1 ] by Sigmoid]Finally, cSE and sSE blocks are directly added, i.e., scSE cSE + sSE. Four maximum pooling downsampling operations on the coding path of the model reduce the signature size, four transpose convolutions on the decoding path restore the signature size, and combine features with the same resolution in the coding path and the decoding path using skip connections, using 1 × 1 × 1 convolution at the last layer of the network to reduce the signature size output by the decoding path at the end.

Step 33, counting the number of images preprocessed in step 2Data set X₁∈R^{(N+2)×H×W×D}As input to the coarse network, its output is a coarse dose distribution Y_coarse∈R^H×W×D。

And step 34, constructing a fine network model by taking the 3D-UNet as a basic model. The number of feature maps of all the convolution layers of the fine network model except the last layer is 2 times of that of the rough network, and the structure of the fine network model is completely the same as that of the rough network.

Step 35, the fine network processes the image data set X preprocessed in step 2₁∈R^{(N+2)×H×W×D}And a feature map F of the coarse network output_last∈R^Ch×H×W×DCombined into X along the channel dimension₂∈R^{(N+2+Ch)×H×W×D}As input to the fine network. Wherein Ch represents F_lastThe number of channels. Mixing X₂As input to the fine network, the output is a fine dose distribution Y_fine∈R^{H ×W×D}And confidence map M ∈ R^H×W×D。

As a possible implementation manner of this embodiment, the step 4 includes the following steps:

step 41, the discriminator for generating the countermeasure network is a Markov discriminator consisting of 5 convolutional layers;

step 42, the discriminators are respectively represented by Y_fine∈R^H×W×DAnd X₁∈R^{(N+2)×H×W×D}Combined into X along the channel dimension₃∈R^{(N +3)×H×W×D}And by Y ∈ R^H×W×DAnd X₁∈R^{(N+2)×H×W×D}Combined into X along the channel dimension₄∈R^{(N+3)×H×W×D}As its input, the output is a matrix of NxN, N being a positive integer and N ≧ 1, each element of which represents a degree of confidence that the input image is a true image.

As a possible implementation manner of this embodiment, the step 5 includes the following steps:

step 51, the loss function of the generator is represented as:

L_AttG＝0.5×||Y-G(X₁)||₁+M^β⊙||Y-G(X₂)||₁，

L_G＝L_AttG+λ_LADV，

wherein G (-) is a generator, D (-) is an discriminator, M is a confidence map, β is an attention super parameter, and l is a two-matrix element-by-element multiplication operation. L is_GThe total loss function of the generator is represented by a loss function L combining the reconstruction loss and the reconstruction loss with confidence weight_AttGAnd a penalty function L_ADVIn the composition, λ is an equilibrium coefficient. For L_ADV，p_data(X₂) Representing the probability distribution to which the input data of the fine network obeys,

is a mathematical expectation.

Step 52, the loss function of the discriminator is expressed as:

wherein, G (-) is a generator, and D (-) is a discriminator. L is_DRepresenting the total loss function p of the discriminator_data(X₃) Predicting a dose distribution obeyed probability distribution, P, for a fine network_data(X₄) The probability distribution to which the true dose distribution obeys,

is a mathematical expectation.

And step 53, performing optimization training on the generator and discriminator network by using an Adam optimization algorithm:

first estimate m by computing the first moment of the gradient_tAnd second moment estimate v_t：

m_t＝μ₁*m_t-1+(1-μ₁)*g_t，

Wherein u is₁And u₂Representing two different preset parameters, g_tAnd

representing first and second order gradients, respectively, and then calculating corrections for first and second order moment estimates, respectively

Finally, the correction value is calculated according to the calculated correction value

And

calculating to obtain an updated value delta theta by combining the learning rate eta and the minimum deviation epsilon_t：

Using the update value delta theta_tAnd optimizing and learning the neural network parameters.

In a second aspect, an embodiment of the present invention provides a three-dimensional dose prediction system in an individualized precise radiotherapy plan, which is characterized by including:

the image acquisition module is used for acquiring radiotherapy related information such as an electronic computed tomography image, a dangerous organ structure mask image, a planning target area image, a three-dimensional dose distribution image and the like;

the data preprocessing module is used for preprocessing the four types of image data;

the model building module is used for building a two-stage generation network and a Markov discriminator network, and specifically comprises a prediction model generation network module and a Markov discrimination network module, wherein the prediction model generation network module consists of a rough network model and a fine network model, and the Markov discrimination network module consists of a Markov discriminator network model;

the model training module is used for inputting the acquired image data into a two-stage generator network to generate a three-dimensional dose distribution image and a confidence map, and a Markov discriminator is adopted to resist against a three-dimensional dose distribution predicted image and a three-dimensional dose distribution real image so as to improve the prediction quality of the three-dimensional dose distribution image;

and the radiotherapy dose prediction module predicts new data by using the trained two-stage generator network to generate three-dimensional dose distribution.

As a possible implementation manner of this embodiment, the image data obtaining module includes:

and the electronic computer tomography image acquisition module is used for carrying out computer tomography imaging on the patient by using the electronic computer tomography imaging equipment to obtain an electronic computer tomography image.

And the danger organ mask image acquisition module is used for delineating danger organs according to the electronic computed tomography images of the patient and creating a structural mask image for each danger organ so as to effectively protect normal tissues and organs and reduce the irradiated dose and volume of the tissues and organs when a radiotherapy plan is implemented.

The planning target area mask image acquisition module is used for preliminarily processing the data of the electronic computer tomography image of the patient, drawing a planning radiotherapy target area through discussion of a clinician and a physicist, and creating a planning target area mask image.

And the three-dimensional dose distribution image acquisition module is used for determining an optimal radiotherapy plan according to the requirements of a clinician after the contour of the radiotherapy target area and the dangerous organ is sketched, so that the irradiation dose of important organ tissues is controlled as far as possible not to exceed the tolerance dose while the tumor obtains enough radiotherapy dose, and a three-dimensional dose distribution image is created.

As a possible implementation manner of this embodiment, the image data preprocessing module includes:

and the electronic computer tomography image preprocessing module is used for carrying out normalization processing on the electronic computer tomography image.

And the danger organ mask image preprocessing module is used for carrying out independent thermal coding on the delineated danger organ structure mask image and combining the coded danger organ image along the channel dimension.

And the planning target area image preprocessing module is used for carrying out normalization processing on the planning target area mask image.

And the image data set construction module combines the preprocessed electronic computed tomography image, the dangerous organ structure mask image and the planning target area image along the channel dimension to be used as the input of the model.

As a possible implementation manner of this embodiment, the model training module includes:

and the rough network training module takes the preprocessed image data set as the input of a rough network, and the output of the rough network training module is rough dose distribution.

And the fine network training module is used for combining the preprocessed image data set with the feature map output by the coarse network along the channel dimension by the fine network to serve as the input of the fine network, and the output is a fine dose distribution and confidence map.

The Markov identification network training module comprises two parts of identifier input: 1) combining the preprocessed image dataset with the fine dose distribution output by the fine network along the channel dimension as input; 2) the preprocessed image dataset is combined with the true dose distribution along the channel dimension as input. The output of the markov decision network is a matrix, each element of which represents the degree of confidence that the input image is a true image.

And the model optimization module is used for performing optimization training on the generator and the discriminator by using an Adam optimization algorithm.

The beneficial effects of the invention are:

the invention obtains the radiotherapy related information such as an electronic computed tomography image, a dangerous organ structure mask image, a plan target area image, a three-dimensional dose distribution image and the like; performing data preprocessing operation on the image; inputting the acquired image data into a two-stage generator network to generate a three-dimensional dose distribution image and a confidence map; the Markov discriminator is adopted to resist the three-dimensional dose distribution predicted image and the three-dimensional dose distribution real image, so that the prediction quality of the three-dimensional dose distribution image is improved; jointly optimizing a prediction model through a reconstruction loss function, a reconstruction loss function with confidence coefficient weight and a countervailing loss function; and generating three-dimensional dose distribution by using the trained prediction model. By the technical scheme, manual intervention in radiotherapy planning can be reduced, the accuracy of dose prediction is improved, and personalized accurate radiotherapy is realized.

Drawings

Fig. 1 is a flow diagram illustrating a method for three-dimensional dose prediction in personalized precision radiotherapy planning, according to an exemplary embodiment;

figure 2 is a block diagram illustrating a three-dimensional dose prediction system in personalized precision radiotherapy planning, according to an exemplary embodiment;

FIG. 3 is a flow chart of a method of three-dimensional dose prediction using the present invention;

FIG. 4a is an electron computed tomography image shown in accordance with an exemplary embodiment;

FIG. 4b is an illustration of a critical organ structure mask image according to an exemplary embodiment;

FIG. 4c is a schematic diagram of a planning target volume image shown in accordance with an exemplary embodiment;

FIGS. 5a, 5b, 5c, 5d, 5e are schematic diagrams illustrating a three-dimensional dose prediction result according to an exemplary embodiment;

fig. 6a, 6b, 6c, 6d and 6e are schematic diagrams of the true three-dimensional dose prediction results given by comparing fig. 5a, 5b, 5c, 5d and 5 e.

Detailed Description

The invention is further illustrated with reference to the following figures and examples.

The structures, proportions, sizes, and other dimensions shown in the drawings and described in the specification are for understanding and reading the present disclosure, and are not intended to limit the scope of the present disclosure, which is defined in the claims, and are not essential to the art, and any structural modifications, changes in proportions, or adjustments in size, which do not affect the efficacy or achievement of the intended purposes of the present disclosure, are intended to be included within the scope of the present disclosure. In addition, the terms "upper", "lower", "left", "right", "middle" and "one" used in the present specification are for clarity of description, and are not intended to limit the scope of the present invention, and the relative relationship between the terms and the terms is not to be construed as a scope of the present invention.

Fig. 1 is a flow chart illustrating a method of three-dimensional dose prediction in personalized accurate radiotherapy planning, according to an exemplary embodiment. As shown in fig. 1, a three-dimensional dose prediction method in an individualized precise radiotherapy plan provided by an embodiment of the present invention includes the following steps:

step 1, acquiring radiotherapy related information such as an electronic computed tomography image, a dangerous organ structure mask image, a planned target area image, a three-dimensional dose distribution image and the like;

step 11, using an electronic computed tomography imaging device to scan and image the patient to obtain an electronic computed tomography image, wherein the electronic computed tomography image is defined as C e R^H×W×D. Where R represents the entire image area, H, W are the length and width of the image, respectively, and D is the depth of the image.

Step (ii) ofBefore radiotherapy planning, N danger organs are sketched according to the electron computer tomography scanning image of a patient, and a structural mask image is created for each danger organ, so that a normal tissue organ is effectively protected when the radiotherapy planning is carried out, and the irradiated dose and the irradiated volume of the tissue organ are reduced. The mask image of the dangerous organ structure is defined as O_n∈R^H×W×D，n∈[1,N]Wherein R represents the whole image area, H and W are the length and width of the image respectively, D is the depth of the image, N is a positive integer and N is more than or equal to 1.

Step 13, after the positioning scanning image data of the patient is preliminarily processed, a planning radiotherapy target area is drawn by the discussion of a clinician and a physicist, and a mask image P epsilon R of the planning target area is created^H×W×DWhere R represents the entire image area, H, W are the length and width of the image, respectively, and D is the depth of the image.

Step 14, after the contour of the target area and the dangerous organ of the radiotherapy is outlined, the physicist determines the optimal radiotherapy plan according to the requirements of the clinician, so as to ensure that the irradiation dose of the important organ tissues does not exceed the tolerance dose of the important organ tissues while the tumor obtains enough radiotherapy dose, and a three-dimensional dose distribution image Y epsilon R is created^H×W×DWhere R represents the entire image area, H, W are the length and width of the image, respectively, and D is the depth of the image.

Step 2, carrying out data preprocessing operation on the image in the step 1;

step 21, carrying out electron computer tomography image C e R^H×W×DAnd (3) carrying out normalization treatment:

wherein C (H, W, D) represents a pixel value of the electron computed tomography image at coordinates (H, W, D), H ∈ [0, H ], W ∈ [0, W), and D ∈ [0, D). C_max，C_minRespectively representing the maximum and minimum values among all pixel values within the electron computed tomography image.

Step 22, mask images O of the N danger organ structures delineated in step 12_n∈R^H×W×D，n∈[1,N]Performing one-hot encoding, and combining the encoded N danger organ images into O along the channel dimension_one-hot∈R^N×H×W×D。

Step 23, mask image P E R of planning target area^H×W×DAnd (3) carrying out normalization treatment:

P_ptv70(h,w,d)、P_ptv63(h,w,d)、P_ptv56(h, w, d) respectively indicate pixel values at coordinates (h, w, d) in images representing planned target doses of 70Gy, 63Gy, 56Gy, and P (h, w, d) indicates pixel values at coordinates (h, w, d) after normalization and summation of the planned target images. Wherein H is equal to 0, H), W is equal to 0, W), and D is equal to 0, D).

Step 24, making the preprocessed electron computer tomography image C E R^H×W×DMask image of dangerous organ Structure O_one-hot∈R^N×H×W×DPlanning target region image P ∈ R^H×W×DCombined into X along the channel dimension₁∈R^{(N+2)×H×W×D}And used as input data for the model.

Step 3, inputting the acquired image data into a two-stage generator network to generate a three-dimensional dose distribution image and a confidence map;

for 3D-UNet, there is one coding path with three downsamplings and a decoding path with three upsamplings, step 31. The encoding path contains two convolution and one max-pooling operation per layer, and the decoding path contains one transposed convolution and two convolution operations per layer. The features with the same resolution in the encoding path and the decoding path are combined through jump connection, and the last layer is a convolution layer with a convolution kernel of 1 × 1 × 1, which is used for reducing output channels and generating a final output result.

And step 32, constructing a rough network model by taking the 3D-UNet as a basic model. In order to improve the calculation efficiency and the dose distribution prediction result, the residual learning thought is used for reference, and the internal structure of the model has the capacity of input-to-output identity mappingTo ensure that no degradation occurs by continuing stacking during the stacking of the network. Assuming that the input of the residual block is defined as x, the output y can be expressed as y ═ x + F (x, { W)_i}) of the formula (I), wherein F (x, { W)_i}) is a learning objective, i.e. directly taking identity mapping as a part of the network, translating the problem into learning a residual function F (x, { W)_iY-x, provided that F (x, { W)_i0, the identity map y x is formed, so the residual part can be represented as F W₃σ(W₂σ(W₁x)), where σ denotes the ReLU activation function, and W₁,W₂,W₃Refers to the weight of a three-layer network. Based on the idea, two convolution operations of the 3D-UNet are replaced by residual modules, one residual module is composed of 1 multiplied by 1, 3 multiplied by 3 and 1 multiplied by 1 and 3 convolution layers in sequence, and the input and the output of the residual modules are added through residual connection to obtain the final output result of the module. In addition, in order to effectively utilize the low-level features of the coding network, the scSE attention module is embedded into the last convolution layer of the residual module, and the attention module is composed of two modules, namely a channel attention cSE module and a space attention sSE module. cSE are compressed and excited in the channel, and its input characteristic diagram U ═ U₁,u₂,...,u_c]Wherein each channel u_c∈R^H×W×DObtaining a vector z after U passes through a global pooling layer, wherein z belongs to R^1×1×1×cEach position k has a value of

Then, the two full-connection layers are passed through, and normalized to [0,1 ] by using Sigmoid activation function](ii) a sSE are compressed and excited in space, and its input characteristic diagram U ═ U^1,1,1,u^1,1,2,...,u^{h ,w,d},...,u^H,W,D]Spatial extrusion is achieved by convolution of 1 × 1 × 1, and then normalized to [0,1 ] by Sigmoid]Finally, cSE and sSE blocks are directly added, i.e., scSE cSE + sSE. Four maximum pooled downsampling operations on the encoding path of the model reduce the feature map size, four transpose convolutions on the decoding path restore the feature map size, and a skip connection is used to place the encoding path and the decoding path inFeatures with the same resolution are combined using a convolution of 1 x 1 at the last layer of the network to reduce the size of the feature map output at the end of the decoding path.

Step 33, the image data set X preprocessed in step 2 is processed₁∈R^{(N+2)×H×W×D}As input to the coarse network, its output is a coarse dose distribution Y_coarse∈R^H×W×D。

And step 34, constructing a fine network model by taking the 3D-UNet as a basic model. The number of feature maps of all the convolution layers of the fine network model except the last layer is 2 times of that of the rough network, and the structure of the fine network model is completely the same as that of the rough network.

Step 35, the fine network processes the image data set X preprocessed in step 2₁∈R^{(N+2)×H×W×D}And a feature map F of the coarse network output_last∈R^Ch×H×W×DCombined into X along the channel dimension₂∈R^{(N+2+Ch)×H×W×D}As input to the fine network. Wherein Ch represents F_lastThe number of channels. Mixing X₂As input to the fine network, the output is a fine dose distribution Y_fine∈R^{H ×W×D}And confidence map M ∈ R^H×W×D。

Step 4, a Markov discriminator is adopted to resist the three-dimensional dose distribution predicted image and the three-dimensional dose distribution real image, and the prediction quality of the three-dimensional dose distribution image is improved;

step 41, the discriminator for generating the countermeasure network is a Markov discriminator consisting of 5 convolutional layers;

step 42, the discriminators are respectively represented by Y_fine∈R^H×W×DAnd X₁∈R^{(N+2)×H×W×D}Combined into X along the channel dimension₃∈R^{(N +3)×H×W×D}And by Y ∈ R^H×W×DAnd X₁∈R^{(N+2)×H×W×D}Combined into X along the channel dimension₄∈R^{(N+3)×H×W×D}As its input, the output is a matrix of NxN, N being a positive integer and N ≧ 1, each element of which represents a degree of confidence that the input image is a true image.

Step 5, jointly optimizing a prediction model through a reconstruction loss function, a reconstruction loss function with confidence coefficient weight and an antagonistic loss function;

step 51, the loss function of the generator is represented as:

L_AttG＝0.5×||Y-G(X₁)||₁+M^β⊙||Y-G(X₂)||₁，

L_G＝L_AttG+λ_LADV，

wherein G (-) is a generator, D (-) is an discriminator, M is a confidence map, β is an attention super parameter, and l is a two-matrix element-by-element multiplication operation. L is_GThe total loss function of the generator is represented by a loss function L of the reconstruction loss combined with the reconstruction loss with confidence weight_AttGAnd a penalty function L_ADVIn the composition, λ is an equilibrium coefficient. For L_ADV，p_data(X₂) Representing the probability distribution to which the input data of the fine network obeys,

is a mathematical expectation.

Step 52, the loss function of the discriminator is expressed as:

wherein, G (-) is a generator, and D (-) is a discriminator. L is a radical of an alcohol_DRepresenting the total loss function p of the discriminator_data(X₃) Predicting a dose distribution obeyed probability distribution, P, for a fine network_data(X₄) Is a probability distribution of the true dose,

is a mathematical expectation.

And step 53, performing optimization training on the generator and discriminator network by using an Adam optimization algorithm:

first estimate m by computing the first moment of the gradient_tAnd second moment estimate v_t：

m_t＝μ₁*m_t-1+(1-μ₁)*g_t，

Wherein u is₁And u₂Representing two different preset parameters, g_tAnd g_t ²Representing first and second order gradients, respectively, and then calculating corrections for first and second order moment estimates, respectively

Finally, according to the calculated correction value

And

calculating to obtain an updated value delta theta by combining the learning rate eta and the minimum deviation epsilon_t：

Using the update value delta theta_tAnd optimizing and learning the neural network parameters.

And 6, generating three-dimensional dose distribution by using the trained prediction model, namely predicting new data by using the trained two-stage generator network to generate the three-dimensional dose distribution.

Figure 2 is a block diagram illustrating a three-dimensional dose prediction system in personalized accurate radiotherapy planning, according to an exemplary embodiment. As shown in fig. 2, a three-dimensional dose prediction system in personalized accurate radiotherapy planning provided by an embodiment of the present invention includes:

the image acquisition module is used for acquiring radiotherapy related information such as an electronic computed tomography image, a dangerous organ structure mask image, a planning target area image, a three-dimensional dose distribution image and the like;

the data preprocessing module is used for preprocessing the four types of image data;

the model building module is used for building a two-stage generation network and a Markov discriminator network, and specifically comprises a prediction model generation network module and a Markov discrimination network module, wherein the prediction model generation network module consists of a rough network model and a fine network model, and the Markov discrimination network module consists of a Markov discriminator network model;

the model training module is used for inputting the acquired image data into a two-stage generator network to generate a three-dimensional dose distribution image and a confidence map, and a Markov discriminator is adopted to resist against a three-dimensional dose distribution predicted image and a three-dimensional dose distribution real image so as to improve the prediction quality of the three-dimensional dose distribution image;

and the radiotherapy dose prediction module predicts new data by using the trained two-stage generator network to generate three-dimensional dose distribution.

As a possible implementation manner of this embodiment, the image data obtaining module includes:

and the electronic computer tomography image acquisition module is used for carrying out computer tomography imaging on the patient by using the electronic computer tomography imaging equipment to obtain an electronic computer tomography image.

And the danger organ mask image acquisition module is used for delineating danger organs according to the electronic computed tomography images of the patient and creating a structural mask image for each danger organ so as to effectively protect normal tissues and organs and reduce the irradiated dose and volume of the tissues and organs when a radiotherapy plan is implemented.

The planning target area mask image acquisition module is used for preliminarily processing the data of the electronic computer tomography image of the patient, drawing a planning radiotherapy target area through discussion of a clinician and a physicist, and creating a planning target area mask image.

And the three-dimensional dose distribution image acquisition module is used for determining an optimal radiotherapy plan according to the requirements of a clinician after the contour of the radiotherapy target area and the dangerous organ is sketched, so that the irradiation dose of important organ tissues is controlled as far as possible not to exceed the tolerance dose while the tumor obtains enough radiotherapy dose, and a three-dimensional dose distribution image is created.

As a possible implementation manner of this embodiment, the image data preprocessing module includes:

and the electronic computer tomography image preprocessing module is used for carrying out normalization processing on the electronic computer tomography image.

And the danger organ mask image preprocessing module is used for carrying out independent thermal coding on the delineated danger organ structure mask image and combining the coded danger organ image along the channel dimension.

And the planned target area image preprocessing module is used for carrying out normalization processing on the planned target area mask image.

And the image data set construction module combines the preprocessed electronic computed tomography image, the dangerous organ structure mask image and the planning target area image along the channel dimension to be used as the input of the model.

As a possible implementation manner of this embodiment, the model training module includes:

and the rough network training module takes the preprocessed image data set as the input of a rough network, and the output of the rough network training module is rough dose distribution.

And the fine network training module is used for combining the preprocessed image data set with the feature map output by the coarse network along the channel dimension by the fine network to serve as the input of the fine network, and the output is a fine dose distribution and confidence map.

The Markov identification network training module comprises two parts of identifier input: 1) combining the preprocessed image dataset with the fine dose distribution output by the fine network along the channel dimension as input; 2) the preprocessed image dataset is combined with the true dose distribution along the channel dimension as input. The output of the markov decision network is a matrix, each element of which represents the degree of confidence that the input image is a true image.

And the model optimization module is used for performing optimization training on the generator and the discriminator by using an Adam optimization algorithm.

As shown in fig. 3, the process of three-dimensional dose prediction by using the three-dimensional dose prediction system in the personalized precise radiotherapy plan of the present invention is as follows:

step 1, acquiring radiotherapy related information such as an electronic computed tomography image, a dangerous organ structure mask image, a planned target area image, a three-dimensional dose distribution image and the like;

step 11, using an electronic computed tomography imaging device to carry out computed tomography imaging on a patient to obtain an electronic computed tomography image;

step 12, sketching out dangerous organs according to the electron computed tomography image of the patient, and establishing a structural mask image for each dangerous organ so as to effectively protect normal tissues and organs and reduce the irradiated dose and volume of the tissues and organs when a radiotherapy plan is implemented;

step 13, preliminarily processing the data of the electron computed tomography image of the patient, and drawing a planned radiotherapy target area by the discussion of a clinician and a physicist to create a mask image of the planned target area;

step 14, after the contour of the target area and the dangerous organ of the radiotherapy is outlined, a physicist determines an optimal radiotherapy plan according to the requirements of a clinician, so that the irradiation dose of important organ tissues is controlled as far as possible not to exceed the tolerance dose while the tumor obtains enough radiotherapy dose, and a three-dimensional dose distribution image is created;

step 2, carrying out data preprocessing operation on the image in the step 1;

step 21, carrying out normalization processing on the electron computer tomography image;

step 22, carrying out independent thermal coding on the sketched dangerous organ structure mask images, and combining the coded dangerous organ images along the channel dimension;

step 23, normalizing the mask image of the planned target area;

step 24, combining the preprocessed electron computed tomography image, the dangerous organ structure mask image and the planned target area image along the channel dimension to be used as the input of the model;

step 3, inputting the acquired image data into a two-stage generator network to generate a three-dimensional dose distribution image and a confidence map;

step 31, inputting the preprocessed image data into a rough network, wherein the output is rough dose distribution;

step 32, combining the preprocessed image data set with a characteristic graph output by the rough network along a channel dimension, and inputting the combined characteristic graph into a fine network, wherein the output is a fine dose distribution and confidence map;

step 4, a Markov discriminator is adopted to resist the three-dimensional dose distribution predicted image and the three-dimensional dose distribution real image, and the prediction quality of the three-dimensional dose distribution image is improved;

step 41, the discriminator input consists of two parts: 1) the fine dose distribution output with the pre-processed image dataset and the fine network is combined along the channel dimension as input 2) the pre-processed image dataset and the true dose distribution are combined along the channel dimension as input. The output of the Markov identification network is a matrix, and each element of the matrix represents the credibility of the input image as a real image;

step 5, jointly optimizing a prediction model through a reconstruction loss function, a reconstruction loss function with confidence coefficient weight and an antagonistic loss function;

51, performing optimization training on the generator and the discriminator by using an Adam optimization algorithm;

step 6, predicting the dose distribution by using the trained prediction model;

and step 61, predicting new data by using the trained two-stage generator network to generate three-dimensional dose distribution.

Calculation example: the invention takes an electronic computer tomography image, a dangerous organ structure mask image and a plan target area image as input, and adopts the three-dimensional dose prediction method in the personalized precise radiotherapy plan to predict the three-dimensional dose distribution of clinical radiotherapy.

The flow of this example is shown in fig. 3, and the input data is an electronic computed tomography image, a dangerous organ structure mask image, and a planned target region image. Fig. 4a, 4b and 4c show schematic views of the images, respectively an electron computed tomography image, a critical organ structure mask image, a planning target area image. In order to avoid the influence of the data problem on the performance and the result of the model, the original image data is preprocessed in the first step. Firstly, normalizing an electronic computed tomography image, secondly, performing independent thermal coding on a dangerous organ structure mask image, combining the coded dangerous organ images along the channel dimension, thirdly, normalizing a planned target area image, and lastly, combining the preprocessed electronic computed tomography image, the dangerous organ structure mask image and the planned target area image along the channel dimension to be used as the input of a model.

And constructing a data set by utilizing the preprocessed image, inputting the preprocessed data set into a rough network, outputting the data set into rough dose distribution, combining the preprocessed data set with a characteristic graph output by the rough network along a channel dimension, inputting the combined characteristic graph into a fine network, and outputting the combined characteristic graph into fine dose distribution and a confidence coefficient graph. A markov discriminator is used to counter the three dimensional dose distribution predictive image and the three dimensional dose distribution real image. The Markov discriminator input consists of two parts: 1) the preprocessed image dataset is combined with the fine dose distribution output by the fine network along the channel dimension as input, 2) the preprocessed image dataset is combined with the true dose distribution along the channel dimension as input. The output of the markov decision network is a matrix, each element of which represents the degree of confidence that the input image is a true image. And jointly optimizing the prediction model through a reconstruction loss function, a reconstruction loss function with confidence coefficient weight and a countervailing loss function.

After training is completed, the test set is used for testing, and a final radiotherapy dose prediction result is obtained, and the result is shown in fig. 5 a-6 e. The model training error is small, the difference between the predicted dose distribution and the real dose distribution is small, and the prediction result is accurate. Under 100 cases of verification samples, three-dimensional dose distribution shows that the dose difference is small, the point-to-point average dose difference is not higher than 2.55408 +/-1.04627 Gy, and the prediction performance is good.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, a division of modules is merely a division of logical functions, and an actual implementation may have another division, and for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or modules through some communication interfaces, and may be in an electrical, mechanical or other form.

Modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

In addition, functional modules in the embodiments provided in the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules are integrated into one module.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Although the embodiments of the present invention have been described with reference to the accompanying drawings, it is not intended to limit the scope of the present invention, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive efforts by those skilled in the art based on the technical solution of the present invention.

Claims (7)

1. A three-dimensional dose prediction method in personalized accurate radiotherapy planning is characterized by comprising the following steps:

step 1, acquiring an electron computer tomography image, a dangerous organ structure mask image, a planning target area image and information related to radiotherapy of a three-dimensional dose distribution image;

step 11, using an electronic computed tomography imaging device to scan and image the patient to obtain an electronic computed tomography image, wherein the electronic computed tomography image is defined as C e R^H×W×DWherein R represents the whole image area, H, W are the length and width of the image respectively, and D is the depth of the image;

step 12, before the radiotherapy plan, drawing N dangerous organs according to the electron computer tomography image of the patient, and creating a structural mask image for each dangerous organ so as to effectively protect the normal tissue and organ and reduce the irradiated dose and volume of the tissue and organ when the radiotherapy plan is implemented, wherein the structural mask image of the dangerous organ is defined as O_n∈R^H×W×D，n∈[1，N]Wherein R represents the whole image area, H and W are the length and width of the image respectively, D is the depth of the image, N is a positive integer and N is more than or equal to 1;

step 13, after the positioning scanning image data of the patient is preliminarily processed, a clinician and a physicist discuss and draw a planning radiotherapy target area, and a mask image P belonging to R of the planning target area is created^H×W×DWherein R represents the whole image area, H, W are the length and width of the image respectively, and D is the depth of the image;

step 14, after the contour of the target area and the dangerous organ of the radiotherapy is outlined, the physicist determines the optimal radiotherapy plan according to the requirements of the clinician, so as to ensure that the irradiation dose of the important organ tissues does not exceed the tolerance dose of the important organ tissues while the tumor obtains enough radiotherapy dose, and a three-dimensional dose distribution image Y epsilon R is created^H×W×DWherein R represents the whole image area, H, W are the length and width of the image respectively, and D is the depth of the image;

step 2, carrying out data preprocessing operation on the image in the step 1;

step 3, inputting the acquired image data into a two-stage generator network to generate a three-dimensional dose distribution image and a confidence map;

step 31, for the 3D-UNet, there is a coding path of three times down-sampling and a decoding path of three times up-sampling, each layer of the coding path includes two convolution and a maximum pooling operation, each layer of the decoding path includes a transposed convolution and two convolution operations, the features with the same resolution in the coding path and the decoding path are combined through jump connection, the last layer is a convolution layer with convolution kernel of 1 × 1 × 1, which is used to reduce output channels and generate final output result;

step 32, constructing a rough network model by using the 3D-UNet as a basic model, and in order to improve the calculation efficiency and the dose distribution prediction result, taking the residual learning idea as a reference, making the internal structure of the model have the capability of constant mapping from input to output, so as to ensure that the degradation caused by continuous stacking is avoided in the process of stacking the network, and assuming that the input of the residual module is defined as x, the output y can be represented as y ═ x + F (x, { W ═ W)_i}) of the formula (I), wherein F (x, { W)_i}) is a learning objective, i.e. directly taking identity mapping as a part of the network, translating the problem into learning a residual function F (x, { W)_iY-x, provided that F (x, { W)_i0, the identity map y x is formed, so the residual part can be represented as F W₃σ(W₂σ(W₁x)), where σ denotes the ReLU activation function, and W₁，W₂，W₃Weight referring to three-layer network, and based on the idea, 3D-U is replaced by residual error moduleNet, two convolution operations, one residual module is composed of 1 × 1 × 1, 3 × 3 × 3, 1 × 1 × 1, 3 convolution layers in sequence, and the input and output of the residual module are added through residual connection to obtain the final output result of the module, furthermore, in order to effectively utilize the low-level features of the coding network, the scSE attention module is embedded into the last convolution layer of the residual module, the attention module is composed of two modules of channel attention cSE and space attention sSE, cSE compresses and excites in the channel, and the input feature graph U is [ U ═ U [ U ] of the attention module₁，u₂，...，u_c]Wherein each channel u_c∈R^H×W×DAnd obtaining a vector z after U passes through the global pooling layer, wherein the z belongs to R^1×1×1×cEach position k has a value of

Then, the two full-connection layers are passed through, and normalized to [0,1 ] by using Sigmoid activation function](ii) a sSE are compressed and excited in space, and its input characteristic diagram U ═ U^1，1，1，u^1，1，2，...，u^h，w，d，...，u^H，W，D]Spatial extrusion is achieved by convolution of 1 × 1 × 1, and then normalized to [ o, 1 ] by Sigmoid]Finally, the cSE and sSE modules are directly added, namely scSE cSE + sSE, the feature map size is reduced by four maximum pooling downsampling operations carried out on the coding path of the model, the feature map size is restored by four times of transposition convolution carried out on the decoding path, the features with the same resolution in the coding path and the decoding path are combined by using jump connection, and the feature map size finally output by the decoding path is reduced by using convolution of 1 × 1 × 1 at the last layer of the network;

step 33, the image data set X after the preprocessing of step 2 is processed₁∈R^{(N+2)×H×W×D}As input to the coarse network, its output is a coarse dose distribution Y_coarse∈R^H×W×D；

Step 34, constructing a fine network model by taking the 3D-UNet as a basic model, wherein the number of characteristic graphs of all convolution layers of the fine network model except the last layer is 2 times of that of the rough network, and the structure of the fine network model is completely the same as that of the rough network;

step 35, the fine network processes the image data set X preprocessed in step 2₁∈R^{(N+2)×H×W×D}And a feature map F of the coarse network output_last∈R^Ch×H×H×DCombined into X along the channel dimension₂∈R^{(N+2+Ch)×H×W×D}As input to the fine network, where Ch denotes F_lastNumber of channels of (2), will X₂As input to the fine network, the output is a fine dose distribution Y_fine∈R^H×W×DAnd confidence map M ∈ R^H×W×D；

Step 4, a Markov discriminator is adopted to resist the three-dimensional dose distribution predicted image and the three-dimensional dose distribution real image, and the prediction quality of the three-dimensional dose distribution image is improved;

step 5, jointly optimizing a prediction model through a reconstruction loss function, a reconstruction loss function with confidence coefficient weight and an antagonistic loss function;

and 6, generating three-dimensional dose distribution by using the trained prediction model, namely predicting new data by using the trained two-stage generator network to generate the three-dimensional dose distribution.

2. The method of predicting three-dimensional dose in personalized precision radiotherapy planning as set forth in claim 1, wherein the step 2 comprises the steps of:

step 21, carrying out electron computer tomography image C e R^H×W×DAnd (3) carrying out normalization treatment:

wherein C (H, W, D) represents the pixel value of the electron computer tomography image in the coordinate (H, W, D), H belongs to [0, H ], W belongs to [0, W ], D belongs to [0, D), C_max，C_minRespectively representing the maximum value and the minimum value in all pixel values in the electronic computed tomography image;

step 22, mask images O of the N danger organ structures delineated in step 12_n∈R^H×W×D，n∈[1，N]Performing one-hot encoding, and combining the encoded N danger organ images into O along the channel dimension_one-hot∈R^N×H×W×D；

Step 23, mask image P E R of planning target area^H×W×DAnd (3) carrying out normalization treatment:

P_ptv70(h，w，d)、P_ptv63(h，w，d)、P_ptv56(H, W, D) respectively represents a pixel value of coordinates (H, W, D) in images representing planned target doses of 70Gy, 63Gy, 56Gy, and P (H, W, D) represents a pixel value at coordinates (H, W, D) after normalization and summation of the planned target images, wherein H belongs to [0, H ], W belongs to [0, W ], and D belongs to [0, D);

step 24, making the preprocessed electron computer tomography image C E R^H×W×DMask image of dangerous organ Structure O_one-hot∈R^N×H×W×DPlanning target region image P ∈ R^H×W×DCombined into X along the channel dimension₁∈R^{(N+2)×H×W×D}And used as input data for the model.

3. The method of predicting three-dimensional dose in personalized precision radiotherapy planning as set forth in claim 2, wherein the step 4 comprises the steps of:

step 41, the discriminator for generating the countermeasure network is a Markov discriminator consisting of 5 convolutional layers;

step 42, the discriminators are respectively set by Y_fine∈R^H×W×DAnd X₁∈R^{(N+2)×H×W×D}Combined into X along the channel dimension₃∈R^{(N +3)×H×W×D}And by Y ∈ R^H×W×DAnd X₁∈R^{(N+2)×H×W×D}Combined into X along the channel dimension₄∈R^{(N+3)×H×W×D}As its input, the output is a matrix of NxN, N is a positive integer and N ≧ 1, each element of which represents a pair-input graphLike the degree of confidence of the real image.

4. The method of predicting three-dimensional dose in personalized precision radiotherapy planning as set forth in claim 3, wherein said step 5 comprises the steps of:

step 51, the loss function of the generator is represented as:

L_AttG＝0.5×||Y-G(X₁)||₁+M^β⊙||Y-G(X₂)||₁，

L_G＝L_AttG+λ_LADV，

wherein G (-) is a generator, D (-) is an discriminator, M is a confidence map, β is an attention over parameter, L is a two-matrix element-by-element multiplication operation_GThe total loss function of the generator is represented by a loss function L of the reconstruction loss combined with the reconstruction loss with confidence weight_AttGAnd a penalty function L_ADVComposition, λ is the equilibrium coefficient, for L_ADV，p_data(X₂) Representing the probability distribution to which the input data of the fine network obeys,

is a mathematical expectation; g (X)₁) For preprocessing the image data set X in step 2₁The prediction result output after the coarse network G (-) sent to step 32 has a shape of 1 × H × W × D; g (X)₂) To convert X in step 35₂The prediction result output after the fine network G (-) of step 34 has a shape of 1 × H × W × D;

represents (D (G (X))₂)-1))²Of (2), wherein X₂Obeying a Gaussian probability distribution Pdata (X)₂)；D(G(X₂) Denotes that G (X)₂) The output result sent to the discriminator D (-) of step 41 has a shape of 1 × H × W × D;

step 52, the loss function of the discriminator is expressed as:

wherein G (-) is a generator, D (-) is a discriminator, and L (-) is a signal generator_DRepresenting the total loss function p of the discriminator_data(X₃) Predicting a dose distribution obeyed probability distribution, P, for a fine network_data(X₄) The probability distribution to which the true dose distribution obeys,

is a mathematical expectation;

represents (D (G (X))₃)))²Of (2), wherein X₃Obeying a Gaussian probability distribution Pdata (X)₃)；

Represents (D (X)₄)-1)²Of (2), wherein X₄Obeying a Gaussian probability distribution Pdata (X)₄)；D(X₄) X denotes a step 42₄The output result sent to the discriminator D (-) of step 41 has a shape of 1 × H × W × D; d (G (X)₃) Denotes that G (X)₃) The output result sent to the discriminator D (-) of step 41 has a shape of 1 × H × W × D; g (X)₃) Indicates that X in step 42 is₃The prediction result output after the fine network G (-) of step 34 has a shape of 1 × H × W × D;

and step 53, performing optimization training on the generator and discriminator network by using an Adam optimization algorithm:

first estimate m by computing the first moment of the gradient_tAnd second moment estimate v_t：

m_t＝μ₁*m_t-1+(1-μ₁)*g_t，

Wherein u is₁And u₂Representing two different preset parameters, g_tAnd g_t ²Representing first and second order gradients, respectively, and then calculating corrections for first and second order moment estimates, respectively

And

is μ at time t₁And mu₂Respectively controlling the weight distribution and the influence of the square of the gradient before control; finally, the correction value is calculated according to the calculated correction value

And with

Calculating to obtain an updated value delta theta by combining the learning rate eta and the minimum deviation epsilon_t：

Using the update value delta theta_tAnd optimizing and learning the neural network parameters.

5. A three-dimensional dose prediction system in personalized accurate radiotherapy planning is characterized by comprising:

the image acquisition module is used for acquiring an electronic computed tomography image, a dangerous organ structure mask image, a planning target area image and information related to three-dimensional dose distribution image radiotherapy;

the data preprocessing module is used for preprocessing the four types of image data;

the model building module is used for building a two-stage generation network and a Markov discriminator network, and specifically comprises a prediction model generation network module and a Markov discrimination network module, wherein the prediction model generation network module consists of a rough network model and a fine network model, and the Markov discrimination network module consists of a Markov discriminator network model;

the rough network training module takes the preprocessed image data set as the input of a rough network, and the output of the rough network training module is rough dose distribution;

the fine network training module is used for combining the preprocessed image data set and the feature map output by the coarse network along the channel dimension by the fine network to serve as the input of the fine network, and the output of the fine network is a fine dose distribution and confidence map;

the model training module is used for inputting the acquired image data into a two-stage generator network to generate a three-dimensional dose distribution image and a confidence map, and a Markov discriminator is adopted to resist against a three-dimensional dose distribution predicted image and a three-dimensional dose distribution real image so as to improve the prediction quality of the three-dimensional dose distribution image;

the Markov identification network training module comprises two parts of identifier input: 1) combining the preprocessed image dataset with the fine dose distribution output by the fine network along the channel dimension as input 2) combining the preprocessed image dataset with the real dose distribution along the channel dimension as input; the output of the Markov identification network is a matrix, and each element of the matrix represents the credibility of the input image as a real image;

the model optimization module is used for performing optimization training on the generator and the discriminator by using an Adam optimization algorithm;

and the radiotherapy dose prediction module comprises a model prediction module, and predicts new data by using the trained two-stage generator network to generate three-dimensional dose distribution.

6. The system of claim 5, wherein the image data acquisition module comprises:

the electronic computer tomography image acquisition module is used for carrying out computer tomography imaging on the patient by using an electronic computer tomography imaging device to obtain an electronic computer tomography image;

the dangerous organ mask image acquisition module is used for delineating dangerous organs according to the electronic computed tomography image of the patient and establishing a structural mask image for each dangerous organ so as to effectively protect normal tissues and organs and reduce irradiated dose and volume of the tissues and organs when a radiotherapy plan is implemented;

the planning target area mask image acquisition module is used for preliminarily processing the data of the electronic computed tomography image of the patient, drawing a planning radiotherapy target area through the discussion of a clinician and a physicist and creating a planning target area mask image;

and the three-dimensional dose distribution image acquisition module is used for determining an optimal radiotherapy plan according to the requirements of a clinician after the contour of the radiotherapy target area and the dangerous organ is sketched, so that the irradiation dose of important organ tissues is controlled as far as possible not to exceed the tolerance dose while the tumor obtains enough radiotherapy dose, and a three-dimensional dose distribution image is created.

7. The system of claim 5, wherein the image data preprocessing module comprises:

the electronic computer tomography image preprocessing module is used for carrying out normalization processing on the electronic computer tomography image;

the danger organ mask image preprocessing module is used for carrying out independent thermal coding on the delineated danger organ structure mask image and combining the coded danger organ image along the channel dimension;

the planned target area image preprocessing module is used for carrying out normalization processing on the planned target area mask image;

and the image data set construction module combines the preprocessed electronic computed tomography image, the dangerous organ structure mask image and the planning target area image along the channel dimension to be used as the input of the model.

Images