CN112991355B - 3D brain lesion segmentation method based on optimal transmission - Google Patents

3D brain lesion segmentation method based on optimal transmission Download PDF

Info

Publication number
CN112991355B
CN112991355B CN202110519585.8A CN202110519585A CN112991355B CN 112991355 B CN112991355 B CN 112991355B CN 202110519585 A CN202110519585 A CN 202110519585A CN 112991355 B CN112991355 B CN 112991355B
Authority
CN
China
Prior art keywords
brain
optimal transmission
mapping
cuboid
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110519585.8A
Other languages
Chinese (zh)
Other versions
CN112991355A (en
Inventor
丘成桐
林文伟
李铁香
黄聪明
乐美亨
王瀚
谭忠恒
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Applied Mathematics Center
Original Assignee
Nanjing Applied Mathematics Center
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Applied Mathematics Center filed Critical Nanjing Applied Mathematics Center
Priority to CN202110519585.8A priority Critical patent/CN112991355B/en
Publication of CN112991355A publication Critical patent/CN112991355A/en
Application granted granted Critical
Publication of CN112991355B publication Critical patent/CN112991355B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06T3/04—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G06T2207/10012—Stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30016—Brain
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion

Abstract

The invention designs a 3D brain lesion segmentation method based on two-stage optimal transmission, which mainly comprises the following steps: developing an effective, reliable and stable two-stage optimal transmission numerical algorithm aiming at the 3D brain medical image, mapping the two-stage optimal transmission of the 3D brain image to a cuboid, and only reducing the precision loss generated by the conversion between the two to about 0.5%; selecting a cuboid image obtained by mapping with least precision loss, putting the cuboid image into a neural network for training, and obtaining a prediction result of a data set by using a data set training model provided by an authoritative website; the optimal transmission mapping is used for inverse mapping, the optimal transmission mapping is restored to the original brain image, and the overall lesion segmentation precision of the obtained training set and the test set respectively reaches 98.5% and 92.0%.

Description

3D brain lesion segmentation method based on optimal transmission
Technical Field
The invention relates to an optimal transmission application method of brain intelligent medical images, and belongs to the field of medical image prediction processing.
Background
The segmentation and quantitative evaluation of lesions require very specialized knowledge and techniques, which correspond to the segmentation of brain tumors, which are mainly reflected on the brain; this has prompted great progress in recent years in the development of techniques of Artificial Intelligence (AI) and Machine Learning (ML) to perform precise segmentation of tumors within human organs, thereby reducing the interpretation burden on professional physicians and providing visual and pixel-level semantic interpretations for clinicians.
However, the medical image segmentation problem, especially the three-dimensional image segmentation, needs to perform efficient preprocessing on the original image so as to apply the original image to a corresponding machine learning algorithm. For brain medical images, the global information of the brain is required to be efficiently and completely obtained from images, and foreign teams randomly select 16 groups of 128^3 cubes to cover a 3D brain image by utilizing Gaussian distribution so as to achieve the effect, but the method has complicated steps and expensive calculation process; other teams use a sliding window combination IoU approach, which is also very local to one small window, and global information of data is easily lost. The best segmentation results are obtained from the german cancer research center team, the test precision of which on the whole tumor reaches 88.95%, on the core tumor reaches 85.06%, and on the reinforced tumor reaches 82.03%, while the 3D brain tumor segmentation precision can be further improved by the method.
Disclosure of Invention
The purpose of the invention is as follows: aiming at the problems in the prior art, the invention provides a 3D brain lesion segmentation method based on optimal transmission, which aims to effectively reduce the precision loss caused by brain image preprocessing, ensure the integrity of image global information, enlarge the influence of lesion information and improve the lesion segmentation precision by using a proper neural network model.
The technical scheme is as follows: A3D brain lesion segmentation method based on optimal transmission is characterized by comprising the following steps:
step 1: constructing a transmission cost function aiming at 3D brain medical images, establishing a corresponding optimal transmission model, and designing an algorithm to quickly solve the model problem; calculating two-stage optimal transmission mapping from the 3D brain to a cuboid by using an optimal transmission model to obtain a 3D brain medical image data set represented by a tensor;
step 1.1: build up of the 3D brain: (
Figure 849484DEST_PATH_IMAGE001
) A guaranteed quality optimal transmission model to a unit sphere, calculating a corresponding optimal transmission map
Figure 297783DEST_PATH_IMAGE002
. The optimal transmission model is as follows,
Figure 608678DEST_PATH_IMAGE003
wherein the content of the first and second substances,
Figure 535046DEST_PATH_IMAGE004
representing a set of 3D brain to unit sphere mappings,
Figure 880577DEST_PATH_IMAGE005
representation collection
Figure 234197DEST_PATH_IMAGE004
The mapping of (1) to (2),
Figure 32389DEST_PATH_IMAGE006
a collection of 3D mid-brain points is represented,
Figure 762448DEST_PATH_IMAGE007
to represent
Figure 165747DEST_PATH_IMAGE006
The point (b) in (c) is,
Figure 690270DEST_PATH_IMAGE008
indicating points
Figure 975757DEST_PATH_IMAGE007
Is transmitted to a point
Figure 243928DEST_PATH_IMAGE009
The cost of the transmission of (a) is,
Figure 298471DEST_PATH_IMAGE010
indicating points
Figure 525053DEST_PATH_IMAGE007
The mass of (c);
step 1.2: build lengthCube (
Figure 297837DEST_PATH_IMAGE011
) A guaranteed quality optimal transmission model to a unit sphere, calculating a corresponding optimal transmission map
Figure 104119DEST_PATH_IMAGE012
. The optimal transmission model is as follows,
Figure 278749DEST_PATH_IMAGE013
wherein the content of the first and second substances,
Figure 145074DEST_PATH_IMAGE014
representing the set of mappings from cuboids to unit spheres,
Figure 608416DEST_PATH_IMAGE015
representation collection
Figure 218389DEST_PATH_IMAGE014
The mapping of (1) to (2),
Figure 247525DEST_PATH_IMAGE016
a collection of the mid-points of the cuboid is shown,
Figure 815909DEST_PATH_IMAGE017
to represent
Figure 297706DEST_PATH_IMAGE016
The point (b) in (c) is,
Figure 711370DEST_PATH_IMAGE018
indicating points
Figure 595012DEST_PATH_IMAGE017
Is transmitted to a point
Figure 271981DEST_PATH_IMAGE019
The cost of the transmission of (a) is,
Figure 241074DEST_PATH_IMAGE020
indicating points
Figure 192850DEST_PATH_IMAGE017
The mass of (c);
step 1.3: the brain MRI image is different from a general image and is a four-channel 3D image, and each channel stores images shot by different signal sources of the same brain and displays the images with different resolutions; among them, the MRI brain image contains a large amount of air, and the volume of the air accounts for about 83% of the image, which brings extra work and negative influence on the image segmentation. The optimal transmission can change irregular brain into cuboid in a quality-preserving manner, so that the influence of air on brain lesion segmentation is eliminated; solving the two optimal transmission models by using a projection gradient method, and utilizing the solved optimal transmission mapping
Figure 462157DEST_PATH_IMAGE021
And
Figure 106765DEST_PATH_IMAGE002
and calculating two-stage optimal transmission mapping from the 3D brain to the cuboid:
Figure 563154DEST_PATH_IMAGE022
double shot
Figure 318621DEST_PATH_IMAGE023
The one-to-one correspondence between the 3D brain and the cuboid is realized;
step 2: adopting a convolution neural network model most suitable for 3D brain image segmentation processing, and inputting tensor form data of 3D brain medical image data provided by an authoritative website media data.
And step 3: restoring the prediction result on the cuboid to the 3D brain by means of the inverse mapping of the two-stage optimal transmission mapping
Figure 114538DEST_PATH_IMAGE023
Is inverse ofAnd (5) restoring the prediction result of the tensor form to the original 3D brain to obtain the prediction result of the 3D brain lesion segmentation.
Has the advantages that:
1. the invention realizes the one-to-one mapping transformation of the brain image, namely the corresponding transformation of a 3D brain to a cuboid, thereby saving the input data volume. In a general method, the input data volume is about 8 times of the original data volume; the method of the invention ensures one-to-one conversion, so that the input data volume is consistent with the original data volume;
2. ensuring that the image is converted into a cuboid shape and accords with the input rule of a neural network model;
3. by means of data enhancement, the precision loss caused by brain image transformation processing is effectively reduced, and the transformation precision of the whole tumor, the tumor core and the strengthened tumor can be respectively improved by 1.49%, 2.88% and 1.43%;
4. after image processing, the global information of the brain is reserved, the lesion volume is enlarged, the proportion of the whole tumor in the brain is enlarged by 1.86 times, and the influence of the lesion is enhanced while the integrity of the global information of the image is ensured;
5. the most appropriate convolutional neural network model is selected, the lesion segmentation precision is effectively improved, and compared with the existing best result, the segmentation precision of the whole tumor, the tumor core and the strengthened tumor can be respectively improved by 3.07%, 2.88% and 2.17%.
Drawings
FIG. 1 is a flow chart of the method steps of the present invention.
Fig. 2 is a schematic diagram of the effect of the optimal transmission transformation, wherein,
Figure 930048DEST_PATH_IMAGE002
represents a 3D brain to unit sphere mapping;
Figure 873733DEST_PATH_IMAGE021
representing the mapping of a cuboid to a unit sphere,
Figure 698469DEST_PATH_IMAGE024
representing its inverse mapping;
Figure 411210DEST_PATH_IMAGE025
represents
Figure 132042DEST_PATH_IMAGE002
And
Figure 563023DEST_PATH_IMAGE024
combined two-phase mapping.
FIG. 3 is a diagram of a U-net neural network architecture used in an embodiment; wherein concat refers to the tandem layer; the conv (+ BN) + ReLu means that normalization processing is added by a convolution layer, and a ReLu activation function is output; maxpool refers to the largest pooling layer; up-conv refers to up-sampling or deconvolution.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some embodiments of the present invention, but not all embodiments. Thus, the following detailed description of the embodiments of the invention presented in the figures is not intended to limit the scope of the invention as claimed.
Example 1
As shown in fig. 1, the invention provides a 3D brain lesion segmentation method based on optimal transmission, which comprises the following specific steps:
step 1: constructing a transmission cost function aiming at 3D brain medical images, establishing a corresponding optimal transmission model, and designing an algorithm to quickly solve the model problem; calculating two-stage optimal transmission mapping from the 3D brain to a cuboid by using an optimal transmission model to obtain a 3D brain medical image data set represented by a tensor;
step 1.1: build up of the 3D brain: (
Figure 863554DEST_PATH_IMAGE001
) A guaranteed quality optimal transmission model to a unit sphere, calculating a corresponding optimal transmission map
Figure 696381DEST_PATH_IMAGE002
. The optimal transmission model is as follows,
Figure 588114DEST_PATH_IMAGE003
wherein the content of the first and second substances,
Figure 506391DEST_PATH_IMAGE004
representing a set of 3D brain to unit sphere mappings,
Figure 407351DEST_PATH_IMAGE005
representation collection
Figure 94685DEST_PATH_IMAGE004
The mapping of (1) to (2),
Figure 422898DEST_PATH_IMAGE006
a collection of 3D mid-brain points is represented,
Figure 562892DEST_PATH_IMAGE007
to represent
Figure 470805DEST_PATH_IMAGE006
The point (b) in (c) is,
Figure 809383DEST_PATH_IMAGE008
indicating points
Figure 511759DEST_PATH_IMAGE007
Is transmitted to a point
Figure 139050DEST_PATH_IMAGE009
The cost of the transmission of (a) is,
Figure 647392DEST_PATH_IMAGE010
indicating points
Figure 43738DEST_PATH_IMAGE007
The mass of (c);
step 1.2: build up a rectangular parallelepiped
Figure 713754DEST_PATH_IMAGE011
) A guaranteed quality optimal transmission model to a unit sphere, calculating a corresponding optimal transmission map
Figure 828340DEST_PATH_IMAGE012
. The optimal transmission model is as follows,
Figure 812477DEST_PATH_IMAGE013
wherein the content of the first and second substances,
Figure 328909DEST_PATH_IMAGE014
representing the set of mappings from cuboids to unit spheres,
Figure 169826DEST_PATH_IMAGE015
representation collection
Figure 771708DEST_PATH_IMAGE014
The mapping of (1) to (2),
Figure 356273DEST_PATH_IMAGE016
a collection of the mid-points of the cuboid is shown,
Figure 727212DEST_PATH_IMAGE017
to represent
Figure 4609DEST_PATH_IMAGE016
The point (b) in (c) is,
Figure 828209DEST_PATH_IMAGE018
indicating points
Figure 216465DEST_PATH_IMAGE017
Is transmitted to a point
Figure 645172DEST_PATH_IMAGE019
The cost of the transmission of (a) is,
Figure 827892DEST_PATH_IMAGE020
indicating points
Figure 404367DEST_PATH_IMAGE017
The mass of (c);
step 1.3: the brain MRI image is different from a general image and is a four-channel 3D image, and each channel stores images shot by different signal sources of the same brain and displays the images with different resolutions; among them, the MRI brain image contains a large amount of air, and the volume of the air accounts for about 83% of the image, which brings extra work and negative influence on the image segmentation. The optimal transmission can change irregular brain into cuboid in a quality-preserving manner, so that the influence of air on brain lesion segmentation is eliminated; solving the two optimal transmission models by using a projection gradient method, and utilizing the solved optimal transmission mapping
Figure 330734DEST_PATH_IMAGE021
And
Figure 676265DEST_PATH_IMAGE002
and calculating two-stage optimal transmission mapping from the 3D brain to the cuboid:
Figure 295465DEST_PATH_IMAGE022
double shot
Figure 93657DEST_PATH_IMAGE023
The one-to-one correspondence between the 3D brain and the cuboid is realized;
step 2: building a multi-layer U-net model architecture, inputting 3D brain image data represented by tensor into a model for training, and obtaining prediction data of test set data on the basis of the trained model; by adopting a U-net model which is most suitable for 3D brain image segmentation processing and comprises a plurality of coding layers and a plurality of decoding layers, the coding layers are connected with the decoding layers through bridge layers and are linked in series, and the model is autonomously programmed by Python or MATLAB; com, the tensor form data of the medical image data are input into a program by means of an authoritative website, and the prediction of image segmentation is realized;
and step 3: restoring the prediction result on the cuboid to the 3D brain by means of the inverse mapping of the two-stage optimal transmission mapping
Figure 26978DEST_PATH_IMAGE023
And (3) the inverse mapping of (1) and restoring the prediction result of the tensor form to the original 3D brain to obtain the prediction result of the 3D brain lesion segmentation. After the pathological change result of the cuboid is predicted through the U-net model, the result needs to be restored to the 3D brain, the two-stage optimal transmission mapping obtained in the step 1 is bijection, the 3D brain can be mapped into the cuboid, and meanwhile, the cuboid can be restored to the 3D brain through inverse mapping.
Example 2
As shown in fig. 1, the invention provides a 3D brain lesion segmentation method based on optimal transmission, which mainly performs tumor segmentation on a brain, and specifically includes the following steps:
step 1: constructing a transmission cost function aiming at 3D brain medical images, establishing a corresponding optimal transmission model, and designing an algorithm to quickly solve the model problem; calculating two-stage optimal transmission mapping from the 3D brain to a cuboid by using an optimal transmission model to obtain a 3D brain medical image data set represented by a tensor;
step 1.1: using the square of the Euclidean distance
Figure 961436DEST_PATH_IMAGE026
As a function of the transmission cost, a 3D brain is built (
Figure 751537DEST_PATH_IMAGE027
) A guaranteed quality optimal transmission model to a unit sphere, calculating a corresponding optimal transmission map
Figure 37025DEST_PATH_IMAGE028
. The optimal transmission model is as follows,
Figure 570775DEST_PATH_IMAGE029
wherein the content of the first and second substances,
Figure 625318DEST_PATH_IMAGE004
representing a set of 3D brain to unit sphere mappings,
Figure 586321DEST_PATH_IMAGE005
representation collection
Figure 93526DEST_PATH_IMAGE004
The mapping of (1) to (2),
Figure 368649DEST_PATH_IMAGE006
a collection of 3D mid-brain points is represented,
Figure 543279DEST_PATH_IMAGE007
to represent
Figure 409604DEST_PATH_IMAGE006
The point (b) in (c) is,
Figure 669684DEST_PATH_IMAGE010
indicating points
Figure 545236DEST_PATH_IMAGE007
The mass of (c);
step 1.2: build up a rectangular parallelepiped
Figure 574372DEST_PATH_IMAGE030
) A guaranteed quality optimal transmission model to a unit sphere, calculating a corresponding optimal transmission map
Figure 611598DEST_PATH_IMAGE031
. The optimal transmission model is as follows,
Figure 358974DEST_PATH_IMAGE032
wherein the content of the first and second substances,
Figure 975900DEST_PATH_IMAGE014
representing the set of mappings from cuboids to unit spheres,
Figure 859543DEST_PATH_IMAGE015
representation collection
Figure 333249DEST_PATH_IMAGE014
The mapping of (1) to (2),
Figure 567921DEST_PATH_IMAGE016
a collection of the mid-points of the cuboid is shown,
Figure 519697DEST_PATH_IMAGE017
to represent
Figure 523425DEST_PATH_IMAGE016
The point (b) in (c) is,
Figure 902454DEST_PATH_IMAGE020
indicating points
Figure 358843DEST_PATH_IMAGE017
The mass of (c);
step 1.3: using projection gradient method, solving
Figure 317572DEST_PATH_IMAGE031
And
Figure 175806DEST_PATH_IMAGE028
optimal transmission model of (2): inputting three-dimensional manifold
Figure 991315DEST_PATH_IMAGE033
(including 3D brains
Figure 935001DEST_PATH_IMAGE027
And a rectangular parallelepiped
Figure 494158DEST_PATH_IMAGE030
) Subdivision of tetrahedron (
Figure 206899DEST_PATH_IMAGE034
A set of vertices is represented that is representative of,
Figure 193310DEST_PATH_IMAGE035
the set of representative edges is then used to represent the set of edges,
Figure 827553DEST_PATH_IMAGE036
represents a collection of triangular faces of the object,
Figure 924822DEST_PATH_IMAGE037
representing a collection of tetrahedrons), three-dimensional manifold surface triangular elements
Figure 492070DEST_PATH_IMAGE038
Slice constant density function of
Figure 649382DEST_PATH_IMAGE039
Three-dimensional manifold tetrahedral unit
Figure 833238DEST_PATH_IMAGE037
Slice constant density function of
Figure 468619DEST_PATH_IMAGE040
Error limit
Figure 155952DEST_PATH_IMAGE041
(ii) a Computing initial boundary maps by the stretching energy minimization method (ASEM)
Figure 484166DEST_PATH_IMAGE042
Iteratively calculating the optimal rotation
Figure 827422DEST_PATH_IMAGE043
Wherein
Figure 532073DEST_PATH_IMAGE044
Represents a two-dimensional unit sphere surface,
Figure 73913DEST_PATH_IMAGE045
indicating rotationThe set of the conversion operators is then converted,
Figure 573027DEST_PATH_IMAGE046
representing a set of vertices
Figure 465897DEST_PATH_IMAGE047
A point of (1);
order to
Figure 443080DEST_PATH_IMAGE048
，
Figure 105006DEST_PATH_IMAGE049
Respectively represent boundary points and interior points, order
Figure 775021DEST_PATH_IMAGE050
Iterative solution of linear systems
Figure 92870DEST_PATH_IMAGE051
Obtaining an optimal transmission mapping
Figure 873744DEST_PATH_IMAGE052
Induced vector of
Figure 390176DEST_PATH_IMAGE053
I.e. mapping of the three-dimensional manifold to the best transmission of the unit sphere with guaranteed quality
Figure 231094DEST_PATH_IMAGE054
Induced vector of
Figure 98555DEST_PATH_IMAGE053
Wherein
Figure 683120DEST_PATH_IMAGE055
Representing three-dimensional unit spheres, matrices
Figure 54059DEST_PATH_IMAGE056
Is represented as follows:
Figure 269140DEST_PATH_IMAGE058
Figure 92739DEST_PATH_IMAGE060
wherein
Figure 480995DEST_PATH_IMAGE061
Representation matrix
Figure 706440DEST_PATH_IMAGE056
Middle corresponding index set
Figure 154739DEST_PATH_IMAGE062
A row of
Figure 731214DEST_PATH_IMAGE063
Together with the columns of the sub-matrix,
Figure 657582DEST_PATH_IMAGE064
the same process is carried out;
Figure 940795DEST_PATH_IMAGE065
representing a set of tetrahedral edges resulting from a three-dimensional manifold subdivision,
Figure 559996DEST_PATH_IMAGE066
representing the vertices on the corresponding tetrahedral area,
Figure 358187DEST_PATH_IMAGE067
representing triangles
Figure 822667DEST_PATH_IMAGE068
And a triangle
Figure 288283DEST_PATH_IMAGE069
The two-surface angle is formed by the two-surface angle,
Figure 78385DEST_PATH_IMAGE070
、
Figure 98293DEST_PATH_IMAGE071
both represent the edges in the tetrahedron connecting these two points,
Figure 655480DEST_PATH_IMAGE072
representing a set of tetrahedrons
Figure 913286DEST_PATH_IMAGE073
The tetrahedron (a) in (b),
Figure 608710DEST_PATH_IMAGE074
representing tetrahedrons
Figure 381494DEST_PATH_IMAGE072
The volume of (a) to (b),
Figure 453355DEST_PATH_IMAGE075
to represent
Figure 627984DEST_PATH_IMAGE072
(ii) a density of (d); finally outputting the optimal transmission mapping
Figure 759888DEST_PATH_IMAGE052
Induced vector of
Figure 754389DEST_PATH_IMAGE053
，
Figure 833204DEST_PATH_IMAGE053
The result is the discrete value of the optimal transmission mapping;
mapping by optimal transmission
Figure 862339DEST_PATH_IMAGE021
And
Figure 899566DEST_PATH_IMAGE002
calculating two-stage optimal transmission mapping from the 3D brain to the cuboid:
Figure 646942DEST_PATH_IMAGE022
double shot
Figure 326185DEST_PATH_IMAGE023
The one-to-one correspondence between the 3D brain and the cuboid is realized; as shown in FIG. 2, the 3D brain first passes the first stage of mapping
Figure 209827DEST_PATH_IMAGE002
Converting into unit ball, and mapping from cube to unit ball
Figure 417955DEST_PATH_IMAGE021
Inverse mapping of
Figure 387048DEST_PATH_IMAGE076
The unit ball is changed into a cuboid to realize two-stage conversion;
the resulting cuboid can be used directly for image segmentation, thereby converting the 3D brain data set into a cuboid data set, which has the following characteristics with respect to the 3D brain data set: (1) the whole cuboid represents a 3D brain, the influence of air on image segmentation is eliminated, (2) the cuboid is of equal density, a part with high gray value (density) is more likely to be a lesion part, and a high-density area in an original image can be enlarged after the cuboid is mapped, so that the image segmentation is more accurate, (3) the cuboid eliminates the air, and compared with the original image, the cuboid image is smaller, and the training speed of model training can be accelerated;
Figure 807665DEST_PATH_IMAGE077
TABLE 1 loss of conversion precision between 3D brain images and cubes of 96^3 and 128^3, respectively
As can be seen from Table 1, the transformation into the cube of 128^3 has less precision loss, so we have chosen to use the data of 128^3 for experiments in this case;
step 2: building a three-layer U-net model architecture, inputting 3D brain image data represented by tensor into a model for training, and obtaining prediction data of test set data on the basis of the trained model; by adopting a U-net model which is most suitable for 3D brain image segmentation processing, as shown in FIG. 3, each cuboid and upper number represent the output shape and channel value of each layer, the model comprises three coding layers and three decoding layers, and the coding layers and the decoding layers with the same resolution are connected through a bridge layer and are connected in series; com medical image data of an authoritative website is used for inputting tensor morphological data of the medical image data into a program by using Python and MATLAB autonomous programming, so that the prediction of image segmentation is realized;
Figure 545814DEST_PATH_IMAGE078
TABLE 2 number of parameters for each layer in the three-layer U-net model
Table 2 shows the change in the number of parameters in each layer of the U-net model, and the dimensions of the input data and the output data are still consistent through the learning of the model;
besides a three-layer U-net model, a four-layer U-net model architecture is also built, the model is similar to the three-layer U-net model, comprises four coding layers and four decoding layers, and is a similar model which is formed by adding one layer on the basis of the three-layer U-net model;
Figure 190421DEST_PATH_IMAGE079
TABLE 3 number of parameters for each layer in the four-layer U-net model
As can be seen from the above table, the four layers of U-nets increase parameters of tens of millions of levels compared with the three layers of U-nets, on the same equipment, the average time increased by each iteration is 167s, the accuracy improved by the previous 50 iterations is only within 0.02%, and the improvement of the subsequent iterations is almost negligible, so that the model selected in the study is a three-layer U-net model;
and step 3: restoring the prediction result of the cube to the 3D brain by means of the inverse mapping of the two-stage optimal transmission mapping, wherein the specific method is to utilize the two-stage optimal transmission
Figure 646811DEST_PATH_IMAGE080
And (3) the inverse mapping of (1) and restoring the prediction result of the tensor form to the original 3D brain to obtain the prediction result of the 3D brain lesion segmentation. After the pathological change result of the cube is predicted through the U-net model, the result needs to be restored to the 3D brain, the two-stage optimal transmission mapping obtained in the step 1 is bijection, the 3D brain can be mapped into the cube, and meanwhile, the cube can be restored to the brain through inverse mapping. In actual operation, the process only causes about 0.5% of precision loss, the loss is caused by objective hardware conditions and is inevitable, and the precision loss is acceptable compared with the segmentation precision improvement obtained by using the method;
Figure 667856DEST_PATH_IMAGE081
TABLE 4.3D segmentation accuracy of brain tumors
The table shows the brain tumor segmentation results based on the two-stage optimal transmission method, and it is seen from the table that after 1000 times of model iterative training, the training precision of each tumor can respectively reach 0.9852, 0.9743 and 0.9433, and the testing precision can respectively reach 0.9202, 0.8794 and 0.8420, and compared with the existing best results, the testing precision of the whole tumor, the tumor core and the strengthened tumor can be respectively improved by 3.07%, 2.88% and 2.17%.
Based on the tumor segmentation method of the two-stage optimal mapping, similar data processing can be carried out on medical images of other organs, such as pancreas, eyeballs, liver and the like, and then lesions in the organs are segmented through an efficient segmentation model; a set of complete algorithm library is expected to be established, whether the organ of the patient contains the tumor can be judged by inputting a medical image, and if the organ of the patient contains the tumor, the segmentation image of the tumor is output, so that extremely convenient service can be provided for the patient, assistance can also be provided for doctors, and the working efficiency of the doctors is greatly improved; in the future, a set of formed medical software is designed, the algorithm library is combined with the algorithm library and packaged into mature organ lesion distinguishing and segmenting software which is provided for hospitals with requirements, patients can be independently placed into machines of the hospitals after medical images are obtained, paper results are directly output by means of the lesion segmenting software in the machines and sent to hands of the patients, doctors can timely give medicines according to symptoms, and therefore medical efficiency is greatly improved, cure rate is improved, and death rate is reduced.
The technical means disclosed in the invention scheme are not limited to the technical means disclosed in the above embodiments, but also include the technical scheme formed by any combination of the above technical features. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and such improvements and modifications are also considered to be within the scope of the present invention.

Claims (3)

1. The 3D brain lesion segmentation method based on the optimal transmission is characterized by comprising the following steps of:
step 1: constructing a transmission cost function aiming at the 3D brain medical image, establishing a corresponding optimal transmission model, and solving the model problem by using an algorithm; calculating two-stage optimal transmission mapping from the 3D brain to a cuboid by using an optimal transmission model to obtain a 3D brain medical image data set represented by a tensor;
step 2: building an image segmentation mathematical model, inputting 3D brain image data expressed by tensor into the model for training, and obtaining a prediction result of a test set by using the trained model;
and step 3: restoring the prediction result on the cuboid to the 3D brain by means of the inverse mapping of the two-stage optimal transmission mapping to obtain the prediction result of the 3D brain lesion segmentation;
wherein, the step 1 comprises the following steps:
step 1.1: establishing a quality-guaranteed optimal transmission model from the 3D brain to the unit sphere, and calculating corresponding optimal transmission mapping
Figure 443537DEST_PATH_IMAGE001
(ii) a The optimal transmission model is as follows,
Figure 196729DEST_PATH_IMAGE003
wherein the content of the first and second substances,
Figure 216113DEST_PATH_IMAGE004
a representation of the 3D brain is shown,
Figure 447375DEST_PATH_IMAGE005
representing a set of 3D brain to unit sphere mappings,
Figure 769903DEST_PATH_IMAGE006
representation collection
Figure 693996DEST_PATH_IMAGE005
The mapping of (1) to (2),
Figure 187295DEST_PATH_IMAGE007
a collection of 3D mid-brain points is represented,
Figure 222247DEST_PATH_IMAGE008
to represent
Figure 868123DEST_PATH_IMAGE007
The point (b) in (c) is,
Figure 963118DEST_PATH_IMAGE009
indicating points
Figure 146974DEST_PATH_IMAGE008
Is transmitted to a point
Figure 595404DEST_PATH_IMAGE010
The cost of the transmission of (a) is,
Figure 220421DEST_PATH_IMAGE011
indicating points
Figure 610951DEST_PATH_IMAGE008
The mass of (c);
step 1.2: establishing a quality-guaranteed optimal transmission model from a cuboid to a unit sphere, and calculating corresponding optimal transmission mapping
Figure 954207DEST_PATH_IMAGE012
(ii) a The optimal transmission model is as follows,
Figure 468978DEST_PATH_IMAGE014
wherein the content of the first and second substances,
Figure 948501DEST_PATH_IMAGE015
which represents a rectangular parallelepiped,
Figure 509932DEST_PATH_IMAGE016
representing the set of mappings from cuboids to unit spheres,
Figure 74906DEST_PATH_IMAGE017
representation collection
Figure 865138DEST_PATH_IMAGE016
The mapping of (1) to (2),
Figure 792643DEST_PATH_IMAGE018
a collection of the mid-points of the cuboid is shown,
Figure 400342DEST_PATH_IMAGE019
to represent
Figure 327978DEST_PATH_IMAGE018
The point (b) in (c) is,
Figure 46535DEST_PATH_IMAGE020
indicating points
Figure 890863DEST_PATH_IMAGE019
Is transmitted to a point
Figure 669463DEST_PATH_IMAGE021
The cost of the transmission of (a) is,
Figure 84395DEST_PATH_IMAGE022
indicating points
Figure 606643DEST_PATH_IMAGE019
The mass of (c);
step 1.3: using projection gradient method, solving
Figure 39899DEST_PATH_IMAGE023
And
Figure 723821DEST_PATH_IMAGE024
optimal transmission model of (2): inputting three-dimensional manifold
Figure 623119DEST_PATH_IMAGE025
In which the three-dimensional manifold
Figure 276955DEST_PATH_IMAGE025
Including the 3D brain
Figure 236820DEST_PATH_IMAGE026
And a rectangular parallelepiped
Figure 357223DEST_PATH_IMAGE027
In the subdivision of tetrahedron
Figure 746747DEST_PATH_IMAGE028
A set of vertices is represented that is representative of,
Figure 938694DEST_PATH_IMAGE029
the set of representative edges is then used to represent the set of edges,
Figure 956329DEST_PATH_IMAGE030
represents a collection of triangular faces of the object,
Figure 388578DEST_PATH_IMAGE031
representing a set of tetrahedrons, three-dimensional manifold surface triangular elements
Figure 124453DEST_PATH_IMAGE032
Slice constant density function of
Figure 651249DEST_PATH_IMAGE033
Three-dimensional manifold tetrahedral unit
Figure 54549DEST_PATH_IMAGE031
Slice constant density function of
Figure 657700DEST_PATH_IMAGE034
Error limit
Figure 615291DEST_PATH_IMAGE035
(ii) a Computing initial boundary map by stretching energy minimization method
Figure 211358DEST_PATH_IMAGE036
Iteratively calculating the optimal rotation
Figure 203585DEST_PATH_IMAGE037
Wherein
Figure 732565DEST_PATH_IMAGE038
Represents a two-dimensional unit sphere surface,
Figure 770928DEST_PATH_IMAGE039
a set of rotation operators is represented, and,
Figure 780473DEST_PATH_IMAGE040
indicating topPoint collection
Figure 502572DEST_PATH_IMAGE041
A point of (1);
order to
Figure 572159DEST_PATH_IMAGE042
，
Figure 628977DEST_PATH_IMAGE043
，
Figure 442212DEST_PATH_IMAGE044
And
Figure 549977DEST_PATH_IMAGE045
respectively represent boundary points and interior points, order
Figure 852782DEST_PATH_IMAGE046
Iterative solution of linear systems
Figure 537841DEST_PATH_IMAGE047
Obtaining an optimal transmission mapping
Figure 764555DEST_PATH_IMAGE048
Induced vector of
Figure 585880DEST_PATH_IMAGE049
I.e. mapping of the three-dimensional manifold to the best transmission of the unit sphere with guaranteed quality
Figure 856324DEST_PATH_IMAGE050
Induced vector of
Figure 763100DEST_PATH_IMAGE049
Wherein
Figure 790575DEST_PATH_IMAGE051
A three-dimensional unit sphere is represented,matrix array
Figure 466407DEST_PATH_IMAGE052
Is represented as follows:
Figure 173332DEST_PATH_IMAGE053
Figure 567404DEST_PATH_IMAGE054
wherein
Figure 135920DEST_PATH_IMAGE055
Representation matrix
Figure 931837DEST_PATH_IMAGE052
Middle corresponding index set
Figure 809663DEST_PATH_IMAGE056
A row of
Figure 425453DEST_PATH_IMAGE057
Together with the columns of the sub-matrix,
Figure 797659DEST_PATH_IMAGE058
the same process is carried out;
Figure 448083DEST_PATH_IMAGE059
representing a set of tetrahedral edges resulting from a three-dimensional manifold subdivision,
Figure 762390DEST_PATH_IMAGE060
representing the vertices on the corresponding tetrahedral area,
Figure 131054DEST_PATH_IMAGE061
representing triangles
Figure 41373DEST_PATH_IMAGE062
And a triangle
Figure 546303DEST_PATH_IMAGE063
The two-surface angle is formed by the two-surface angle,
Figure 765932DEST_PATH_IMAGE064
、
Figure 90734DEST_PATH_IMAGE065
both represent the edges in the tetrahedron connecting these two points,
Figure 67393DEST_PATH_IMAGE066
representing a set of tetrahedrons
Figure 692409DEST_PATH_IMAGE067
The tetrahedron (a) in (b),
Figure 82940DEST_PATH_IMAGE068
representing tetrahedrons
Figure 160617DEST_PATH_IMAGE066
The volume of (a) to (b),
Figure 412738DEST_PATH_IMAGE069
to represent
Figure 626682DEST_PATH_IMAGE066
(ii) a density of (d); finally outputting the optimal transmission mapping
Figure 453692DEST_PATH_IMAGE048
Induced vector of
Figure 18666DEST_PATH_IMAGE049
，
Figure 74477DEST_PATH_IMAGE049
The result is the discrete value of the optimal transmission mapping;
mapping by optimal transmission
Figure 674086DEST_PATH_IMAGE070
And
Figure 406419DEST_PATH_IMAGE001
calculating two-stage optimal transmission mapping from the 3D brain to the cuboid:
Figure 193109DEST_PATH_IMAGE071
double shot
Figure 52612DEST_PATH_IMAGE072
The one-to-one correspondence between the 3D brain and the cuboid is realized; 3D brain first pass first phase mapping
Figure 506727DEST_PATH_IMAGE001
Converting into unit ball, and mapping from cube to unit ball
Figure 409961DEST_PATH_IMAGE070
Inverse mapping of
Figure 683947DEST_PATH_IMAGE073
The unit ball is changed into a cuboid to realize two-stage conversion;
step 1.4: mapping by optimal transmission
Figure 78632DEST_PATH_IMAGE070
And
Figure 715150DEST_PATH_IMAGE001
calculating two-stage optimal transmission mapping from the 3D brain to the cuboid:
Figure 664651DEST_PATH_IMAGE074
double shot
Figure 566879DEST_PATH_IMAGE075
And 3D brains are in one-to-one correspondence with the cuboids.
2. The optimal transmission based 3D brain lesion segmentation method according to claim 1, wherein the step 2 adopts a neural network model most suitable for 3D brain image segmentation processing, and uses medical image data to input tensor morphological data thereof into a program, so as to realize prediction of image lesion segmentation.
3. The optimal transmission based 3D brain lesion segmentation method according to claim 1, wherein the step 3 applies two-stage optimal transmission mapping
Figure 627239DEST_PATH_IMAGE075
And (3) the inverse mapping, namely restoring the prediction result of the tensor form to the original 3D brain form to obtain the final 3D brain lesion segmentation result.
CN202110519585.8A 2021-05-13 2021-05-13 3D brain lesion segmentation method based on optimal transmission Active CN112991355B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110519585.8A CN112991355B (en) 2021-05-13 2021-05-13 3D brain lesion segmentation method based on optimal transmission

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110519585.8A CN112991355B (en) 2021-05-13 2021-05-13 3D brain lesion segmentation method based on optimal transmission

Publications (2)

Publication Number Publication Date
CN112991355A CN112991355A (en) 2021-06-18
CN112991355B true CN112991355B (en) 2021-08-31

Family

ID=76337617

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110519585.8A Active CN112991355B (en) 2021-05-13 2021-05-13 3D brain lesion segmentation method based on optimal transmission

Country Status (1)

Country Link
CN (1) CN112991355B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111814871A (en) * 2020-06-13 2020-10-23 浙江大学 Image classification method based on reliable weight optimal transmission
CN112419267A (en) * 2020-11-23 2021-02-26 齐鲁工业大学 Brain glioma segmentation model and method based on deep learning
CN112766425A (en) * 2021-03-25 2021-05-07 浙江师范大学 Deep missing clustering machine learning method and system based on optimal transmission

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190091195A1 (en) * 2015-09-10 2019-03-28 Del Mar Pharmaceuticals Use of dianhydrogalactitol and derivatives thereof in the treatment of glioblastoma, lung cancer, and ovarian cancer

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111814871A (en) * 2020-06-13 2020-10-23 浙江大学 Image classification method based on reliable weight optimal transmission
CN112419267A (en) * 2020-11-23 2021-02-26 齐鲁工业大学 Brain glioma segmentation model and method based on deep learning
CN112766425A (en) * 2021-03-25 2021-05-07 浙江师范大学 Deep missing clustering machine learning method and system based on optimal transmission

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
图像分割算法和CT缺血性脑中风检测算法研究;钱晓华;《中国博士学位论文全文数据库 信息科技辑》;20130415;I138-58 *

Also Published As

Publication number Publication date
CN112991355A (en) 2021-06-18

Similar Documents

Publication Publication Date Title
Han et al. Combining noise-to-image and image-to-image GANs: Brain MR image augmentation for tumor detection
RU2720440C1 (en) Image segmentation method using neural network
Liu et al. An encoder-decoder neural network with 3D squeeze-and-excitation and deep supervision for brain tumor segmentation
Sinha et al. Deep learning 3D shape surfaces using geometry images
CN112465827B (en) Contour perception multi-organ segmentation network construction method based on class-by-class convolution operation
US20230104173A1 (en) Method and system for determining blood vessel information in an image
Dong et al. Multi-view secondary input collaborative deep learning for lung nodule 3D segmentation
CN110490917A (en) Three-dimensional rebuilding method and device
Han et al. Automated pathogenesis-based diagnosis of lumbar neural foraminal stenosis via deep multiscale multitask learning
CN110610478B (en) Medical image three-dimensional reconstruction method based on neighborhood topology
CN113450396B (en) Three-dimensional/two-dimensional image registration method and device based on bone characteristics
Shi et al. Multichannel convolutional neural network-based fuzzy active contour model for medical image segmentation
CN117078692B (en) Medical ultrasonic image segmentation method and system based on self-adaptive feature fusion
CN115018809A (en) Target area segmentation and identification method and system of CT image
CN113436237A (en) High-efficient measurement system of complicated curved surface based on gaussian process migration learning
CN107610121B (en) A kind of initial pose setting method of liver statistical shape model
CN115375897A (en) Image processing method, apparatus, device and medium
Ge et al. Grayscale medical image segmentation method based on 2D&3D object detection with deep learning
CN108597589B (en) Model generation method, target detection method and medical imaging system
Chen et al. Learning mesh representations via binary space partitioning tree networks
Lei et al. Learning and meshing from deep implicit surface networks using an efficient implementation of analytic marching
CN112991355B (en) 3D brain lesion segmentation method based on optimal transmission
Jin Rotation prediction based representative view locating framework for 3d object recognition
CN115760874A (en) Multi-scale U-Net medical image segmentation method based on joint spatial domain
Zheng et al. L2-norm scaled transformer for 3d head and neck primary tumors segmentation in pet-ct

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CB03 Change of inventor or designer information
CB03 Change of inventor or designer information

Inventor after: Qiu Chengtong

Inventor after: Li Tiexiang

Inventor after: Wang Han

Inventor after: Tan Zhongheng

Inventor before: Qiu Chengtong

Inventor before: Lin Wenwei

Inventor before: Li Tiexiang

Inventor before: Huang Congming

Inventor before: Yue Meiheng

Inventor before: Wang Han

Inventor before: Tan Zhongheng