CN114511581B - Multi-task multi-resolution collaborative esophageal cancer lesion segmentation method and device - Google Patents
Multi-task multi-resolution collaborative esophageal cancer lesion segmentation method and device Download PDFInfo
- Publication number
- CN114511581B CN114511581B CN202210415926.1A CN202210415926A CN114511581B CN 114511581 B CN114511581 B CN 114511581B CN 202210415926 A CN202210415926 A CN 202210415926A CN 114511581 B CN114511581 B CN 114511581B
- Authority
- CN
- China
- Prior art keywords
- lesion
- esophageal cancer
- segmentation
- result
- cancer lesion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4007—Interpolation-based scaling, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10068—Endoscopic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
Abstract
The invention discloses a multitask multiresolution collaborative esophageal cancer lesion segmentation method and a multitask multiresolution collaborative esophageal cancer lesion segmentation device, and mainly solves the problems that the boundary of a lesion region and a normal region cannot be identified with high precision by the existing esophageal cancer lesion region identification method in the prior art, and the judgment of the condition of an illness and the determination of a subsequent treatment scheme are influenced. Firstly, acquiring an esophageal cancer lesion image dataset, and preprocessing the esophageal cancer lesion image dataset; then constructing a multitask multiresolution collaborative esophageal cancer lesion segmentation model; then after training the segmentation model, inputting esophageal cancer lesion image data to be predicted into the trained segmentation model, and outputting a lesion segmentation result and a lesion contour result; combining the lesion contour result with a lesion segmentation result after the lesion contour result is processed by using a conditional random field to form a final segmentation result; the invention fully combines the outline information of the pathological changes, extracts and combines rich characteristics, effectively separates the characteristics of the pathological changes and realizes high-precision segmentation of the pathological change area.
Description
Technical Field
The invention relates to the technical field of esophageal cancer lesion identification, in particular to a multitask and multiresolution collaborative esophageal cancer lesion segmentation method and device.
Background
Esophageal cancer is one of main malignant tumors threatening human health, is a malignant tumor with the 4 th mortality rate and the 6 th morbidity rate in China, and accounts for half of the number of cases all over the world. The survival rate of the early esophageal cancer patient after the operation treatment can reach more than 90 percent in 5 years, and the survival rate of the advanced esophageal cancer patient after the operation treatment in 5 years is less than 20 percent. Therefore, early diagnosis and early treatment of esophageal cancer are key measures for improving the prognosis of patients, the gastroenterology combined biopsy histopathology examination is an important means for diagnosing early esophageal cancer, and effective biopsy of lesions is key for ensuring that the lesions are not missed. Endoscopic submucosal resection is a preferred mode for treating early esophageal cancer, and accurate assessment of lesion range before operation is crucial to endoscopic treatment; therefore, on the basis of diagnosing early esophageal cancer, accurate assessment of lesion boundaries is critical to selection of biopsy sites and selection of treatment regimens for later stages of patients.
At present, the lesion area of the esophageal cancer is identified through artificial identification and Artificial Intelligence (AI); the artificial identification is influenced by a plurality of factors such as the operation level of a doctor, the diagnosis and identification capability of pathological changes and the like, and simultaneously, the judgment of the boundary of the pathological changes of the early esophageal cancer is brought about seriously by the fact that the pathological changes of the early esophageal cancer are difficult to identify the characteristics of the pathological changes of the early esophageal cancer, such as chromatic aberration, morphology, esophageal background mucosa, esophagitis and the like and the influence of factors such as digestive tract mucus and the like.
The artificial intelligence is identified through an AI model, and at present, a BP neural network is used for identifying blood vessels after segmentation training; recognizing after the U-net network training; Nasr-Esfahani and the like divide the contrast image into a blood vessel and a background area, send the blood vessel and the background area into a CNN network, combine global information and local information, and inject a canny edge detector for training so as to obtain a good result; jun et al introduced a new network of T-Net to overcome the limitation that U-Net had only one set of cascade layers between encoding and encoding blocks, and added pooling layers and upper application layers during encoding to make the predicted mask more accurate.
The above methods all achieve certain effects, but researchers do not comprehensively consider the outline information of the lesion and the feature information of different coding layers, but the outline information of the lesion and the feature information of different coding layers are both helpful for segmenting the lesion, so that the existing esophageal cancer lesion area identification method cannot accurately identify the boundary between the lesion area and the normal area, and influences the judgment of the lesion and the determination of the subsequent treatment scheme.
Disclosure of Invention
The invention aims to provide a multitask and multiresolution collaborative esophageal cancer lesion segmentation method and a multitask and multiresolution collaborative esophageal cancer lesion segmentation device, and aims to solve the problem that the boundary of a lesion region and a normal region cannot be identified with high precision by the existing esophageal cancer lesion region identification method, so that the judgment of a disease condition and the determination of a subsequent treatment scheme are influenced.
In order to solve the above problems, the present invention provides the following technical solutions:
a multitask and multi-resolution collaborative esophageal cancer lesion segmentation method comprises the following steps:
s1, acquiring an esophageal cancer lesion image dataset and preprocessing the esophageal cancer lesion image dataset; the esophageal cancer lesion image data is any one or more of esophageal cancer endoscopic images, esophageal cancer endoscopic images and esophageal cancer CT images;
s2, constructing a multitask and multiresolution collaborative esophageal cancer lesion segmentation model, wherein the segmentation model comprises four decoding blocks and four coding blocks; the decoding block and the part corresponding to the coding block adopt jump connection; the segmentation model is fused with an attention module; the decoding block comprises a convolution layer, a batch normalization layer and a MaxPholing layer; the coding block comprises a convolution layer, a batch normalization layer, a Max scaling layer and an upper coding block.
S3, training the segmentation model in the step S2 by adopting the esophageal cancer lesion image data obtained in the step S1;
s4, inputting the esophageal cancer lesion image data to be predicted into the segmentation model trained in the step S3, and outputting a lesion segmentation result and a lesion contour result;
and S5, combining the lesion contour result processed by the conditional random field and the lesion segmentation result in the step S4 to form a final segmentation result.
Firstly, processing esophageal cancer lesion image data, then building a multitask multiresolution collaborative segmentation model, and then training the segmentation model; inputting esophageal cancer lesion image data to be predicted, outputting a lesion segmentation result and a lesion contour result, and combining the lesion contour result with the lesion segmentation result after conditional random field processing to form a final segmentation result; the invention fully combines the outline information of the pathological changes, extracts and combines rich characteristics, effectively separates the characteristics of the pathological changes, finds the segmentation boundary of the pathological change area and the normal area and realizes the high-precision segmentation of the pathological change area.
Furthermore, an esophageal cancer lesion area is marked on the esophageal cancer lesion image to form a label, and then esophageal cancer lesion image data are read and normalized.
Further, the preprocessing process also comprises the step of randomly dividing the esophageal cancer lesion image data into a plurality of subblocks after the data after the normalization processing is subjected to rotation, cutting and noise addition to amplify the esophageal cancer lesion image data, wherein each subblock is marked with a label with a corresponding size.
Further, the training process in step S3 is as follows:
s301, enabling the sub-blocks to enter the sampling features of the coding blocks, and fusing the sampling features of the bottommost coding block with the sampling features of the three coding blocks to obtain a fusion feature;
s302, sending the fusion features to an attention module to output the adjusted features; the attention module comprises a channel attention module and a space attention module;
s303, sending the adjusted features into a decoding block, interpolating each feature point by using a bidirectional bilinear interpolation method in the decoding block, performing convolution, standardization and ReLU operation to finish up-sampling, and then entering an output layer;
s304, the output layer divides the characteristics into two branches, wherein one branch is a lesion segmentation result, and the other branch is a lesion contour result;
s305 and S304 train the segmentation model by using the joint loss function to obtain the trained segmentation model.
Furthermore, the coding block adopts MaxPoint to carry out downsampling, and the decoding block adopts bilinear interpolation.
Further, the activation function of the dice coefficient in the joint loss function is defined as:
wherein, in the step (A),
the result of the prediction is represented by,
a label graph after the label is manually set is shown; the cross entropy loss function in the joint loss function is defined as:
(ii) a Wherein, K represents the number of categories,
indicating that the sign function is 0 or 1, where sample i belongs to the true class k taking 1, otherwise 0,
the probability that pixel point i belongs to category k.
Further, the specific process of step S5 is: definition of
Is a pixel
Probability of whether or not it is a lesion;
is a pixel
The lesion contour probability of (a); processing the lesion contour result by using a conditional random field, and then merging the processed lesion contour result with a lesion segmentation result, wherein the calculation formula is as follows:
wherein CRF is a conditional random field.
Further, the method for segmenting the esophageal cancer lesion by multi-task multi-resolution cooperation further comprises the step of evaluating the segmentation model trained in the step S3, wherein the evaluation process comprises the following steps: and measuring by using a joint loss function, comparing the output final segmentation result with the overlapped part of the real cancerous data labeling part, and judging to reserve the optimal segmentation model according to a preset evaluation index.
A multi-task multi-resolution collaborative esophageal cancer lesion segmentation device comprises a memory: for storing executable instructions; a processor: the executable instructions stored in the memory are executed to realize a multitask and multi-resolution collaborative esophageal cancer lesion segmentation method.
Compared with the prior art, the invention has the following beneficial effects:
(1) firstly, processing esophageal cancer lesion image data, then constructing a multi-task multi-resolution collaborative segmentation model, then training the segmentation model, inputting esophageal cancer lesion image data to be predicted, outputting a lesion segmentation result and a lesion contour result, and combining the lesion contour result with the lesion segmentation result after the lesion contour result is processed by using a conditional random field to form a final segmentation result; the invention fully combines the outline information of the pathological changes, extracts and combines rich characteristics, effectively separates the characteristics of the pathological changes, finds the segmentation boundary of the pathological change area and the normal area and realizes the high-precision segmentation of the pathological change area.
(2) In the invention, different resolution characteristics of the coding blocks are fused and then sent to a convolution attention mechanism to strengthen the relation of useful information of the segmentation labels; and constructing a convolution attention module through a channel attention and space attention module by using the multi-resolution features, and enabling a network to pay attention to the boundary of the lesion by using the contour of the lesion so as to obtain a more accurate segmentation result.
(3) The invention constructs multi-scale information fusion by utilizing each step of down-sampling information in the segmentation module, and can more effectively fuse the information of different decoding blocks, thereby enhancing the maximum feeling, obtaining more accurate characteristics and further improving the segmentation accuracy.
(4) The method can be used for rapidly segmenting the lesion of the esophageal cancer, realizing the segmentation of the lesion data of the esophageal cancer in batches, realizing unattended batch operation, and being rapid in speed, saving manpower and material resources for lesion marking, and providing a more powerful basis for the auxiliary diagnosis of doctors.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without inventive efforts, wherein:
FIG. 1 is a schematic flow chart of training a segmentation model according to the present invention.
FIG. 2 is a flow chart of the test segmentation model of the present invention.
FIG. 3 is a schematic view of the segmentation process of esophageal cancer lesion image data by the segmentation model of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail with reference to fig. 1 to 3, the described embodiments should not be construed as limiting the present invention, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
Example 1
As shown in fig. 1 to 3, a method for segmenting esophageal cancer lesion by multi-task and multi-resolution cooperation comprises the following steps:
s1, acquiring an esophageal cancer lesion image dataset and preprocessing the esophageal cancer lesion image dataset; marking the esophageal cancer lesion area on the esophageal cancer lesion image to form a label, reading esophageal cancer lesion image data, performing normalization processing, amplifying the esophageal cancer lesion image data by adopting rotation, cutting and noise adding, and then randomly dividing the esophageal cancer lesion image data into a plurality of subblocks, wherein each subblock is marked with a label with a corresponding size.
S2, constructing a multitask and multiresolution collaborative esophageal cancer lesion segmentation model, wherein the segmentation model comprises four decoding blocks and four coding blocks; the decoding block and the part corresponding to the coding block adopt jump connection; the segmentation model is fused with an attention module; the coding block adopts MaxPoint to carry on the downsampling, the decoding block adopts the bilinear interpolation;
each coding block layer comprises a convolution layer, a batch normalization layer, a ReLu layer and a MaxPholing layer, the characteristics of the four coding blocks with different resolutions are fused, then the coding blocks are sent to a convolution attention module, then four decoding blocks are carried out, each decoding block layer comprises the convolution layer, the batch normalization layer and the ReLu layer, the last output layer is provided with two branches, one branch is a lesion segmentation result, the other branch is used for calculating a lesion contour result, and the corresponding layers in the U shape are subjected to jump connection to construct a multi-task multi-resolution collaborative segmentation model;
s3, training the segmentation model in the step S2 by adopting the esophageal cancer lesion image data obtained in the step S1;
s4, inputting the esophageal cancer lesion image data to be predicted into the segmentation model trained in the step S3, and outputting a lesion segmentation result and a lesion contour result;
and S5, combining the lesion contour result processed by the conditional random field and the lesion segmentation result in the step S4 to form a final segmentation result.
Example 2
As shown in fig. 1 and fig. 2, the specific process of training the segmentation model by the training set is as follows:
step 1: and sending the sub-block data in the training set into the coding block.
Step 2: and carrying out convolution, batch normalization and maximum pooling on the subblock data in the coding block so as to obtain the maximum receptive field.
And step 3: and at the bottommost coding block, fusing the characteristics of the former coding blocks and the bottom down-sampling characteristic, wherein the characteristics of different characteristic scale sizes are processed into the same size by utilizing convolution, and then the same size is sent to an attention module.
And 4, step 4: the attention module is a module for calculating the attention of one channel by inputting a feature map; the channel attention module is characterized by comprising a channel attention module, a three-layer perceptron, a maximum amplitude vector (Max) sensor, a Global average amplitude vector (Global average) sensor and a Global average amplitude vector (Global average) sensor, wherein the characteristic diagram is input according to the result of the Max Pooling and the Global average Pooling of each channel, the result is respectively sent to the three-layer perceptron, the output result is directly added, and then the result is sent to a Max Pooling activation function to obtain the characteristic diagram of the channel attention module; then multiplying the module according to the channel sequence; then, continuously calculating a space attention module; the spatial attention module is a feature diagram processed by a channel feature diagram mechanism, two feature diagrams of Global Max Pooling and Global Average Pooling are calculated on channel dimensions at first and are spliced together, then a traditional convolution process is used for one time, and finally a feature diagram used by the spatial attention module is obtained by using Sigmoid activation; then, the module is multiplied by the width dimension and the height dimension continuously, and finally, the adjusted feature graph is output.
And 5: and the characteristics acquired from the coding block and the convolution attention module are sent to a decoding block, each point is interpolated by using a bilinear interpolation method in the decoding block, and then convolution, standardization and ReLU operations are performed to finish up-sampling.
And 6: the four coding blocks and the decoding blocks form a U-shaped shape, and the down-sampling edges of the U-shaped shape are symmetrically spliced with the up-sampling edges through a cross-layer connecting structure.
And 7: after each subblock passes through four coding blocks and decoding blocks, the final output layer has two branches, one branchFor the lesion segmentation result, the other branch calculates the lesion contour result, and both branches train the segmentation model by using a joint loss function, wherein the activation function of the dice coefficient is defined as follows:
wherein the content of the first and second substances,
the result of the prediction is represented by,
a label graph after the label is manually set is shown;
another cross entropy loss function, defined as follows:
wherein, K represents the number of categories,
representing a sign function (0 or 1), taking 1 if the sample i belongs to the true class k, otherwise taking 0,
the probability that pixel i belongs to category k.
And 8: on the trained model prediction, the results of the two branches are computed separately, wherein,
is a pixel
Probability of whether or not it is a lesion;
is a pixel
Probability of belonging to a lesion contour. And post-processing the lesion contour result by using a conditional random field, wherein the segmentation result of the last lesion can be obtained by summing the evolvable coronary artery and the segmentation result, and the calculation formula is as follows:
wherein CRF is a conditional random field.
Example 3
As shown in fig. 1 to 3, the operation process of training the segmentation model is as follows: changing the augmented esophageal cancer lesion image data into a plurality of subblocks segmented by a blocking operation, sending the subblocks into a segmentation model in batches, setting the learning rate to be 0.001 by a network, attenuating the learning rate by ten times after 20 learning iterations, initializing a convolution weight by Gaussian distribution, setting a training batch to be 16, setting the learning iteration number to be 200, calculating the gradient and updating the weight by the network training by adopting a BP (back propagation) algorithm, updating a parameter for each batch by the network learning, judging the segmented evaluation result by the segmentation model after each iteration learning, saving the current segmentation model if the current error is smaller than the error of the last iteration, and continuing training until the maximum iteration number is reached.
Example 4
The method for segmenting the esophageal cancer lesion by multi-task and multi-resolution cooperation further comprises the step of evaluating the segmentation model trained in the step S3, wherein the evaluation process comprises the following steps: and measuring by using a joint loss function, comparing the output final segmentation result with the overlapped part of the real cancerous data labeling part, and judging to reserve the optimal segmentation model according to a preset evaluation index.
Example 5
A multi-task multi-resolution collaborative esophageal cancer lesion segmentation device comprises a memory: for storing executable instructions; a processor: the executable instructions stored in the memory are executed to realize a multitask and multi-resolution collaborative esophageal cancer lesion segmentation method.
According to the method, a lesion segmentation result and a lesion contour result are obtained, conditional random field filling is performed on the result of the contour segmentation module, and then the result is merged with lesion segmentation to obtain a final segmentation result, so that multi-scale information can be fully captured, segmentation of the boundary of the esophageal cancer lesion is enhanced, and the result of lesion segmentation is subjected to post-processing by using the conditional random field, so that the effectiveness of esophageal cancer lesion segmentation is improved, and a judgment basis can be relatively accurately provided for doctors.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative and, for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, the functional modules in the embodiments of the present invention may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily think of the changes or substitutions within the technical scope of the present invention, and shall cover the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (8)
1. A multitask and multiresolution collaborative esophageal cancer lesion segmentation method is characterized by comprising the following steps:
s1, acquiring an esophageal cancer lesion image dataset and preprocessing the esophageal cancer lesion image dataset; the preprocessing comprises the steps of randomly dividing esophageal cancer lesion image data into a plurality of sub-blocks;
s2, constructing a multitask and multiresolution collaborative esophageal cancer lesion segmentation model, wherein the segmentation model comprises four decoding blocks and four coding blocks; the decoding block and the part corresponding to the coding block adopt jump connection; the segmentation model is fused with an attention module;
s3, training the segmentation model in the step S2 by adopting the esophageal cancer lesion image data obtained in the step S1;
s4, inputting the esophageal cancer lesion image data to be predicted into the segmentation model trained in the step S3, and outputting a lesion segmentation result and a lesion contour result;
s5, combining the lesion contour result processed by the conditional random field and the lesion segmentation result in the step S4 to form a final segmentation result;
the training process in step S3 is as follows:
s301, enabling the sub-blocks to enter the sampling features of the coding blocks, and fusing the sampling features of the bottommost coding block with the sampling features of the three coding blocks to obtain a fusion feature;
s302, sending the fusion features to an attention module to output the adjusted features; the attention module comprises a channel attention module and a space attention module;
s303, sending the adjusted features into a decoding block, interpolating each feature point by using a bidirectional bilinear interpolation method in the decoding block, performing convolution, standardization and ReLU operation to finish up-sampling, and then entering an output layer;
s304, the output layer divides the characteristics into two branches, wherein one branch is a lesion segmentation result, and the other branch is a lesion contour result;
s305 and S304 train the segmentation model by using the joint loss function to obtain the trained segmentation model.
2. The method for segmenting esophageal cancer lesions with multitask and multiresolution cooperation as claimed in claim 1, wherein the specific process of preprocessing in step S1 is as follows: marking the esophageal cancer lesion area on the esophageal cancer lesion image to form a label, and then reading esophageal cancer lesion image data and carrying out normalization processing.
3. The method according to claim 2, wherein the preprocessing further comprises dividing the esophageal cancer lesion image data into a plurality of sub-blocks randomly after the data after normalization processing is augmented by rotating, clipping, and adding noise, wherein each sub-block is marked with a label with a corresponding size.
4. The method as claimed in claim 3, wherein the coding block is downsampled by Max boosting, and the decoding block is bilinear interpolation.
5. The method for segmenting the esophageal cancer lesion based on the multi-task and multi-resolution cooperation of claim 3, wherein an activation function of a dice coefficient in the joint loss function is defined as:
wherein, in the step (A),
the result of the prediction is represented by,
a label graph after the label is manually set is shown; the cross entropy loss function in the joint loss function is defined as:
(ii) a Wherein, K represents the number of categories,
indicating that the sign function is 0 or 1, where sample i belongs to the true class k taking 1, otherwise 0,
the probability that pixel point i belongs to category k.
6. The method for segmenting the esophageal cancer lesion based on the multi-task and multi-resolution cooperation of claim 5, wherein the specific process of the step S5 is as follows: definition of
Is a pixel
Probability of being a lesion;
is a pixel
The lesion contour probability of (a); processing the lesion contour result by using a conditional random field, and then merging the processed lesion contour result with a lesion segmentation result, wherein the calculation formula is as follows:
wherein CRF is a conditional random field.
7. The method for segmenting the esophageal cancer lesion based on the multi-task and multi-resolution cooperation of claim 6, further comprising the step of evaluating the segmentation model trained in the step S3, wherein the evaluation process comprises: and measuring by using a joint loss function, comparing the output final segmentation result with the overlapped part of the real cancerous data labeling part, and judging to reserve the optimal segmentation model according to a preset evaluation index.
8. A multi-task and multi-resolution cooperative esophageal cancer lesion area segmentation device is characterized by comprising
A memory: for storing executable instructions;
a processor: for executing the executable instructions stored in the memory, a method of multi-tasking multi-resolution collaborative segmentation of esophageal cancer lesions as claimed in any one of claims 1-7 is implemented.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210415926.1A CN114511581B (en) | 2022-04-20 | 2022-04-20 | Multi-task multi-resolution collaborative esophageal cancer lesion segmentation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210415926.1A CN114511581B (en) | 2022-04-20 | 2022-04-20 | Multi-task multi-resolution collaborative esophageal cancer lesion segmentation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114511581A CN114511581A (en) | 2022-05-17 |
| CN114511581B true CN114511581B (en) | 2022-07-08 |
Family
ID=81555330
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210415926.1A Active CN114511581B (en) | 2022-04-20 | 2022-04-20 | Multi-task multi-resolution collaborative esophageal cancer lesion segmentation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114511581B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115082402A (en) * | 2022-06-22 | 2022-09-20 | 济南大学 | Esophageal squamous carcinoma image segmentation method and system based on attention mechanism |
| CN115731243B (en) * | 2022-11-29 | 2024-02-09 | 北京长木谷医疗科技股份有限公司 | Spine image segmentation method and device based on artificial intelligence and attention mechanism |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108734719A (en) * | 2017-04-14 | 2018-11-02 | 浙江工商大学 | Background automatic division method before a kind of lepidopterous insects image based on full convolutional neural networks |
| CN109035263A (en) * | 2018-08-14 | 2018-12-18 | 电子科技大学 | Brain tumor image automatic segmentation method based on convolutional neural networks |
| CN109727253A (en) * | 2018-11-14 | 2019-05-07 | 西安大数据与人工智能研究院 | Divide the aided detection method of Lung neoplasm automatically based on depth convolutional neural networks |
| CN110428432A (en) * | 2019-08-08 | 2019-11-08 | 梅礼晔 | The deep neural network algorithm of colon body of gland Image Automatic Segmentation |
| CN111563906A (en) * | 2020-05-07 | 2020-08-21 | 南开大学 | Knee joint magnetic resonance image automatic segmentation method based on deep convolutional neural network |
| CN111784653A (en) * | 2020-06-28 | 2020-10-16 | 西安电子科技大学 | Multi-scale network MRI pancreas contour positioning method based on shape constraint |
| CN112017191A (en) * | 2020-08-12 | 2020-12-01 | 西北大学 | Method for establishing and segmenting liver pathology image segmentation model based on attention mechanism |
| CN112967287A (en) * | 2021-01-29 | 2021-06-15 | 平安科技（深圳）有限公司 | Gastric cancer focus identification method, device, equipment and storage medium based on image processing |
| CN113256536A (en) * | 2021-06-18 | 2021-08-13 | 之江实验室 | Ultrahigh-dimensional data reconstruction deep learning method based on wavelet analysis |
| CN113362350A (en) * | 2021-07-26 | 2021-09-07 | 海南大学 | Segmentation method and device for cancer medical record image, terminal device and storage medium |
| CN114332462A (en) * | 2021-12-31 | 2022-04-12 | 福州大学 | MRI segmentation method for integrating attention mechanism into cerebral lesion |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11024025B2 (en) * | 2018-03-07 | 2021-06-01 | University Of Virginia Patent Foundation | Automatic quantification of cardiac MRI for hypertrophic cardiomyopathy |
| WO2020028352A1 (en) * | 2018-07-31 | 2020-02-06 | Washington University | Methods and systems for segmenting organs in images using a cnn-based correction network |
| CN111402268B (en) * | 2020-03-16 | 2023-05-23 | 苏州科技大学 | Liver in medical image and focus segmentation method thereof |
| CN112464579B (en) * | 2021-02-02 | 2021-06-01 | 四川大学 | Identification modeling method for searching esophageal cancer lesion area based on evolutionary neural network structure |
-
2022
- 2022-04-20 CN CN202210415926.1A patent/CN114511581B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108734719A (en) * | 2017-04-14 | 2018-11-02 | 浙江工商大学 | Background automatic division method before a kind of lepidopterous insects image based on full convolutional neural networks |
| CN109035263A (en) * | 2018-08-14 | 2018-12-18 | 电子科技大学 | Brain tumor image automatic segmentation method based on convolutional neural networks |
| CN109727253A (en) * | 2018-11-14 | 2019-05-07 | 西安大数据与人工智能研究院 | Divide the aided detection method of Lung neoplasm automatically based on depth convolutional neural networks |
| CN110428432A (en) * | 2019-08-08 | 2019-11-08 | 梅礼晔 | The deep neural network algorithm of colon body of gland Image Automatic Segmentation |
| CN111563906A (en) * | 2020-05-07 | 2020-08-21 | 南开大学 | Knee joint magnetic resonance image automatic segmentation method based on deep convolutional neural network |
| CN111784653A (en) * | 2020-06-28 | 2020-10-16 | 西安电子科技大学 | Multi-scale network MRI pancreas contour positioning method based on shape constraint |
| CN112017191A (en) * | 2020-08-12 | 2020-12-01 | 西北大学 | Method for establishing and segmenting liver pathology image segmentation model based on attention mechanism |
| CN112967287A (en) * | 2021-01-29 | 2021-06-15 | 平安科技（深圳）有限公司 | Gastric cancer focus identification method, device, equipment and storage medium based on image processing |
| CN113256536A (en) * | 2021-06-18 | 2021-08-13 | 之江实验室 | Ultrahigh-dimensional data reconstruction deep learning method based on wavelet analysis |
| CN113362350A (en) * | 2021-07-26 | 2021-09-07 | 海南大学 | Segmentation method and device for cancer medical record image, terminal device and storage medium |
| CN114332462A (en) * | 2021-12-31 | 2022-04-12 | 福州大学 | MRI segmentation method for integrating attention mechanism into cerebral lesion |
Non-Patent Citations (2)
| Title |
|---|
| "Precisely identify the geometry of catalyst particles from S/TEM images via a boundary attention deep learning network";Shuhui Liu等;《Materials Today Communications》;20210818;第1-7页 * |
| "基于全局与局部特征聚合的云检测算法研究";王弢;《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》;20220115;第21,41页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114511581A (en) | 2022-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114511581B (en) | Multi-task multi-resolution collaborative esophageal cancer lesion segmentation method and device | |
| CN112258530A (en) | Neural network-based computer-aided lung nodule automatic segmentation method | |
| CN111476757A (en) | Coronary artery patch data detection method, system, storage medium and terminal | |
| CN112991346B (en) | Training method and training system for learning network for medical image analysis | |
| CN111667478A (en) | Method and system for identifying carotid plaque through CTA-MRA cross-modal prediction | |
| CN113223005B (en) | Thyroid nodule automatic segmentation and grading intelligent system | |
| CN112862830A (en) | Multi-modal image segmentation method, system, terminal and readable storage medium | |
| CN117078692B (en) | Medical ultrasonic image segmentation method and system based on self-adaptive feature fusion | |
| CN113034507A (en) | CCTA image-based coronary artery three-dimensional segmentation method | |
| CN112991363A (en) | Brain tumor image segmentation method and device, electronic equipment and storage medium | |
| CN113112559A (en) | Ultrasonic image segmentation method and device, terminal equipment and storage medium | |
| CN115018863A (en) | Image segmentation method and device based on deep learning | |
| Kong et al. | Data enhancement based on M2-Unet for liver segmentation in Computed Tomography | |
| Bozdağ et al. | Pyramidal nonlocal network for histopathological image of breast lymph node segmentation | |
| CN116645380A (en) | Automatic segmentation method for esophageal cancer CT image tumor area based on two-stage progressive information fusion | |
| Chen et al. | Direction-guided and multi-scale feature screening for fetal head–pubic symphysis segmentation and angle of progression calculation | |
| Mani | Deep learning models for semantic multi-modal medical image segmentation | |
| Pang et al. | Correlation matters: multi-scale fine-grained contextual information extraction for hepatic tumor segmentation | |
| CN113379691A (en) | Breast lesion deep learning segmentation method based on prior guidance | |
| Nour et al. | Skin lesion segmentation based on edge attention vnet with balanced focal tversky loss | |
| Xie et al. | Analysis of the diagnosis model of peripheral non-small-cell lung cancer under computed tomography images | |
| CN113643317B (en) | Coronary artery segmentation method based on depth geometric evolution model | |
| Zuo | CDAE-C: A Fully Convolutional Denoising Auto-Encoder with 2.5 D Convolutional Classifier | |
| CN117522890A (en) | Image segmentation method, device, computer equipment and storage medium | |
| Li et al. | 3D Multiple-Contextual ROI-Attention Network for Efficient and Accurate Volumetric Medical Image Segmentation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |