CN114119458A - Thyroid medullary cancer ultrasonic image identification method based on clinical priori knowledge guidance - Google Patents
Thyroid medullary cancer ultrasonic image identification method based on clinical priori knowledge guidance Download PDFInfo
- Publication number
- CN114119458A CN114119458A CN202111073772.4A CN202111073772A CN114119458A CN 114119458 A CN114119458 A CN 114119458A CN 202111073772 A CN202111073772 A CN 202111073772A CN 114119458 A CN114119458 A CN 114119458A
- Authority
- CN
- China
- Prior art keywords
- thyroid
- network
- segmentation
- image
- segmentation network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 210000001685 thyroid gland Anatomy 0.000 title claims abstract description 22
- 206010028980 Neoplasm Diseases 0.000 title description 2
- 201000011510 cancer Diseases 0.000 title description 2
- 230000011218 segmentation Effects 0.000 claims abstract description 95
- 208000023356 medullary thyroid gland carcinoma Diseases 0.000 claims abstract description 42
- 208000009453 Thyroid Nodule Diseases 0.000 claims abstract description 39
- 238000002604 ultrasonography Methods 0.000 claims abstract description 31
- 208000037196 Medullary thyroid carcinoma Diseases 0.000 claims abstract description 30
- 208000013818 thyroid gland medullary carcinoma Diseases 0.000 claims abstract description 30
- 208000024770 Thyroid neoplasm Diseases 0.000 claims abstract description 28
- 206010033701 Papillary thyroid cancer Diseases 0.000 claims abstract description 7
- 208000030045 thyroid gland papillary carcinoma Diseases 0.000 claims abstract description 7
- 239000000284 extract Substances 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 9
- 239000007787 solid Substances 0.000 claims description 8
- 238000013145 classification model Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 5
- 208000009018 Medullary thyroid cancer Diseases 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 206010054107 Nodule Diseases 0.000 abstract description 8
- 230000002308 calcification Effects 0.000 abstract description 8
- 230000003211 malignant effect Effects 0.000 abstract description 6
- 239000013598 vector Substances 0.000 description 7
- 238000013527 convolutional neural network Methods 0.000 description 4
- 238000003745 diagnosis Methods 0.000 description 3
- 208000033781 Thyroid carcinoma Diseases 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 201000002510 thyroid cancer Diseases 0.000 description 2
- 208000013077 thyroid gland carcinoma Diseases 0.000 description 2
- 208000007054 Medullary Carcinoma Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2431—Multiple classes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10132—Ultrasound image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30101—Blood vessel; Artery; Vein; Vascular
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
- Image Processing (AREA)
Abstract
The invention relates to a thyroid medullary carcinoma ultrasonic image identification method based on clinical priori knowledge guidance. The method comprises the steps of firstly designing a cascade segmentation network to obtain a thyroid nodule area, then designing a multi-branch classification network, and adding important thyroid ultrasound image characteristics such as solidity and calcification as priori knowledge to assist network classification according to the guidance of clinical knowledge of a radiologist. The method can distinguish the benign and malignant nodules in the image, and can accurately distinguish the malignant nodules into medullary thyroid carcinoma and papillary thyroid carcinoma.
Description
Technical Field
The invention belongs to the field of image segmentation and classification, and particularly relates to a thyroid medullary carcinoma ultrasonic image identification method based on clinical priori knowledge guidance.
Background
The existing thyroid ultrasound image auxiliary diagnosis systems are mostly used for classifying benign and malignant nodules, particularly thyroid papillary carcinoma. However, medullary thyroid carcinoma in an ultrasound image has both characteristics of benign and malignant nodules (as shown in fig. 1), and it is difficult to accurately diagnose medullary thyroid carcinoma from the image. Meanwhile, due to the low imaging quality of the ultrasonic image and the existence of artificial artifacts, in order to obtain a better model, the CNN usually needs to train a large number of pictures, and due to the characteristic of few thyroid medullary carcinoma case samples, it is relatively difficult to obtain so many pictures, which also limits the application of the CNN in the identification of the thyroid medullary carcinoma ultrasonic image.
Thyroid nodule classification techniques include direct classification and location-first classification. The direct classification method assists in diagnosing thyroid ultrasound images on the premise that the position of thyroid nodules is unknown, for example: article [1] was the first work to fine-grained classify thyroid nodules, including the identification of medullary thyroid carcinomas. However, the sample size of medullary thyroid cancer is small, so that the generalization of the trained model is poor, and in supervised classification learning, a single class label provides far less supervised information than a pixel-level semantic label. The nodule position can be obtained and the nodule classification can be carried out simultaneously by a method of positioning first and then classifying. For example: the document [2] proposes a method, in which the positions of nodules are detected by a scale pyramid, and then the detected regions are sent to a multi-branch classification network for further classification. The network also considers some clinical judgment features, such as the context features of surrounding tissues and the edge features of tissue edges, so as to improve the classification accuracy. However, the network only classifies the benign and malignant states, and the general characteristics are used, and the key ultrasonic image discrimination characteristics are not considered (as shown in fig. 2 and table 1).
Table 1 five discriminant ultrasound features of thyroid nodules.
Note that "+" indicates that the feature is more likely to appear in a particular type of nodule, and "-" indicates that the likelihood of the feature appearing in a particular type of nodule is uncertain.
The existing medullary thyroid cancer identification scheme has the following defects:
1. medullary thyroid carcinoma accounts for only 2-3% of all thyroid carcinomas, the data is less, medullary thyroid carcinoma in an ultrasonic image has the characteristics of benign and malignant nodules, and medullary thyroid carcinoma is difficult to accurately diagnose from the image, so that an ultrasonic image dataset of medullary thyroid carcinoma is difficult to construct, and the conventional auxiliary thyroid carcinoma diagnostic system does not have the identification capability of medullary thyroid carcinoma.
2. The existing classification method only considers some common features of the image when realizing thyroid nodule classification, and does not consider important distinguishing features (such as calcification and solid part features) in the thyroid ultrasound image in a targeted manner.
Reference documents:
【1】Li S, Guo Y, Song W, et al. Fine-grained thyroid nodule classification via multi-semantic attention network[C]//2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 2019: 826-833.
【2】Liu T, Guo Q, Lian C, et al. Automated detection and classification of thyroid nodules in ultrasound images using clinical-knowledge-guided convolutional neural networks[J]. Medical image analysis, 2019, 58: 101555.。
disclosure of Invention
The invention aims to provide a clinical priori knowledge guidance-based medullary thyroid carcinoma ultrasonic image identification method, and provides a two-stage method combining segmentation and classification.
In order to achieve the purpose, the technical scheme of the invention is as follows: a clinical priori knowledge guidance-based medullary thyroid carcinoma ultrasonic image identification method is provided, namely a clinical priori knowledge guidance-based two-stage method is provided for automatic identification of medullary thyroid carcinoma in an ultrasonic image, and benign papillary thyroid carcinoma and medullary thyroid carcinoma in the image can be classified. The method is concretely realized as follows:
(1) preprocessing an original thyroid ultrasound image, automatically cutting a background by using a threshold segmentation and maximum connected domain method, and obtaining a training sample, wherein the training sample comprises the thyroid original ultrasound image, a binarization mask label of a real thyroid nodule, cystic and calcified part, and a label corresponding to each ultrasound image;
(2) building a cascading segmentation network, outputting to obtain a nodule position through a strategy from coarse to fine, namely obtaining a roughly coarse nodule position and probability by a first front-end segmentation network, sending the roughly coarse nodule position and probability to a second rear-end segmentation network, and finely tuning a coarse nodule segmentation result by the rear-end network on the basis to obtain a more fine segmentation result;
(3) establishing a multi-classification network guided by thyroid nodule priori feature knowledge, and positioning a calcified part and a solid part in a thyroid ultrasonic picture by training two independent segmentation networks; when the classification network is trained, the network extracts the global features of thyroid nodules, also extracts the features corresponding to the calcified part and the real part, and fuses with the global features and sends the features into a classifier for classification;
(4) in the training stage, a segmentation and classification network is respectively trained by using a supervision training method based on the true value and the predicted value of the preprocessed data in the step (1) and by using a preset loss function as supervision; carrying out image enhancement in the training process to obtain a final segmentation classification model; the purpose of classifying the medullary thyroid carcinoma in the ultrasonic image is realized through the segmentation classification model.
In one embodiment of the present invention, in step (1), the original thyroid ultrasound image is first segmented by a threshold to obtain a binary image after background clipping, that is, a pixel value of the grayscale image lower than the threshold is set to 0, otherwise, the pixel value is set to 255; then, all connected components on the image are calculated by using a maximum connected domain method, and finally, a bounding box of the maximum connected components is selected to cut the target image.
In one embodiment of the invention, in the step (2), a cascaded segmentation network, namely a coarse-to-fine segmentation network C2F-SegNet, is composed of a first front-end segmentation network and a second rear-end segmentation network which are connected in a cascaded manner, the first front-end segmentation network is a coarse segmentation network and directly outputs a pixel-level probability map, and the output probability map represents the confidence of each pixel belonging to a thyroid nodule region, and the range is from 0 to 1; the probability map is input into a second later back-end segmentation network, namely a fine segmentation network, and the edge region of the fine segmentation result is learned.
In one embodiment of the invention, the rough cut network and the fine cut network use similar UNet + + architecture; the rough segmentation network consists of a series of symmetrical dense encoders and decoders, and the encoders and decoders on corresponding layers combine low-level detail information and high-level semantic information from feature maps of different scales through jump connection; the convolution block uses a convolution kernel of size 3 x 3 with a step size of 1; the input of the rough segmentation network is 1 input channel, and the output channels of each layer are respectively 32, 64, 128 and 256; the maximum pooling layer is used for downsampling a feature map derived from the convolution block, and the scale factor is 2; the decoder, consisting of the DeConv2d block, has a similar structure to the encoder; the coarse segmentation network outputs a preliminary segmentation of the thyroid nodules, represented by probability maps from the softmax operation, given the last connected feature map; after obtaining the coarse probability map, obtaining the confidence coefficient of each pixel belonging to the target nodule region, and then multiplying the probability map with the original gray level image to reserve the pixel value of the high confidence coefficient region and attenuate the value of the low confidence coefficient region; the multiplication image and the original image are connected and then input into a fine segmentation network, and the fine segmentation network and the coarse segmentation network have the same structure except that the input channel is 2.
In an embodiment of the present invention, in step (4), the preset loss functions are cross entropy loss and Dice loss functions.
Compared with the prior art, the invention has the following beneficial effects:
1. only one of the existing classification methods realizes classification and judgment of medullary thyroid carcinoma, but only 21 medullary carcinoma ultrasonic images are in a used data set, so that the algorithm cannot be fully verified to have good generalization, and in supervised classification learning, supervision information provided by a single class label is far less than a pixel-level semantic label.
The invention provides a two-stage network which is divided and classified firstly to identify medullary thyroid carcinoma, a network model is trained and tested on a larger medullary thyroid carcinoma data set, and the generalization is better.
2. The existing classification method only considers some common features of the image when realizing thyroid nodule classification, and does not consider specific important discriminant features (such as calcification and cystic part features) of the thyroid ultrasound image in a targeted manner.
According to the invention, key ultrasonic features (concerned by radiologists) are combined with a deep convolutional network during classification, a multi-branch classification network is designed, thyroid nodule regions and global features are extracted through a general extraction feature network, meanwhile, a special feature detection extraction network is designed according to medical priori knowledge, corresponding features such as real and calcification features are extracted, and through feature fusion, the priori features can be better sent to a classifier together to serve as the basis of classification judgment, so that the accuracy is improved.
Drawings
Figure 1 compares medullary thyroid carcinoma to benign nodules (left), medullary thyroid carcinoma to papillary thyroid carcinoma (right) similar ultrasound features.
Fig. 2 shows five discriminant features of the thyroid ultrasound image, which are (a) composition, (b) echo, (c) shape, (d) edge, and (e) calcification.
Fig. 3 shows the original ultrasound image (left) and the cropped image (right) with the additional information area.
FIG. 4 is a block diagram of the method of the present invention.
Detailed Description
The technical scheme of the invention is specifically explained below with reference to the accompanying drawings.
As shown in fig. 4, the invention provides a clinical prior knowledge-guided ultrasound image identification method for medullary thyroid carcinoma, which is used for automatically identifying medullary thyroid carcinoma in an ultrasound image and can classify benign thyroid carcinoma, papillary thyroid carcinoma and medullary thyroid carcinoma in the image. The method is concretely realized as follows:
(1) preprocessing an original thyroid ultrasound image, automatically cutting a background by using a threshold segmentation and maximum connected domain method, and obtaining a training sample, wherein the training sample comprises the thyroid original ultrasound image, a binarization mask label of a real thyroid nodule, cystic and calcified part, and a label corresponding to each ultrasound image;
(2) building a cascading segmentation network, outputting to obtain a nodule position through a strategy from coarse to fine, namely obtaining a roughly coarse nodule position and probability by a first front-end segmentation network, sending the roughly coarse nodule position and probability to a second rear-end segmentation network, and finely tuning a coarse nodule segmentation result by the rear-end network on the basis to obtain a more fine segmentation result;
(3) establishing a multi-classification network guided by thyroid nodule priori feature knowledge, and positioning a calcified part and a solid part in a thyroid ultrasonic picture by training two independent segmentation networks; when the classification network is trained, the network extracts the global features of thyroid nodules, also extracts the features corresponding to the calcified part and the real part, and fuses with the global features and sends the features into a classifier for classification;
(4) in the training stage, a segmentation and classification network is respectively trained by using a supervision training method based on the true value and the predicted value of the preprocessed data in the step (1) and by using a preset loss function as supervision; carrying out image enhancement in the training process to obtain a final segmentation classification model; the purpose of classifying the medullary thyroid carcinoma in the ultrasonic image is realized through the segmentation classification model.
The following is a specific implementation process of the present invention.
The invention relates to a thyroid medullary carcinoma ultrasonic image identification method based on clinical prior knowledge guidance, which comprises the following overall implementation processes:
(1) the thyroid ultrasound image is preprocessed, and a background (containing patient information, manual marks and the like) is automatically cut by using a threshold segmentation method and a maximum connected domain method, so that a training sample is obtained, wherein the training sample comprises a thyroid original ultrasound image, a binary mask label of a real thyroid nodule, a cystic part and a calcified part, and a label (medullary thyroid carcinoma, papillary thyroid carcinoma and benign nodule) corresponding to each ultrasound image.
(2) And (3) building a cascading segmentation network, outputting to obtain a nodule position through a strategy from coarse to fine, namely, obtaining a roughly coarse nodule position and probability by a first front-end segmentation network, and sending the roughly coarse nodule position and probability to a second rear-end segmentation network, wherein on the basis, the coarse nodule segmentation result is finely adjusted by the rear-end network, so that a more fine segmentation result is obtained.
(3) A thyroid nodule prior feature knowledge guided multi-classification network is built, and calcified parts and solid parts (cystic part area + solid part area = thyroid nodule part) in a thyroid ultrasound image are positioned by training two independent segmentation networks. When the classification network is trained, the network extracts the global features of thyroid nodules, extracts the features corresponding to the calcified parts and the real parts, fuses with the global features and sends the features and the features to a classifier for classification.
(4) And (3) in the training stage, respectively training the segmentation and classification networks by using a designed loss function as supervision based on the true value and the predicted value of the data preprocessed in the stage (1) through a supervision training method. And an image enhancement technology is added in the training process, so that the obtained model is more robust (random cutting, turnover change and the like), and the final segmentation classification model is obtained. The purpose of classifying medullary thyroid carcinoma in the ultrasonic image is realized through the model.
The following describes the steps in detail:
(1) image pre-processing
The original ultrasound image has some areas of additional information that are not useful, including hospital information and detailed parameters of the device, preventing further training of the network. A threshold value is used to obtain a binary image after cropping the background, taking into account that the background area has lower pixel values. Specifically, the present invention sets the pixel value of the grayscale image below the threshold to 0, otherwise to 255. The invention then uses a connected component analysis algorithm to compute all connected components on the image. Finally, the invention selects the bounding box of the largest connected component to crop the target image. Thus, the present invention may automatically delete potential background portions for better identification. As shown in fig. 2, the clipped image (right) is obtained by removing the unnecessary area from the original image (left).
(2) Coarse-to-fine split network
The invention adopts a cascading segmentation strategy to obtain the segmentation result and improve the segmentation quality. The cascaded segmentation may gradually improve the accuracy of the pixel-level prediction. Finally, the accurate segmentation of the nodules excludes surrounding tissue, thereby facilitating the next classification of the classification network. Therefore, the present invention proposes a coarse-to-fine segmentation network (C2F-SegNet) to better locate thyroid nodules. The C2F-SegNet module of the present invention is composed of two split networks in a cascaded fashion. Firstly, the former network is a rough segmentation network, and a pixel-level probability map is directly output. The output probability map represents the confidence of each pixel belonging to the thyroid nodule region, and the range is from 0 to 1. Such a probability map is input into the latter subdivided network for subsequent fine-tuning. Given a coarse pixel-by-pixel probability map, the subdivision network learns the edge regions of the fine-tuning segmentation result. The coarsely partitioned network and the finely partitioned network use a similar UNet + + architecture. The coarse segmentation network consists of a series of symmetric dense encoders and decoders, which combine low-level detail information from different scale feature maps with high-level semantic information through a skip connection. The convolution block (i.e., Conv2 d) uses a convolution kernel of size 3 x 3 with a step size of 1. The input of the coarse division network is 1 input channel, and the output channels of each layer are 32, 64, 128 and 256 respectively. The maximum pooling layer is used to downsample the feature map derived from the convolution block with a scale factor of 2. The decoder, which consists of the DeConv2d block, has a similar structure to the encoder. The coarse segmentation network outputs a preliminary segmentation of the thyroid nodules, represented by probability maps from the softmax operation, given the last connected feature map. And obtaining the confidence of each pixel belonging to the target nodule region after obtaining the coarse probability map. The probability map is then multiplied with the original grayscale image to retain the pixel values of the high confidence regions and attenuate the values of the low confidence regions. The invention also connects the multiplication image and the original image, and then inputs the multiplication image and the original image into a following fine segmentation network, wherein the fine segmentation network and the rough segmentation network have the same structure except that the input channel is 2. The multiplicative image provides a coarse localization of the thyroid nodule, the underlying clue thyroid nodule region. The subdivision network learns the optimized segmentation, especially considering the edge part around the coarse positioning area. This cascaded structure limits the search space, resulting in better segmentation results.
(3) Thyroid nodule classification based on priori knowledge guidance
The classification of thyroid nodules relies on a preprocessing step for segmentation. Accurate segmentation results provide constraints for subsequent classification tasks, providing them with valuable a priori regions. Different types of ultrasound features, such as solid features and calcification features, can be used for the aided diagnosis of thyroid nodule types. Thus, the present invention further explores a priori knowledge and integrates it into the network of the present invention for better classification. To further improve diagnostic performance, the present invention contemplates learning specific features under the direction of a priori knowledge.
Therefore, the present invention proposes a priori knowledge guided CNN network to fuse more specific regional details, which fuses multiple features to obtain better diagnosis, thereby improving the overall performance of thyroid nodule identification. In order to better aggregate information from a priori knowledge, the invention trains a specific feature segmentation network and then fuses the features for classification. Firstly, considering the advantages of the fine labeling data set, the invention trains a plurality of segmentation networks with specific characteristics aiming at different prior characteristics. The split network uses UNet + + structure, and different specific features split the network to have the same architecture but different parameters.
(4) Training network
In order to effectively train the segmented network of the present invention, the present invention uses cross-entropy loss and Dice loss functions to supervise the segmented network training. The invention only considers calcification and substantiality characteristics; thus, the present invention has two segmented networks to unambiguously segment these types of diagnostic regions (real and calcified).
Secondly, after specific feature regions (real regions and calcified regions) are extracted, the invention uses the modified ResNet-34 as the main network of the invention, and the invention extracts the feature vector of the original ResNet-34 after the maximum pool layer, wherein the feature vector is 512-dimensional output features. The thyroid nodule global feature vector (512 dimensionality) is extracted through a convolutional network, meanwhile, the calcification (512 dimensionality) and the feature vector (512 dimensionality) of the solid tissue are extracted, the three feature vectors are fused, the obtained 1536 dimensionality feature vector is sent to a classifier for classification, and the thyroid nodule global feature vector can be better diagnosed by combining feature knowledge of a priori region. This approach helps the present invention to better diagnose thyroid nodules and improve the interpretability of the network.
The above are preferred embodiments of the present invention, and all changes made according to the technical scheme of the present invention that produce functional effects do not exceed the scope of the technical scheme of the present invention belong to the protection scope of the present invention.
Claims (6)
1. A thyroid medullary carcinoma ultrasonic image identification method based on clinical priori knowledge guidance is characterized in that a two-stage method based on clinical priori knowledge guidance is provided, is used for automatic identification of thyroid medullary carcinoma in an ultrasonic image, and can classify benign thyroid papillary carcinoma, thyroid medullary carcinoma in the image.
2. The method for identifying medullary thyroid carcinoma ultrasonic images based on clinical prior knowledge guidance according to claim 1, is implemented as follows:
(1) preprocessing an original thyroid ultrasound image, automatically cutting a background by using a threshold segmentation and maximum connected domain method, and obtaining a training sample, wherein the training sample comprises the thyroid original ultrasound image, a binarization mask label of a real thyroid nodule, cystic and calcified part, and a label corresponding to each ultrasound image;
(2) building a cascading segmentation network, outputting to obtain a nodule position through a strategy from coarse to fine, namely obtaining a roughly coarse nodule position and probability by a first front-end segmentation network, sending the roughly coarse nodule position and probability to a second rear-end segmentation network, and finely tuning a coarse nodule segmentation result by the rear-end network on the basis to obtain a more fine segmentation result;
(3) establishing a multi-classification network guided by thyroid nodule priori feature knowledge, and positioning a calcified part and a solid part in a thyroid ultrasonic picture by training two independent segmentation networks; when the classification network is trained, the network extracts the global features of thyroid nodules, also extracts the features corresponding to the calcified part and the real part, and fuses with the global features and sends the features into a classifier for classification;
(4) in the training stage, a segmentation and classification network is respectively trained by using a supervision training method based on the real value and the predicted value of the preprocessed data in the step (1) and by using a preset loss function as supervision; carrying out image enhancement in the training process to obtain a final segmentation classification model; the purpose of classifying the medullary thyroid carcinoma in the ultrasonic image is realized through the segmentation classification model.
3. The method for identifying medullary thyroid carcinoma ultrasonic image based on clinical prior knowledge guidance of claim 2, wherein in step (1), the original thyroid ultrasound image is first segmented by a threshold to obtain a binary image after background cropping, i.e. the pixel value of the gray image below the threshold is set to 0, otherwise, the pixel value is set to 255; then, all connected components on the image are calculated by using a maximum connected domain method, and finally, a bounding box of the maximum connected components is selected to cut the target image.
4. The clinical priori knowledge guidance-based medullary thyroid cancer ultrasound image identification method according to claim 2, wherein in the step (2), a cascaded segmentation network, namely a coarse-to-fine segmentation network C2F-SegNet, is composed of a first front-end segmentation network and a second rear-end segmentation network which are connected in a cascaded manner, wherein the first front-end segmentation network is a coarse segmentation network, a pixel-level probability map is directly output, and the output probability map represents the confidence of each pixel belonging to a thyroid nodule region, and ranges from 0 to 1; the probability map is input into a second later back-end segmentation network, namely a fine segmentation network, and the edge region of the fine segmentation result is learned.
5. The method for identifying medullary thyroid carcinoma ultrasonic images based on clinical prior knowledge guidance of claim 4, wherein the rough segmentation network and the fine segmentation network use similar UNet + + architecture; the rough segmentation network consists of a series of symmetrical dense encoders and decoders, and the encoders and decoders on corresponding layers combine low-level detail information and high-level semantic information from feature maps of different scales through jump connection; the convolution block uses a convolution kernel of size 3 x 3 with a step size of 1; the input of the rough segmentation network is 1 input channel, and the output channels of each layer are respectively 32, 64, 128 and 256; the maximum pooling layer is used for downsampling a feature map derived from the convolution block, and the scale factor is 2; the decoder, consisting of the DeConv2d block, has a similar structure to the encoder; the coarse segmentation network outputs a preliminary segmentation of the thyroid nodules, represented by probability maps from the softmax operation, given the last connected feature map; after obtaining the coarse probability map, obtaining the confidence coefficient of each pixel belonging to the target nodule region, and then multiplying the probability map with the original gray level image to reserve the pixel value of the high confidence coefficient region and attenuate the value of the low confidence coefficient region; the multiplication image and the original image are connected and then input into a fine segmentation network, and the fine segmentation network and the coarse segmentation network have the same structure except that the input channel is 2.
6. The method for identifying medullary thyroid carcinoma ultrasonic images based on clinical prior knowledge guidance of claim 2, wherein in the step (4), the preset loss functions are cross entropy loss and Dice loss functions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111073772.4A CN114119458A (en) | 2021-09-14 | 2021-09-14 | Thyroid medullary cancer ultrasonic image identification method based on clinical priori knowledge guidance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111073772.4A CN114119458A (en) | 2021-09-14 | 2021-09-14 | Thyroid medullary cancer ultrasonic image identification method based on clinical priori knowledge guidance |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114119458A true CN114119458A (en) | 2022-03-01 |
Family
ID=80441356
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111073772.4A Pending CN114119458A (en) | 2021-09-14 | 2021-09-14 | Thyroid medullary cancer ultrasonic image identification method based on clinical priori knowledge guidance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114119458A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114881925A (en) * | 2022-03-30 | 2022-08-09 | 什维新智医疗科技(上海)有限公司 | Good malignant decision maker of node based on elasticity ultrasonic image |
| CN114926486A (en) * | 2022-05-12 | 2022-08-19 | 哈尔滨工业大学人工智能研究院有限公司 | Thyroid ultrasound image intelligent segmentation method based on multi-level improvement |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109087327A (en) * | 2018-07-13 | 2018-12-25 | 天津大学 | A kind of thyroid nodule ultrasonic image division method cascading full convolutional neural networks |
| CN111079862A (en) * | 2019-12-31 | 2020-04-28 | 西安电子科技大学 | Thyroid papillary carcinoma pathological image classification method based on deep learning |
| CN111243042A (en) * | 2020-02-28 | 2020-06-05 | 浙江德尚韵兴医疗科技有限公司 | Ultrasonic thyroid nodule benign and malignant characteristic visualization method based on deep learning |
| US20200405148A1 (en) * | 2019-06-27 | 2020-12-31 | Bao Tran | Medical analysis system |
-
2021
- 2021-09-14 CN CN202111073772.4A patent/CN114119458A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109087327A (en) * | 2018-07-13 | 2018-12-25 | 天津大学 | A kind of thyroid nodule ultrasonic image division method cascading full convolutional neural networks |
| US20200405148A1 (en) * | 2019-06-27 | 2020-12-31 | Bao Tran | Medical analysis system |
| CN111079862A (en) * | 2019-12-31 | 2020-04-28 | 西安电子科技大学 | Thyroid papillary carcinoma pathological image classification method based on deep learning |
| CN111243042A (en) * | 2020-02-28 | 2020-06-05 | 浙江德尚韵兴医疗科技有限公司 | Ultrasonic thyroid nodule benign and malignant characteristic visualization method based on deep learning |
Non-Patent Citations (2)
| Title |
|---|
| 尉智辉: "基于深度学习的甲状腺结节超声图像分割算法研究", 中国优秀硕士论文电子期刊 医药卫生科技辑, 15 June 2020 (2020-06-15) * |
| 魏洪成等: "基于深度卷积神经网络的超声心动图轮廓检测", 微型机与应用, vol. 36, no. 14, 31 December 2017 (2017-12-31), pages 57 - 60 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114881925A (en) * | 2022-03-30 | 2022-08-09 | 什维新智医疗科技(上海)有限公司 | Good malignant decision maker of node based on elasticity ultrasonic image |
| CN114926486A (en) * | 2022-05-12 | 2022-08-19 | 哈尔滨工业大学人工智能研究院有限公司 | Thyroid ultrasound image intelligent segmentation method based on multi-level improvement |
| CN114926486B (en) * | 2022-05-12 | 2023-02-07 | 哈尔滨工业大学人工智能研究院有限公司 | Thyroid ultrasound image intelligent segmentation method based on multi-level improvement |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Tao et al. | Deep learning for unsupervised anomaly localization in industrial images: A survey | |
| Al-Ghaili et al. | Vertical-edge-based car-license-plate detection method | |
| Ladický et al. | What, where and how many? combining object detectors and crfs | |
| Zhang et al. | Image segmentation based on 2D Otsu method with histogram analysis | |
| Nandi et al. | Traffic sign detection based on color segmentation of obscure image candidates: a comprehensive study | |
| CN110580699A (en) | Pathological image cell nucleus detection method based on improved fast RCNN algorithm | |
| Wang et al. | Traffic sign detection using a cascade method with fast feature extraction and saliency test | |
| CN114119458A (en) | Thyroid medullary cancer ultrasonic image identification method based on clinical priori knowledge guidance | |
| CN111145209A (en) | Medical image segmentation method, device, equipment and storage medium | |
| CN109165658B (en) | Strong negative sample underwater target detection method based on fast-RCNN | |
| CN111784711A (en) | Lung pathology image classification and segmentation method based on deep learning | |
| CN111126401A (en) | License plate character recognition method based on context information | |
| Li et al. | Multi-view vehicle detection based on fusion part model with active learning | |
| Barodi et al. | An enhanced artificial intelligence-based approach applied to vehicular traffic signs detection and road safety enhancement | |
| CN111595737B (en) | Optical holographic particle field particle point detection method based on three-dimensional branch network | |
| Sarker et al. | A fast and robust license plate detection algorithm based on two-stage cascade adaboost | |
| CN115775226B (en) | Medical image classification method based on transducer | |
| CN114359493B (en) | Method and system for generating three-dimensional semantic map for unmanned ship | |
| CN114972300A (en) | Material image segmentation and identification method based on computer vision and deep learning | |
| Saber et al. | Integration of color, shape, and texture for image annotation and retrieval | |
| CN113516022B (en) | Fine-grained classification system for cervical cells | |
| Tan et al. | Coarse-to-Fine Context Aggregation Network for Vehicle Make and Model Recognition | |
| Westwańska et al. | Machine Learning in Object Recognition and Segmentation by Certain Class Neural Networks | |
| Saxena et al. | A study on automatic detection and recognition techniques for road signs | |
| CN111950409B (en) | Intelligent identification method and system for road marking line |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |