CN112820399A - Method and device for automatically diagnosing benign and malignant thyroid nodules - Google Patents
Method and device for automatically diagnosing benign and malignant thyroid nodules Download PDFInfo
- Publication number
- CN112820399A CN112820399A CN202110101529.2A CN202110101529A CN112820399A CN 112820399 A CN112820399 A CN 112820399A CN 202110101529 A CN202110101529 A CN 202110101529A CN 112820399 A CN112820399 A CN 112820399A
- Authority
- CN
- China
- Prior art keywords
- soft
- global
- model
- thyroid
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 208000009453 Thyroid Nodule Diseases 0.000 title claims abstract description 48
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000003211 malignant effect Effects 0.000 title claims abstract description 41
- 238000002604 ultrasonography Methods 0.000 claims abstract description 59
- 210000001685 thyroid gland Anatomy 0.000 claims abstract description 50
- 238000012549 training Methods 0.000 claims abstract description 50
- 230000010354 integration Effects 0.000 claims abstract description 29
- 208000024770 Thyroid neoplasm Diseases 0.000 claims abstract description 18
- 230000009467 reduction Effects 0.000 claims abstract description 18
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 13
- 238000007781 pre-processing Methods 0.000 claims description 14
- 239000000203 mixture Substances 0.000 claims description 13
- 239000003814 drug Substances 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 6
- 230000015556 catabolic process Effects 0.000 claims description 4
- 238000006731 degradation reaction Methods 0.000 claims description 4
- 230000003042 antagnostic effect Effects 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 8
- 238000013135 deep learning Methods 0.000 abstract description 6
- 238000005516 engineering process Methods 0.000 abstract description 4
- 201000010099 disease Diseases 0.000 abstract description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 abstract description 2
- 238000012360 testing method Methods 0.000 description 8
- 238000003745 diagnosis Methods 0.000 description 5
- 206010054107 Nodule Diseases 0.000 description 4
- 206010028980 Neoplasm Diseases 0.000 description 3
- 201000011510 cancer Diseases 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000036210 malignancy Effects 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000002405 diagnostic procedure Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
- 201000002510 thyroid cancer Diseases 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10132—Ultrasound image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
Abstract
The invention discloses a method and a device for automatically diagnosing benign and malignant thyroid nodules, wherein the method comprises the following steps: step 1, carrying out noise reduction pretreatment on an original thyroid ultrasound image; step 2, performing data enhancement on the preprocessed thyroid ultrasound image; step 3, putting the data-enhanced thyroid ultrasound images into three convolutional neural network models constructed in advance through ResNet, DenseNet or ResNext for training respectively; and 4, performing model integration on the images trained by different models in the step 3. The method solves the problem of poor automatic classification effect on thyroid nodule benign and malignant diseases, applies a deep learning image automatic identification technology, uses a latest framework to extract image characteristics, completes the thyroid benign and malignant automatic classification task in an end-to-end mode, and simultaneously ensures higher accuracy.
Description
Technical Field
The invention relates to the field of computer artificial intelligent deep learning, in particular to a method and a device for automatically diagnosing benign and malignant thyroid nodules.
Background
Thyroid nodules are very common among the Protov. With the use of thyroid ultrasonography, data have shown that nodules are present in approximately 20% to 76% of adult thyroid glands. Although thyroid nodules are mostly benign, the presence and number of nodules is not as great as the probability of increased lesions. Therefore, to reduce morbidity, it is important to examine the thyroid gland in advance and to screen thyroid nodules for malignancy and well. With the development of the times, thyroid nodule classification technology is continuously evolving and improving. In the early stage, due to the fact that ultrasonic images of thyroid nodules are complex and diverse, benign and malignant diagnosis modes of the thyroid nodules are very diverse, different mechanisms and experts often have different detection means and judgment standards, and therefore thyroid nodule grading is directly complicated and grading difficulty is improved. Once malignant nodules become into a canceration stage, the malignant nodules are also divided into four different stages, thyroid cancer is found at an earlier stage, the treatment effect is better, and the survival period is longer. Therefore, the thyroid gland examination technique is a technique for benefiting mankind, and it is very important to develop a more advanced examination technique to assist diagnosis.
In the time, machine learning and deep learning technologies are hot, the traditional machine learning method needs much human intervention, deep learning mainly realizes feature extraction by a robot instead of a human, and only enough data needs to be provided, a model, namely an expert in a certain sense, can be obtained through training, and each thyroid ultrasound image is diagnosed to give a classification result. For some complex ultrasound images, a single expert may not be able to give a more accurate diagnosis result, and a joint consultation of multiple experts brings a better effect. In view of the mode, a plurality of models are trained, the attention points of each model are different, then the conclusions given by each model are collected and weighted to obtain the final diagnosis conclusion, and the automatic classification effect of the thyroid nodule benign and malignant examination at the present stage is poor, and the accuracy is low.
Disclosure of Invention
The invention aims to provide a method and a device for automatically diagnosing thyroid nodule malignancy and well, which aim to solve the problem of poor automatic classification effect of thyroid nodule malignancy and well inspection and ensure higher accuracy.
The invention provides a method for automatically diagnosing benign and malignant thyroid nodules, which comprises the following steps:
step 1, carrying out noise reduction pretreatment on an original thyroid ultrasound image;
step 2, performing data enhancement on the preprocessed thyroid ultrasound image;
step 3, putting the data-enhanced thyroid ultrasound images into three convolutional neural network models constructed in advance through ResNet, DenseNet or ResNext for training respectively;
and 4, performing model integration on the images trained by different models in the step 3.
Further, the denoising preprocessing of the original thyroid ultrasound image specifically includes:
carrying out graying processing on the image to enable the threshold value to be 0, and obtaining a binarized image; and on the basis of the binary image, performing image opening operation to corrode the noise area of the image to finish noise reduction pretreatment.
Further, the data enhancement of the preprocessed thyroid ultrasound image specifically includes:
and performing data enhancement by using cycle-GAN based on generation of the antagonistic network GAN.
Further, the three convolutional neural network models constructed in advance by means of ResNet or DenseNet or ResNext specifically include: the node comprises a global ultrasonic image Soft model, a global ultrasonic image Soft-mix model and a local nodule ultrasonic image Soft model;
the step of training the thyroid ultrasound image after data enhancement by respectively putting the thyroid ultrasound image into three convolutional neural network models constructed in advance through ResNet or DenseNet or ResNext specifically comprises the following steps:
respectively putting the data-enhanced thyroid ultrasound images into a pre-constructed global ultrasound image Soft model, a global ultrasound image Soft-Mixup model and a local nodule ultrasound image Soft model for training;
and performing optimization adjustment on the trained global ultrasonic image Soft model, global ultrasonic image Soft-mixup model and local ultrasonic image Soft model by using a cosine learning rate degradation mode.
Further, the model integration of the images trained by different models in step 3 specifically includes:
according to formula 1, model integration is performed by using a parameter search method:
score ═ a global-soft-Score + b global-soft-mixup-Score + c local-soft-Score formula 1;
wherein a + b + c is 1, 1> a >0,1> b >0,1> c >0, global-soft-score, global-soft-mix-score and local-soft-score are respectively a global model softening label and a global model softening label sample mixture, and the local model softening label predicts the probability value that the thyroid nodule is malignant;
and traversing all parameter spaces from 0 to 1 in a, b and c at intervals of a preset size, and searching out the maximum value of the area (Auc) under the curve when the medicine is taken.
The invention provides a device for automatically diagnosing benign and malignant thyroid nodules, which comprises:
the preprocessing module is used for carrying out noise reduction preprocessing on the original thyroid ultrasound image;
the data enhancement module is used for enhancing the data of the preprocessed thyroid ultrasound image;
the training module is used for carrying out model training on the thyroid ultrasound image subjected to data enhancement;
and the integration module is used for carrying out model integration on images trained by different models in the training module.
Further, the preprocessing module is specifically configured to: carrying out graying processing on the image to enable the threshold value to be 0, and obtaining a binarized image; on the basis of the binary image, performing image opening operation to corrode the noise area of the image to finish noise reduction pretreatment;
the data enhancement module specifically adopts cycle-GAN based on generation of a countermeasure network (GAN) for data enhancement, and the data enhancement module is composed of 2 generators (G, F) and 2 discriminators (D)X，DY) And (4) forming.
Further, the training module further comprises a global ultrasonic image Soft training module, a global ultrasonic image Soft-mixup training module and a local nodule ultrasonic image Soft training module, and is used for performing different model training on the data-enhanced thyroid ultrasonic image;
the integration method of the integration module is that the model integration is carried out by adopting a parameter searching method according to the formula 1,
score is a global-soft-Score + b global-soft-mixup-Score + c local-soft-Score formula 1
Wherein a + b + c is 1, and 1> a >0,1> b >0,1> c >0
Global-soft-score, Global-soft-mixup-score and local-soft-score are respectively a Global model softening label and a Global model softening label sample mixture, and the local model softening label predicts the probability value that the thyroid nodule is malignant;
and traversing all parameter spaces from 0 to 1 in a, b and c at intervals of a preset size, and searching out the maximum value of the area (Auc) under the curve when the medicine is taken.
The embodiment of the invention also provides equipment for automatically diagnosing benign and malignant thyroid nodules, which is characterized by comprising the following components: a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps of the above method for automatically diagnosing benign and malignant thyroid nodules.
An embodiment of the present invention further provides a computer-readable storage medium, where an implementation program for information transfer is stored on the computer-readable storage medium, and when the program is executed by a processor, the program implements the steps of the above method for automatically diagnosing benign and malignant thyroid nodules.
By adopting the embodiment of the invention, the latest convolutional neural network framework is used, and the thyroid ultrasonic images are automatically classified in an end-to-end mode, so that the automatic classification effect is improved, the thyroid nodules can be identified and classified, and doctors, especially inexperienced doctors, can easily complete screening of thyroid nodules.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flow chart of a method of automatically diagnosing benign and malignant thyroid nodules according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of an apparatus for automatically diagnosing benign and malignant thyroid nodules according to a first embodiment of the present invention;
fig. 3 is a schematic structural diagram of an apparatus for automatically diagnosing benign and malignant thyroid nodules according to a second embodiment of the apparatus of the present invention.
Description of reference numerals:
30: a preprocessing module; 31: a data enhancement module; 32: a training module; 33: a global ultrasonic image Soft training module; 34: a global ultrasonic image Soft-Mixup training module; 35: a local nodule ultrasonic image Soft training module; 36: an integration module; 40: a memory; 42: a processor.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the following embodiments, and it should be understood that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the description of the present invention, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", and the like, indicate orientations and positional relationships based on those shown in the drawings, and are used only for convenience of description and simplicity of description, and do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be considered as limiting the present invention.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", may explicitly or implicitly include one or more of the described features. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise. Furthermore, the terms "mounted," "connected," and "connected" are to be construed broadly and may, for example, be fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
Method embodiment one
According to an embodiment of the present invention, there is provided a method for automatically diagnosing benign and malignant thyroid nodules, fig. 1 is a flowchart of the method for automatically diagnosing benign and malignant thyroid nodules according to an embodiment of the present invention, and as shown in fig. 1, the method for automatically diagnosing benign and malignant thyroid nodules according to an embodiment of the present invention specifically includes:
step 1, carrying out noise reduction pretreatment on an original thyroid ultrasound image; the noise reduction preprocessing specifically comprises: carrying out graying processing on the image to enable the threshold value to be 0, and obtaining a binarized image; and on the basis of the binary image, performing image opening operation to corrode the noise area of the image to finish noise reduction pretreatment.
Step 2, performing data enhancement on the preprocessed thyroid ultrasound image; the data enhancement specifically comprises: rotation, translation, and flipping, which is often used for large datasets due to the inability of some datasets to accept linear level growth. The embodiment of the invention adopts a Cycle-GAN based on a generation countermeasure network (GAN) to perform data enhancement besides a common data enhancement mode, wherein the Cycle-GAN comprises 2 generators (G, F) and 2 discriminators (D)X，DY) The composition is that X can refer to a source domain, namely a training set, Y can refer to a target domain, namely a test set, and a generator G generates the training set X and a generator F
Generating a test set Y, and generating the real test set Y and the generator G by a generator F
Generating a training set X, DX、DYIt is determined X, Y whether the picture is generated or not, and the generator G should cheat D as much as possible by the countermeasures trainingYAnd the generator F is to fool D as much as possibleXWhen the discriminator DX、DYTrue X is generated by generator G, not when it has the ability to distinguish true from false
Will maintain the content of X and the style of Y, generated simultaneously
Consistent with the tag information of X.
Step 3, putting the data-enhanced thyroid ultrasound images into three convolutional neural network models constructed in advance through ResNet or DenseNet or ResNext respectively for training, and specifically comprising the following steps:
respectively putting the data-enhanced thyroid ultrasound images into a pre-constructed global ultrasound image Soft model, a global ultrasound image Soft-Mixup model and a local nodule ultrasound image Soft model for training;
that is to say, the following optimization techniques are adopted to optimize three convolutional neural network models constructed by ResNet or DenseNet or ResNext to obtain a global ultrasonic image Soft model, a global ultrasonic image Soft-mix model and a local nodule ultrasonic image Soft model, and the method specifically includes the following steps:
softening the label: the final result of the classification task in the general deep learning is a N x 1 matrix formed by combining 0 and 1, if only single label classification is considered, only 1 is arranged on the row corresponding to the matrix to represent the predicted classification, and all other classes are 0. This classification is called a Hard Label (Hard Label) and also the conventional test model, which leads to a bias of the model towards giving a non-black or white prediction, which is not desired by the diagnostician in the actual diagnostic process. Therefore, the label softening is added, so that the predicted result is distributed more intensively between softening threshold values, for example, smooth _ value is 0.2, and the benign and malignant distribution is concentrated between 0.2 and 0.8. To achieve this, we input the correct label of the data as such when model training calculates the loss value
First row: benign, second row: malignant disease. The label obtained after smooth _ value is 0.2 is
Thus, a 0.2 weight is assigned to the opposite class when calculating the loss;
sample mixing: suppose a training sample (x)i，yi)、(xj，yj) Where x is the input data and y is the label value, we mix to generate a new sample by
The model trained by the optimization mode only sees the newly mixed generated sample, but not the original sample;
performing optimization adjustment on the trained global ultrasonic image Soft model, global ultrasonic image Soft-mixup model and local ultrasonic image Soft model by using a cosine learning rate degradation mode;
the method comprises the steps of using a field self-adaptive mode to shorten the distance between a test set and a training set, specifically mapping data features of different fields to the same feature space, enhancing training of a target field by using data of other fields, and improving the accuracy of the test set.
And 4, performing model integration on the images trained by different models in the step 3, and specifically comprising the following steps: according to formula 1, model integration is performed by using a parameter search method:
score ═ a global-soft-Score + b global-soft-mixup-Score + c local-soft-Score formula 1;
wherein a + b + c is 1, 1> a >0,1> b >0,1> c >0, global-soft-score, global-soft-mix-score and local-soft-score are respectively a global model softening label and a global model softening label sample mixture, and the local model softening label predicts the probability value that the thyroid nodule is malignant;
and traversing all parameter spaces from 0 to 1 in a, b and c at intervals of a preset size, and searching out the maximum value of the area (Auc) under the curve when the medicine is taken.
By adopting the new data noise reduction mode provided by the embodiment of the invention, the data is subjected to noise reduction before a thyroid nodule benign and malignant classification model is trained to improve the classification performance, the thyroid nodule image is subjected to data enhancement by adopting the traditional image data enhancement and Cycle-GAN, more images with test set styles are generated, and the robustness of the model is improved; using one of the most advanced ResNet, DenseNet and ResNext image recognition frameworks as a backbone network to classify thyroid nodules; the training set and the test set are improved by using a field adaptation method, so that the recognition effect is improved; the parameter space is traversed by using a parameter searching method, an optimal integration mode is searched, the problem that the thyroid nodule benign and malignant automatic classification effect is poor is solved, a deep learning image automatic identification technology is applied, image features are extracted by using a latest framework, the thyroid nodule benign and malignant automatic classification task is completed in an end-to-end mode, and high accuracy is guaranteed. Assisting the doctor in making a diagnosis.
Apparatus embodiment one
According to an embodiment of the present invention, there is provided an apparatus for automatically diagnosing benign and malignant thyroid nodules, fig. 2 is a schematic structural diagram of an apparatus for automatically diagnosing benign and malignant thyroid nodules according to a first embodiment of the apparatus of the present invention, as shown in fig. 2, specifically including:
the preprocessing module 30 is used for performing noise reduction preprocessing on the original thyroid ultrasound image;
the data enhancement module 31 is configured to perform data enhancement on the preprocessed thyroid ultrasound image;
the training module 32 is used for performing model training on the data-enhanced thyroid ultrasound image;
and the integration module 36 is used for performing model integration on images trained by different models in the training module.
Further, the preprocessing module is specifically configured to: carrying out graying processing on the image to enable the threshold value to be 0, and obtaining a binarized image; on the basis of the binary image, performing image opening operation to corrode the noise area of the image to finish noise reduction pretreatment;
the data enhancement module specifically adopts cycle-GAN based on generation of a countermeasure network (GAN) for data enhancement, and the data enhancement module is composed of 2 generators (G, F) and 2 discriminators (D)X，DY) And (4) forming.
Further, the training module further comprises a global ultrasonic image Soft training module 33, a global ultrasonic image Soft-mixup training module 34, and a local nodule ultrasonic image Soft training module 35, which are used for performing different model training on the data-enhanced thyroid ultrasonic image;
the integration method of the integration module is that the model integration is carried out by adopting a parameter searching method according to the formula 1,
score is a global-soft-Score + b global-soft-mixup-Score + c local-soft-Score formula 1
Wherein a + b + c is 1, and 1> a >0,1> b >0,1> c >0
Global-soft-score, Global-soft-mixup-score and local-soft-score are respectively a Global model softening label and a Global model softening label sample mixture, and the local model softening label predicts the probability value that the thyroid nodule is malignant;
and traversing all parameter spaces from 0 to 1 in a, b and c at intervals of a preset size, and searching out the maximum value of the area (Auc) under the curve when the medicine is taken.
The embodiment of the present invention is a system embodiment corresponding to the above method embodiment, and specific operations of each module may be understood with reference to the description of the method embodiment, which is not described herein again.
Device embodiment II
An embodiment of the present invention provides an apparatus for automatically diagnosing benign and malignant thyroid nodules, as shown in fig. 3, including: a memory 40, a processor 42 and a computer program stored on the memory 40 and executable on the processor 42, which computer program, when executed by the processor 42, carries out the following method steps:
step 1, carrying out noise reduction pretreatment on an original thyroid ultrasound image;
step 2, performing data enhancement on the preprocessed thyroid ultrasound image;
step 3, putting the data-enhanced thyroid ultrasound images into three convolutional neural network models constructed in advance through ResNet, DenseNet or ResNext for training respectively;
and 4, performing model integration on the images trained by different models in the step 3.
Further, the denoising preprocessing of the original thyroid ultrasound image specifically includes:
carrying out graying processing on the image to enable the threshold value to be 0, and obtaining a binarized image; and on the basis of the binary image, performing image opening operation to corrode the noise area of the image to finish noise reduction pretreatment.
Further, the data enhancement of the preprocessed thyroid ultrasound image specifically includes:
and performing data enhancement by using cycle-GAN based on generation of the antagonistic network GAN.
Further, the three convolutional neural network models constructed in advance by means of ResNet or DenseNet or ResNext specifically include: the node comprises a global ultrasonic image Soft model, a global ultrasonic image Soft-mix model and a local nodule ultrasonic image Soft model;
the step of training the thyroid ultrasound image after data enhancement by respectively putting the thyroid ultrasound image into three convolutional neural network models constructed in advance through ResNet or DenseNet or ResNext specifically comprises the following steps:
respectively putting the data-enhanced thyroid ultrasound images into a pre-constructed global ultrasound image Soft model, a global ultrasound image Soft-Mixup model and a local nodule ultrasound image Soft model for training;
and performing optimization adjustment on the trained global ultrasonic image Soft model, global ultrasonic image Soft-mixup model and local ultrasonic image Soft model by using a cosine learning rate degradation mode.
Further, the model integration of the images trained by different models in step 3 specifically includes:
according to formula 1, model integration is performed by using a parameter search method:
score ═ a global-soft-Score + b global-soft-mixup-Score + c local-soft-Score formula 1;
wherein a + b + c is 1, 1> a >0,1> b >0,1> c >0, global-soft-score, global-soft-mix-score and local-soft-score are respectively a global model softening label and a global model softening label sample mixture, and the local model softening label predicts the probability value that the thyroid nodule is malignant;
and traversing all parameter spaces from 0 to 1 in a, b and c at intervals of a preset size, and searching out the maximum value of the area (Auc) under the curve when the medicine is taken.
Device embodiment III
An embodiment of the present invention provides a computer-readable storage medium, where an implementation program for information transmission is stored, and when the program is executed by a processor 42, the method steps as described in the first method embodiment are implemented, which are not described herein again.
The computer-readable storage medium of this embodiment includes, but is not limited to: ROM, RAM, magnetic or optical disks, and the like.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for automatically diagnosing benign and malignant thyroid nodules, comprising:
step 1, carrying out noise reduction pretreatment on an original thyroid ultrasound image;
step 2, performing data enhancement on the preprocessed thyroid ultrasound image;
step 3, putting the data-enhanced thyroid ultrasound images into three convolutional neural network models constructed in advance through ResNet, DenseNet or ResNext for training respectively;
and 4, performing model integration on the images trained by different models in the step 3.
2. The method of claim 1, wherein the denoising preprocessing of the raw thyroid ultrasound image specifically comprises:
carrying out graying processing on the image to enable the threshold value to be 0, and obtaining a binarized image; and on the basis of the binary image, performing image opening operation to corrode the noise area of the image to finish noise reduction pretreatment.
3. The method of claim 1, wherein the data enhancement of the preprocessed thyroid ultrasound image comprises:
and performing data enhancement by using cycle-GAN based on generation of the antagonistic network GAN.
4. The method according to claim 1, characterized in that said three convolutional neural network models previously constructed by means of ResNet or DenseNet or ResNext specifically comprise: the node comprises a global ultrasonic image Soft model, a global ultrasonic image Soft-mix model and a local nodule ultrasonic image Soft model;
the step of training the thyroid ultrasound image after data enhancement by respectively putting the thyroid ultrasound image into three convolutional neural network models constructed in advance through ResNet or DenseNet or ResNext specifically comprises the following steps:
respectively putting the data-enhanced thyroid ultrasound images into a pre-constructed global ultrasound image Soft model, a global ultrasound image Soft-Mixup model and a local nodule ultrasound image Soft model for training;
and performing optimization adjustment on the trained global ultrasonic image Soft model, global ultrasonic image Soft-mixup model and local ultrasonic image Soft model by using a cosine learning rate degradation mode.
5. The method according to claim 1, wherein the model integration of the images trained by different models in step 3 specifically comprises:
according to formula 1, model integration is performed by using a parameter search method:
score ═ a global-soft-Score + b global-soft-mixup-Score + c local-soft-Score formula 1;
wherein a + b + c is 1, 1> a >0,1> b >0,1> c >0, global-soft-score, global-soft-mix-score and local-soft-score are respectively a global model softening label and a global model softening label sample mixture, and the local model softening label predicts the probability value that the thyroid nodule is malignant;
and traversing all parameter spaces from 0 to 1 in a, b and c at intervals of a preset size, and searching out the maximum value of the area (Auc) under the curve when the medicine is taken.
6. An apparatus for automatically diagnosing benign and malignant thyroid nodules, comprising:
the preprocessing module is used for carrying out noise reduction preprocessing on the original thyroid ultrasound image;
the data enhancement module is used for enhancing the data of the preprocessed thyroid ultrasound image;
the training module is used for carrying out model training on the thyroid ultrasound image subjected to data enhancement;
and the integration module is used for carrying out model integration on images trained by different models in the training module.
7. The apparatus of claim 6, wherein the preprocessing module is specifically configured to: carrying out graying processing on the image to enable the threshold value to be 0, and obtaining a binarized image; on the basis of the binary image, performing image opening operation to corrode the noise area of the image to finish noise reduction pretreatment;
the data enhancement module specifically adopts cycle-GAN based on generation of a countermeasure network (GAN) for data enhancement, and the data enhancement module is composed of 2 generators (G, F) and 2 discriminators (D)X，DY) And (4) forming.
8. The device of claim 6, wherein the training modules further comprise a global ultrasound image Soft training module, a global ultrasound image Soft-mixup training module, and a local nodule ultrasound image Soft training module, and are used for performing different model training on the data-enhanced thyroid ultrasound images;
the integration method of the integration module is that the model integration is carried out by adopting a parameter searching method according to the formula 1,
score is a global-soft-Score + b global-soft-mixup-Score + c local-soft-Score formula 1
Wherein a + b + c is 1, and 1> a >0,1> b >0,1> c >0
Global-soft-score, Global-soft-mixup-score and local-soft-score are respectively a Global model softening label and a Global model softening label sample mixture, and the local model softening label predicts the probability value that the thyroid nodule is malignant;
and traversing all parameter spaces from 0 to 1 in a, b and c at intervals of a preset size, and searching out the maximum value of the area (Auc) under the curve when the medicine is taken.
9. An apparatus for automatically diagnosing benign and malignant thyroid nodules, comprising: memory, a processor and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps of the method of automatically diagnosing benign and malignant thyroid nodules according to any one of claims 1 to 5.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon an information transfer-implementing program, which when executed by a processor implements the steps of the method for automatically diagnosing benign and malignant thyroid nodules according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110101529.2A CN112820399A (en) | 2021-01-26 | 2021-01-26 | Method and device for automatically diagnosing benign and malignant thyroid nodules |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110101529.2A CN112820399A (en) | 2021-01-26 | 2021-01-26 | Method and device for automatically diagnosing benign and malignant thyroid nodules |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112820399A true CN112820399A (en) | 2021-05-18 |
Family
ID=75859225
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110101529.2A Pending CN112820399A (en) | 2021-01-26 | 2021-01-26 | Method and device for automatically diagnosing benign and malignant thyroid nodules |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112820399A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113314215A (en) * | 2021-05-21 | 2021-08-27 | 北京航空航天大学 | Ultrasonic thyroid nodule sample abundance and benign and malignant automatic auxiliary identification system |
| CN113436154A (en) * | 2021-06-11 | 2021-09-24 | 北京小白世纪网络科技有限公司 | Thyroid nodule edge symptom classification method, device and system |
| CN113449777A (en) * | 2021-06-08 | 2021-09-28 | 上海深至信息科技有限公司 | Automatic thyroid nodule grading method and system |
| CN114419016A (en) * | 2022-01-25 | 2022-04-29 | 华润武钢总医院 | Ultrasonic super-resolution method and system for differential diagnosis of benign and malignant thyroid nodules |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107680678A (en) * | 2017-10-18 | 2018-02-09 | 北京航空航天大学 | Based on multiple dimensioned convolutional neural networks Thyroid ultrasound image tubercle auto-check system |
| CN108510495A (en) * | 2018-04-09 | 2018-09-07 | 沈阳东软医疗系统有限公司 | A kind of lung image data processing method based on artificial intelligence, apparatus and system |
| CN111598876A (en) * | 2020-05-18 | 2020-08-28 | 北京小白世纪网络科技有限公司 | Method, system and equipment for constructing thyroid nodule automatic identification model |
-
2021
- 2021-01-26 CN CN202110101529.2A patent/CN112820399A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107680678A (en) * | 2017-10-18 | 2018-02-09 | 北京航空航天大学 | Based on multiple dimensioned convolutional neural networks Thyroid ultrasound image tubercle auto-check system |
| CN108510495A (en) * | 2018-04-09 | 2018-09-07 | 沈阳东软医疗系统有限公司 | A kind of lung image data processing method based on artificial intelligence, apparatus and system |
| CN111598876A (en) * | 2020-05-18 | 2020-08-28 | 北京小白世纪网络科技有限公司 | Method, system and equipment for constructing thyroid nodule automatic identification model |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113314215A (en) * | 2021-05-21 | 2021-08-27 | 北京航空航天大学 | Ultrasonic thyroid nodule sample abundance and benign and malignant automatic auxiliary identification system |
| CN113449777A (en) * | 2021-06-08 | 2021-09-28 | 上海深至信息科技有限公司 | Automatic thyroid nodule grading method and system |
| CN113436154A (en) * | 2021-06-11 | 2021-09-24 | 北京小白世纪网络科技有限公司 | Thyroid nodule edge symptom classification method, device and system |
| CN114419016A (en) * | 2022-01-25 | 2022-04-29 | 华润武钢总医院 | Ultrasonic super-resolution method and system for differential diagnosis of benign and malignant thyroid nodules |
| CN114419016B (en) * | 2022-01-25 | 2022-11-04 | 华润武钢总医院 | Ultrasonic super-resolution method and system for thyroid nodule benign and malignant differential diagnosis |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11101033B2 (en) | Medical image aided diagnosis method and system combining image recognition and report editing | |
| Domingues et al. | Using deep learning techniques in medical imaging: a systematic review of applications on CT and PET | |
| CN112101451B (en) | Breast cancer tissue pathological type classification method based on generation of antagonism network screening image block | |
| CN110070935B (en) | Medical image synthesis method, classification method and device based on antagonistic neural network | |
| CN112820399A (en) | Method and device for automatically diagnosing benign and malignant thyroid nodules | |
| CN109754361A (en) | The anisotropic hybrid network of 3D: the convolution feature from 2D image is transmitted to 3D anisotropy volume | |
| CN110648337A (en) | Hip joint segmentation method, hip joint segmentation device, electronic apparatus, and storage medium | |
| CN108205806B (en) | Automatic analysis method for three-dimensional craniofacial structure of cone beam CT image | |
| CN113298830B (en) | Acute intracranial ICH region image segmentation method based on self-supervision | |
| JP2020166809A (en) | System, apparatus, and learning method for training models | |
| Sirjani et al. | Automatic cardiac evaluations using a deep video object segmentation network | |
| Pradhan et al. | Lung cancer detection using 3D convolutional neural networks | |
| Dovganich et al. | Automatic out-of-distribution detection methods for improving the deep learning classification of pulmonary X-ray images | |
| Groza et al. | Pneumothorax segmentation with effective conditioned post-processing in chest X-ray | |
| Zhou et al. | Many-to-one distribution learning and k-nearest neighbor smoothing for thoracic disease identification | |
| Rasheed et al. | Use of transfer learning and wavelet transform for breast cancer detection | |
| Liu et al. | Toward automated right ventricle segmentation via edge feature-induced self-attention multiscale feature aggregation full convolution network | |
| Tiwari et al. | Optimized Ensemble of Hybrid RNN-GAN Models for Accurate and Automated Lung Tumour Detection from CT Images | |
| Sengan et al. | Echocardiographic image segmentation for diagnosing fetal cardiac rhabdomyoma during pregnancy using deep learning | |
| Susomboon et al. | Automatic single-organ segmentation in computed tomography images | |
| Kovalev et al. | Automatic detection of pathological changes in chest X-ray screening images using deep learning methods | |
| Kumari et al. | Data efficient deep learning for medical image analysis: A survey | |
| Heker et al. | Hierarchical fine-tuning for joint liver lesion segmentation and lesion classification in ct | |
| Xu et al. | Breast Anatomy Enriched Tumor Saliency Estimation | |
| CN113223104B (en) | Cardiac MR image interpolation method and system based on causal relationship |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |