CN106056595A - Method for automatically identifying whether thyroid nodule is benign or malignant based on deep convolutional neural network - Google Patents

Abstract

The invention relates to auxiliary medical diagnoses, and aims to provide a method for automatically identifying whether a thyroid nodule is benign or malignant based on a deep convolutional neural network. The method for automatically identifying whether the thyroid nodule is benign or malignant based on the deep convolutional neural network comprises the following steps: reading B ultrasonic data of thyroid nodules; performing preprocessing for thyroid nodule images; selecting images, and obtaining nodule portions and non-nodule portions through segmentations; averagely dividing the extracted ROIs (regions of interest) into p groups, extracting characteristics of the ROIs by utilizing a CNN (convolutional neural network), and performing uniformization; taking p-1 groups of data as a training set, taking the remaining one group to make a test, and obtaining an identification model through training to make the test; and repeating cross validation for p times, and then obtaining an optimum parameter of the identification model. The method can obtain the thyroid nodules through the automatic segmentations by means of the deep convolutional neural network, and makes up for the deficiency that a weak boundary problem cannot be solved based on a movable contour and the like; and the method can automatically lean and extract valuable feature combinations, and prevent the complexity of an artificial feature selection.

Description

Method for automatically identifying benign and malignant thyroid nodules based on deep convolutional neural network

Technical Field

The invention relates to the field of auxiliary medical diagnosis, in particular to a method for automatically identifying benign and malignant thyroid nodules based on a deep convolutional neural network.

Background

In recent years, with the rapid development of computer technology and digital image processing technology, digital image processing technology is increasingly applied to the field of auxiliary medical diagnosis, and the principle of the technology is to perform image processing technologies such as segmentation, reconstruction, registration, identification and the like on medical images acquired by different modes so as to obtain valuable medical diagnosis information.

Thyroid nodules are now a ubiquitous epidemic and investigations have shown that the incidence of thyroid nodules in the population is nearly 50%, but only 4% -8% of thyroid nodules are accessible in clinical palpation. Thyroid nodules are classified as benign and malignant, and the incidence of malignancy is 5% -10%. Early detection of lesions is of great significance in identifying benign and malignant lesions, clinical treatment and surgical selection. The thyroid nodule ultrasonic examination based on the ultrasonic imaging technology has the advantages of real-time imaging, relatively low examination cost, no wound to patients and the like. And the thyroid is positioned on the surface layer, so that the thyroid is suitable for ultrasonic image diagnosis. The doctor mainly relies on the puncture biopsy cell examination to diagnose the benign and malignant thyroid gland, so the workload can be very large, and the result of the doctor diagnosing the ultrasonic thyroid gland image is often influenced by the imaging mechanism, the acquisition condition, the display equipment and other factors of the medical imaging equipment, so that misdiagnosis or missed diagnosis is very easy to cause. Therefore, it is necessary to use a computer to perform thyroid image-assisted diagnosis. However, the inherent imaging mechanism causes the quality of the clinically acquired ultrasonic thyroid tumor images to be poor, which causes the accuracy and automation of auxiliary diagnosis to be affected, so that the most current thyroid nodule segmentation is semi-automatic segmentation based on active contour, classification mainly comprises manual feature selection, and then classification recognition such as SVM, KNN, decision tree and the like is utilized, and the classifiers only have good effect on small sample data, but the medical data is massive, and the classification recognition of a large sample can have better auxiliary effect on the medical diagnosis.

Disclosure of Invention

The invention mainly aims to overcome the defects in the prior art and provide a method for automatically identifying benign and malignant thyroid nodules based on a deep convolutional neural network. In order to solve the technical problem, the solution of the invention is as follows:

a method for automatically identifying benign and malignant thyroid nodules based on a deep convolutional neural network is provided, and comprises the following processes:

firstly, reading B ultrasonic data of thyroid nodules;

secondly, preprocessing the thyroid nodule image;

selecting an image, automatically learning and segmenting a nodule part and a non-nodule part by using a Convolutional Neural Network (CNN) (convolutional neural network), wherein the nodule part is an interested Region (ROI), and refining the shape of the nodule;

fourthly, dividing the ROI extracted in the third step into p groups on average, extracting the characteristics of the ROI by using CNN, and normalizing;

fifthly, selecting p-1 group data in the fourth step as a training set, testing the rest group, and training a recognition model through CNN for testing;

sixthly, repeating the step five, performing p times of cross tests to obtain the optimal parameters of the recognition model, and finally determining an auxiliary diagnosis system for automatically recognizing the benign and malignant thyroid nodules based on the deep convolutional neural network;

the first process specifically comprises the following steps: reading thyroid nodule images (which can be in a picture format and can also be standard dicom pictures) comprising at least 5000 images of benign nodules and at least 5000 images of malignant nodules;

the second process specifically comprises the following steps: carrying out image graying on the thyroid nodule image read in the first process, removing marks made by doctors for measuring nodule related quantity in the ultrasonic image by utilizing gray values of surrounding pixel points, denoising by utilizing Gaussian filtering, and finally carrying out equalization enhancement on contrast by utilizing a gray histogram to obtain a preprocessed enhanced image;

the third process specifically comprises the following steps:

step 1: selecting 10000 enhanced images preprocessed by the second process, wherein the enhanced images comprise 5000 good and malignant nodules;

step 2: for each picture, firstly (by an expert) manually cutting out nodule parts and non-nodule parts, and then training out an automatic segmentation model through CNN;

the CNN is a network structure consisting of 13 convolutional layers and 2 downsampling layers; the sizes of convolution kernels of the convolutional layers are respectively: the first layer was 13x13, the second and third layers were 5x5, and the remaining layers were 3x 3; the step sizes of the convolutional layers are respectively as follows: the first two convolutional layers are 2, the remainder are 1; the size of the down-sampling layers is 3x3, and the step size is 2;

the specific method for training the automatic segmentation model through the CNN comprises the following steps:

(1) the method comprises the following steps of automatically learning features through a CNN convolution layer and a down-sampling layer, and extracting the features, wherein the method comprises the following specific steps:

step A: on a convolution layer, the feature maps of the upper layer are convoluted by a convolution kernel which can be learned, and then an output feature map can be obtained through an activation function; each output is the value of a convolution kernel convolving one input or combining multiple convolved inputs (here we choose to combine the values of the convolved multiple incoming and outgoing maps):

$x_j^l=f({\mathrm{\Σ}}_{i\∈M_j}{x}_i^{l-1}*k_{ij}^l+b_j^l)$

wherein the symbol denotes a convolution operationAn operator; the l represents the number of layers; the i represents the ith neuron node of the l-1 layer; the j represents the jth neuron node of the l layer; the M is_jRepresenting a set of selected input maps; the above-mentionedRefers to the output of the l-1 layer as the input of the l layer; f is an activation function, here a sigmoid functionAs an activation function, e denotes the euler number 2.718281828, e^xIs an exponential function; the k is a convolution operator; b is a bias; each output map is given an additional offset b, but the convolution kernel that convolves each input map is different for a particular output map;

this step also requires a gradient calculation to update the sensitivity, which is used to indicate how much b changes, how much the error will change:

wherein l represents the number of layers; the j represents the jth neuron node of the l layer; o represents each element multiplication; the sensitivity of the representation output neuron, i.e. the rate of change of the bias b; s is^l＝W^lx^l-1+b^l，x^l-1Refers to the output of the l-1 layer, W is the weight, b is the offset; f is an activation function, here a sigmoid functionAs an activation function, e denotes the euler number 2.718281828, e^xIs an exponential function; f' (x) is the derivative function of f (x) (i.e. if f takes the sigmoid function)Then f' (x) ═ (1-f (x)); the above-mentionedRepresenting the weight shared by each layer; up () denotes an upsampling operation (if the downsampling sampling factor is n, the upsampling operation is to copy each pixel n times horizontally and vertically, so that the original size can be restored);

then sum all nodes in the sensitivity map in layer l, fast calculate the gradient of bias b:

$\frac{\∂E}{\∂b_j^l}={\mathrm{\Σ}}_{u,v}{\left({\mathrm{\δ}}_j^l\right)}_{u,v}$

wherein l represents the number of layers; the j represents the jth neuron node of the l layer; b represents a bias; the sensitivity of the representation output neuron, i.e. the rate of change of the bias b; the u, v represents the (u, v) position of the output maps; said E is an error function, whereThe C represents the dimension of the label, and if the problem is two-classification, the label can be marked as y_h∈ {0,1}, where C is 1, it can also be written as y_h∈ { (0,1), (1,0) }, when C ═ 2; saidThe h dimension of the label corresponding to the nth sample is represented; the above-mentionedAn h output representing a network output corresponding to the n sample;

and finally, calculating the weight of the convolution kernel by using a BP algorithm:

${\mathrm{\ΔW}}^l=-\mathrm{\η}\frac{\∂E}{\∂W^l}$

wherein W is a weight parameter; said E is an error function, andthe C represents the dimension of the label, and if the problem is two-classification, the label can be marked as y_h∈ {0,1}, where C is 1, it can also be written as y_h∈ { (0,1), (1,0) }, when C ═ 2; saidThe h dimension of the label corresponding to the nth sample is represented; the above-mentionedThe h output representing the net output corresponding to the n sample, η being the learning rate, i.e. the step size, since the weights of many connections are shared, for a given weight, it is necessary to gradient the point for all connections associated with the weight and then sum the gradients:

$\frac{\∂E}{\∂k_{ij}^l}={\mathrm{\Σ}}_{u,v}{\left({\mathrm{\δ}}_j^l\right)}_{u,v}{\left(p_i^{l-1}\right)}_{u,v}$

wherein l represents the number of layers; the i represents the ith neuron node of the l layer; the j represents the jth neuron node of the l layer; b represents the bias, which represents the sensitivity of the output neuron, i.e. the rate of change of bias b; the u, v represents the (u, v) position of the output maps; said E is an error function, whereThe C represents the dimension of the label, and if the problem is two-classification, the label can be marked as y_h∈ {0,1}, where C is 1, it can also be written as y_h∈ { (0,1), (1,0) }, when C ═ 2; saidThe h dimension of the label corresponding to the nth sample is represented; the above-mentionedDenotes the n-thH output of the network output corresponding to the sample; the above-mentionedIs a convolution kernel; the above-mentionedIs thatWhen convolved withThe value of the (u, v) position of the output convolution map is determined by the (u, v) position of the previous layer, which is the region block in all the pictures with the same size as the convolution kernel, and the element-by-element multiplication patchThe result of element-by-element multiplication;

and B: the down-sampling layer has N input maps, and has N output maps, and only if each output map is reduced, then has:

$x_j^l=f\left({\mathrm{\β}}_j^{l}down\right(x_j^{l-1})+b_j^l)$

wherein f is an activation function, here a sigmoid function is takenAs an activation function, e denotes the euler number 2.718281828, e^xIs an exponential function; the above-mentionedSumming all pixels of different nxn blocks of the input image, so that the output image is reduced by n times in two dimensions (in this case, each element of the input image is taken as a block with the size of 3x3, and then all elements are summed as the value of the element in the output image, so that the output image is reduced by 3 times in two dimensions);

parameters β and b are updated by the gradient descent method:

$\frac{\∂E}{\∂b_j}={\mathrm{\Σ}}_{u,v}{\left({\mathrm{\δ}}_j^l\right)}_{u,v}$

wherein the conv2 is a two-dimensional convolution operator; the rot180 is rotated 180 degrees; the 'full' means that complete convolution is performed; the l represents the number of layers; the i represents the ith neuron node of the l layer; the j represents the jth neuron node of the l layer; b represents a bias; the sensitivity of the representation output neuron, i.e. the rate of change of the bias b; the u, v represents the (u, v) position of the output maps; said E is an error function, the expression is as above, i.e.The C represents the dimension of the label, and if the problem is two-classification, the label can be marked as y_h∈ {0,1}, where C is 1, it can also be written as y_h∈ { (0,1), (1,0) }, when C ═ 2; saidThe h dimension of the label corresponding to the nth sample is represented; the above-mentionedRepresenting the h output of the network output corresponding to the n sample, wherein β is a weight parameter (generally taking a value of 0,1)]) (ii) a Down (.) represents a downsampling function; the above-mentionedIs the convolution kernel of layer l + 1; the above-mentionedIs the jth neuron node of the output of layer l-1; s is^l＝W^lx^l-1+b^lWhere W is a weight parameter, b is an offset,is s^lThe jth component of (a);

and C: the CNN automatically learns the combination of the feature maps, and the jth feature map combination is as follows:

$x_j^l=f\left(\underset{i=1}{\overset{N_{in}}{\mathrm{\Σ}}}{\mathrm{\α}}_{ij}\right(x_i^{l-1}*k_i^l)+b_j^l)$

s.t.∑_iα_ij＝1,and 0≤α_ij≤1.

wherein the symbol denotes the convolution operator; the l represents the number of layers; the i represents the ith neuron node of the l layer; the j represents the jth neuron node of the l layer; f is an activation function, here a sigmoid functionAs an activation function, e denotes the euler number 2.718281828, e^xIs an exponential function; the above-mentionedIs the ith component of the l-1 layer output; said N is_inRepresenting the number of maps input; what is needed isThe above-mentionedIs a convolution kernel; the above-mentionedIs an offset, said α_ijRepresenting the output map of the l-1 layer as the input of the l layer, and obtaining the weight or contribution of the ith input map of the jth output map by the l-1 layer;

(2) automatically identifying nodules by using the features extracted in the step (1) and combining Softmax, and determining a model for automatic segmentation; in particular, if a sample is given, the Softmax recognition procedure outputs a probability value indicating the probability that the sample belongs to a class of several classes, and the loss function is:

$J\left(\mathrm{\θ}\right)=-\frac{1}{m}\[\underset{i=1}{\overset{m}{\mathrm{\Σ}}}\underset{j=1}{\overset{c}{\mathrm{\Σ}}}1\{y^{\left(i\right)}=j\}log\frac{e^{{\mathrm{\θ}}_j^T{x}^{\left(i\right)}}}{{\mathrm{\Σ}}_{l=1}^c{e}^{{\mathrm{\θ}}_l^{T}x\left(i\right)}}\]+\frac{\mathrm{\λ}}{2}\underset{i=1}{\overset{c}{\mathrm{\Σ}}}\underset{j=0}{\overset{n}{\mathrm{\Σ}}}{\mathrm{\θ}}_{ij}^2$

wherein m represents a total of m samples; the c represents that the samples can be divided into c types in total; the above-mentionedIs a matrix, each row is the parameter corresponding to a category, namely weight and bias; said 1{ } is an indicative function, i.e. when the value in the brace is true, the result of the function is 1, otherwise the result is 0; λ is a parameter that balances the fidelity term (first term) with the regularization term (second term), where λ takes a positive number (its magnitude is adjusted according to experimental results); the J (theta) refers to a loss function of the system; said e represents the Euler number 2.718281828, e^xIs an exponential function; the T is a transpose operator in the representation matrix calculation; log represents the natural logarithm, i.e., the logarithm based on the euler number; n represents the dimension of the weight and bias parameters；x⁽ⁱ⁾Is the ith dimension of the input vector; y is⁽ⁱ⁾Is the ith dimension of each sample label; then the gradient is used to solve:

${\▿}_{{\mathrm{\θ}}_j}J\left(\mathrm{\θ}\right)=-\frac{1}{m}\[\underset{i=1}{\overset{m}{\mathrm{\Σ}}}{x}^{\left(i\right)}\left(1\right\{y^{\left(i\right)}=j\}-p(y^{\left(i\right)}=j|x^{\left(i\right)};\mathrm{\θ}\left)\right)\]+{\mathrm{\λ\θ}}_j$

wherein,the m represents a total of m samples; the above-mentionedIs a matrix, each row is the parameter corresponding to a category, namely weight and bias; said 1{ } is an indicative function, i.e. when the value in the brace is true, the result of the function is 1, otherwise the result is 0; λ is a parameter that balances the fidelity term (first term) with the regularization term (second term), where λ takes a positive number (its magnitude is adjusted according to experimental results); the J (theta) refers to a loss function of the system;is the J (θ) derivative function; said e represents the Euler number 2.718281828, e^xIs an exponential function; the T is a transpose operator in the representation matrix calculation; log represents the natural logarithm, i.e., the logarithm based on the euler number; x is the number of⁽ⁱ) Is the ith dimension of the input vector; y is⁽ⁱ⁾Is the ith dimension of each sample label;

(where a new Softmax classifier, i.e., a Softmax classifier with only two classes is used, for a thyroid image, a probability map can be obtained for distinguishing all nodule regions from non-nodule regions based on the probability given by Softmax, and a rough segmentation of nodule regions can be obtained based on the map)

(3) Automatically segmenting nodules of all thyroid glands by using CNN (CNN), namely distinguishing a nodule region from a non-nodule region, finding out the boundary of the nodule region, and refining the shapes of the segmented nodules, namely filling holes and removing the connection with the non-nodule region by using corrosion and expansion morphological operators;

and 3, step 3: automatically segmenting all thyroid nodule pictures (namely 10000 pictures) by using the model obtained in the step 2 to obtain an ROI (region of interest), namely all benign and malignant nodules;

the fourth process specifically comprises the following steps: dividing the ROI automatically segmented in the third process into p groups on average, normalizing the data, namely extracting the characteristics of the nodules after automatically segmenting the nodules, and performing linear transformation on the characteristics to map a result value to [0,1 ];

the fifth process specifically comprises the following steps: the method for extracting features from all ROIs (the method for extracting features in a specific process and three automatic segmentation processes is the same, except that the object only aims at a nodule region, three convolutional layers are reduced in a network structure compared with the automatic segmentation process, 3 fully-connected layers are added, the number of neuron nodes is 64, 64 and 1, the sizes of convolutional cores are respectively 13x13 in the first layer, 5x5 in the second layer and the third layer, and 3x3 in the rest layers, step sizes are respectively 2 in the first three convolutional layers and 1 in the rest layers, the sizes of downsampling layers are 3x3 and 2, and features are extracted from non-nodule regions and nodule regions at the same time in the automatic segmentation part);

then, a new Softmax classification is utilized, namely, only two classes of Softmax classifiers are utilized to solve the optimal value of a loss function, namely, optimization J (theta), wherein the class number c of the Softmax classifiers is equal to 2 (namely, benign nodules and malignant nodules); the probability of belonging to a benign nodule or a malignant nodule can be obtained by a gradient descent method, and the specific process is the same as the method of the automatic segmentation process in the third process (only a classification label is predicted according to the probabilities, and then the benign and malignant diagnosis is carried out on one nodule);

the sixth process specifically comprises the following steps: repeating the process five, namely selecting p-1 group of data for training each time for p group of data, and performing the rest tests to finally obtain the optimal parameters of the recognition model, thereby obtaining the auxiliary diagnosis system for automatically recognizing the benign and malignant thyroid nodules based on the deep convolutional neural network;

the thyroid nodule image to be identified is input into the auxiliary diagnosis system, and the benign and malignant diagnosis of the nodule can be obtained.

Compared with the prior art, the invention has the beneficial effects that:

according to the invention, thyroid nodules can be automatically segmented by means of the deep convolutional neural network, the defect that the problem of weak boundary cannot be solved based on the active contour and the like is overcome, valuable feature combinations can be automatically learned and extracted, the complexity of manually selecting features is avoided, the extracted features are more favorable for finding out main rule information of benign and malignant thyroid nodules, the accuracy of a recognition system is improved, and high adaptability is obtained.

Drawings

Fig. 1 is a flow chart for identifying benign and malignant thyroid nodules based on a deep convolutional neural network.

Fig. 2 is a diagram of a convolutional neural network architecture for automatically segmenting and identifying thyroid nodules.

Fig. 3 is a raw picture of thyroid nodules used in the examples.

FIG. 4 is a photograph of a mask drawn by the expert for the thyroid nodule region of FIG. 3.

Fig. 5 is a raw picture of thyroid nodules in the examples.

Fig. 6 is a picture of the effect of automatically segmenting the nodule region of fig. 5 using CNN.

Detailed Description

The invention is described in further detail below with reference to the following detailed description and accompanying drawings:

the following examples are presented to enable those skilled in the art to more fully understand the present invention and are not intended to limit the invention in any way.

A method for automatically identifying benign and malignant thyroid nodules based on a deep convolutional neural network is shown in figure 1 and comprises the following steps:

firstly, reading B ultrasonic data of thyroid nodules;

secondly, preprocessing the thyroid nodule image;

thirdly, selecting images (including as many good and malignant nodule images) and automatically learning and segmenting nodule parts and non-nodule parts by using a Convolutional Neural Network (CNN), wherein the nodule parts are regions of interest (ROI), and refining the shapes of the nodules;

and fourthly, dividing the ROI extracted in the third step into p groups on average, extracting the characteristics of the ROI by using CNN, and normalizing.

Fifthly, selecting p-1 group data in the fourth step as a training set, testing the rest group, and training a model through CNN for testing;

sixthly, repeating the step five, performing p times of cross tests to obtain the optimal parameters of the recognition model, and finally determining an auxiliary diagnosis system for automatically recognizing the benign and malignant thyroid nodules based on the deep convolutional neural network;

the first process specifically comprises the following steps: the B-mode ultrasound data of thyroid nodules are read in a picture format or in a standard dicom picture. Comprising at least 5000 images of benign nodules and at least 5000 images of malignant nodules; in the fifth process, all the pictures (i.e. p-1 group of data) in the training set need to be read in first to train out the auxiliary diagnosis system for automatically identifying the benign and malignant thyroid nodules based on the deep convolutional neural network, and then the data of the remaining 1 group is read in to test the system. When the system is used for auxiliary diagnosis for automatically identifying benign and malignant thyroid nodules, all pictures of the nodules to be diagnosed are read;

the second process specifically comprises the following steps: carrying out image graying on the thyroid nodule image read in the first process, removing marks made by doctors for measuring nodule related quantity in the ultrasonic image by utilizing gray values of surrounding pixel points, denoising by utilizing Gaussian filtering, and finally carrying out equalization enhancement on contrast by utilizing a gray histogram to obtain a preprocessed enhanced image;

the third process specifically comprises the following steps:

step 1: selecting 10000 enhanced images preprocessed by the second process, wherein the enhanced images comprise 5000 good and malignant nodules;

step 2: intercepting a nodule part and a non-nodule part by an expert, and then training an automatic segmentation model through CNN; the CNN described here is a network structure composed of 13 convolutional layers and 2 downsampling layers, and the sizes of the convolutional cores are: the first layer was 13x13, the second and third layers were 5x5, and the remaining layers were 3x3, steps were: the first two convolutional layers are 2, the remainder are 1. The size of the down-sampling layers is 3x3, and the step size is 2; the specific convolutional neural network structure is shown in fig. 2;

the specific method for training the automatic segmentation model through the CNN comprises the following steps:

(1) the method comprises the following steps of automatically learning features through a CNN convolution layer and a down-sampling layer, and extracting the features, wherein the method comprises the following specific steps:

step A: on a convolution layer, the feature maps of the upper layer are convoluted by a convolution kernel which can be learned, and then an output feature map can be obtained through an activation function; each output is the value of a convolution kernel convolving one input or combining multiple convolved inputs (here we choose to combine the values of the convolved multiple incoming and outgoing maps):

$x_j^l=f({\mathrm{\Σ}}_{i\∈M_j}{x}_i^{l-1}*k_{ij}^l+b_j^l)$

wherein the symbol denotes the convolution operator; the l represents the number of layers; the i represents the ith neuron node of the l-1 layer; the j represents the jth neuron node of the l layer; the M is_jRepresenting a set of selected input maps; the above-mentionedIs the output; the above-mentionedIs the output of the l-1 layer and is used as the input of the ll layer; f is an activation function, here a sigmoid functionAs an activation function; said e represents the Euler number 2.718281828, e^xIs an exponential function; the k is a convolution operator; b is a bias; each output map is given an additional offset b, but the convolution kernel that convolves each input map is different for a particular output map;

this step also requires a gradient calculation to update the sensitivity, which is used to indicate how much b changes, how much the error will change:

wherein l represents the number of layers; the j represents the jth neuron node of the l layer; o represents each element multiplication; the sensitivity of the representation output neuron, i.e. the rate of change of the bias b; s is^l＝W^lx^l-1+b^l(ii) a W is a weight; b is an offset; f is an activation function, here a sigmoid functionAs an activation function; said e represents the Euler number 2.718281828, e^xIs an exponential function, f' (x) is the derivative function of f (x), if f is the sigmoid functionThen f' (x) ═ (1-f (x)) f (x); the above-mentionedRepresenting the weight shared by each layer; up (.) represents an upsampling operation, and if the sampling factor of the downsampling is n, the upsampling operation is to copy each pixel in the horizontal direction and the vertical direction for n times, so that the original size can be recovered;

then sum all nodes in the sensitivity map in layer l, fast calculate the gradient of bias b:

$\frac{\∂E}{\∂b_j^l}={\mathrm{\Σ}}_{u,v}{\left({\mathrm{\δ}}_j^l\right)}_{u,v}$

wherein l represents the number of layers; the j represents the jth neuron node of the l layer; b represents a bias; the sensitivity of the representation output neuron, i.e. the rate of change of the bias b; the u, v represents the (u, v) position of the output maps; said E is an error function, whereThe C represents the dimension of the label, and if the problem is two-classification, the label can be marked as y_h∈ {0,1}, where C is 1, it can also be written as y_h∈ { (0,1), (1,0) }, when C ═ 2; saidThe h dimension of the label corresponding to the nth sample is represented; the above-mentionedAn h output representing a network output corresponding to the n sample;

and finally, calculating the weight of the convolution kernel by using a BP algorithm:

${\mathrm{\ΔW}}^l=-\mathrm{\η}\frac{\∂E}{\∂W^l}$

wherein W is a weight parameter; said E is an error function, andthe C represents the dimension of the label, and if the problem is two-classification, the label can be marked as y_h∈ {0,1}, where C is 1, it can also be written as y_h∈ { (0,1), (1,0) }, when C ═ 2; saidThe h dimension of the label corresponding to the nth sample is represented; the above-mentionedAn h output representing a network output corresponding to the n sample; the above-mentionedη is the learning rate, i.e. the step size, since the weights of many connections are shared, for a given weight, it is necessary to gradient the point for all connections associated with the weight and then sum the gradients:

$\frac{\∂E}{\∂k_{ij}^l}={\mathrm{\Σ}}_{u,v}{\left({\mathrm{\δ}}_j^l\right)}_{u,v}{\left(p_i^{l-1}\right)}_{u,v}$

wherein l represents the number of layers; the i represents the ith neuron node of the l layer; the j represents the jth neuron node of the l layer; b represents the bias, which represents the sensitivity of the output neuron, i.e. the rate of change of bias b; the u, v represents the (u, v) position of the output maps; said E is an error function, whereThe C represents the dimension of the label, and if the problem is two-classification, the label can be marked as y_h∈ {0,1}, where C is 1, it can also be written asIs y_h∈ { (0,1), (1,0) }, when C ═ 2; saidThe h dimension of the label corresponding to the nth sample is represented; the above-mentionedAn h output representing a network output corresponding to the n sample; the above-mentionedIs a convolution kernel; the above-mentionedIs thatWhen convolved withThe value of the (u, v) position of the output convolution map is determined by the (u, v) position of the previous layer, which is the region block in all the pictures with the same size as the convolution kernel, and the element-by-element multiplication patchThe result of element-by-element multiplication;

and B: the down-sampling layer has N input maps, and has N output maps, and only if each output map is reduced, then has:

$x_j^l=f\left({\mathrm{\β}}_j^{l}down\right(x_j^{l-1})+b_j^l)$

wherein f is an activation function, here a sigmoid function is takenAs an activation function, e denotes the euler number 2.718281828, e^xIs an exponential function; the above-mentionedSumming all pixels of different nxn blocks of the input image, so that the output image is reduced by n times in two dimensions (in this case, each element of the input image is taken as a block with the size of 3x3, and then all elements are summed as the value of the element in the output image, so that the output image is reduced by 3 times in two dimensions);

parameters β and b are updated by the gradient descent method:

$\frac{\∂E}{\∂b_j}={\mathrm{\Σ}}_{u,v}{\left({\mathrm{\δ}}_j^l\right)}_{u,v}$

wherein the conv2 is a two-dimensional convolution operator; the rot180 is rotated 180 degrees; the 'full' means that complete convolution is performed; the l represents the number of layers; the i represents the ith neuron node of the l layer; the j represents the jth neuron node of the l layer; b represents a bias; the sensitivity of the representation output neuron, i.e. the rate of change of the bias b; the u, v represents the (u, v) position of the output maps; said E is an error function, the expression is as above, i.e.The C represents the dimension of the label, and if the problem is two-classification, the label can be marked as y_h∈ {0,1}, where C is 1, it can also be written as y_h∈ { (0,1), (1,0) }, when C ═ 2; saidThe h dimension of the label corresponding to the nth sample is represented; the above-mentionedRepresenting the h output of the network output corresponding to the n sample, wherein β is a weight parameter (generally taking a value of 0,1)]) (ii) a Down (.) represents a downsampling function; the above-mentionedIs the convolution kernel of layer l + 1; the above-mentionedIs the jth neuron node of the output of layer l-1; what is needed isS is^l＝W^lx^l-1+b^lWhere W is a weight parameter, b is an offset,is s^lThe jth component of (a);

and C: the CNN automatically learns the combination of the feature maps, and the jth feature map combination is as follows:

$x_i^l=f\left(\underset{i=1}{\overset{N_{in}}{\mathrm{\Σ}}}{\mathrm{\α}}_{ij}\right(x_i^{l-1}*k_i^l)+b_j^l)$

s_.t.∑_iα_ij＝1,and 0≤α_ij≤1.

wherein the symbol denotes the convolution operator; the l represents the number of layers; the i represents the ith neuron node of the l layer; the j represents the jth neuron node of the l layer; f is an activation function, here a sigmoid functionAs an activation function, e denotes the euler number 2.718281828, e^xIs an exponential function; the above-mentionedIs the ith component of the l-1 layer output; said N is_inRepresenting the number of maps input; the above-mentionedIs a convolution kernel; the above-mentionedIs an offset, said α_ijRepresenting the output map of the l-1 layer as the input of the l layer, and obtaining the weight or contribution of the ith input map of the jth output map by the l-1 layer;

(2) automatically identifying nodules by using the features extracted in the step (1) and combining Softmax, and determining a model for automatic segmentation; in particular, if a sample is given, the Softmax recognition procedure outputs a probability value indicating the probability that the sample belongs to a class of several classes, and the loss function is:

$J\left(\mathrm{\θ}\right)=-\frac{1}{m}\[\underset{i=1}{\overset{m}{\mathrm{\Σ}}}\underset{j=1}{\overset{c}{\mathrm{\Σ}}}1\{y^{\left(i\right)}=j\}log\frac{e^{{\mathrm{\θ}}_j^T{x}^{\left(i\right)}}}{{\mathrm{\Σ}}_{l=1}^c{e}^{{\mathrm{\θ}}_l^{T}x\left(i\right)}}\]+\frac{\mathrm{\λ}}{2}\underset{i=1}{\overset{c}{\mathrm{\Σ}}}\underset{j=0}{\overset{n}{\mathrm{\Σ}}}{\mathrm{\θ}}_{ij}^2$

wherein m represents a total of m samples; the c represents that the samples can be divided into c types in total; the above-mentionedIs a matrix, each row is the parameter corresponding to a category, namely weight and bias; said 1{ } is an indicative function, i.e. when the value in the brace is true, the result of the function is 1, otherwise the result is 0; what is needed isλ is a parameter for balancing the fidelity term (first term) with the regularization term (second term), where λ is a positive number (its magnitude is adjusted according to experimental results); the J (theta) refers to a loss function of the system; said e represents the Euler number 2.718281828, e^xIs an exponential function; the T is a transpose operator in the representation matrix calculation; log represents the natural logarithm, i.e., the logarithm based on the euler number; n represents the dimension of the weight and bias parameters; x is the number of⁽ⁱ⁾Is the ith dimension of the input vector; y is⁽ⁱ⁾Is the ith dimension of each sample label; then the gradient is used to solve:

${\▿}_{{\mathrm{\θ}}_j}J\left(\mathrm{\θ}\right)=-\frac{1}{m}\[\underset{i=1}{\overset{m}{\mathrm{\Σ}}}{x}^{\left(i\right)}\left(1\right\{y^{\left(i\right)}=j\}-p(y^{\left(i\right)}=j|x^{\left(i\right)};\mathrm{\θ}\left)\right)\]+{\mathrm{\λ\θ}}_j$

wherein,the m represents a total of m samples; the above-mentionedIs a matrix, each row is the parameter corresponding to a category, namely weight and bias; said 1{ } is an indicative function, i.e. when the value in the brace is true, the result of the function is 1, otherwise the result is 0; λ is a parameter that balances the fidelity term (first term) with the regularization term (second term), where λ takes a positive number (its magnitude is adjusted according to experimental results); the J (theta) refers to a loss function of the system;is the J (θ) derivative function; said e represents the Euler number 2.718281828, e^xIs an exponential function; the T is a transpose operator in the representation matrix calculation; log represents the natural logarithm, i.e., the logarithm based on the euler number; x is the number of⁽ⁱ⁾Is the ith dimension of the input vector; y is⁽ⁱ⁾Is the ith dimension of each sample label;

(where a new Softmax classifier, i.e., a Softmax classifier with only two classes is used, for a thyroid image, a probability map can be obtained for distinguishing all nodule regions from non-nodule regions based on the probability given by Softmax, and a rough segmentation of nodule regions can be obtained based on the map)

(3) Automatically segmenting nodules of all thyroid glands by using CNN (CNN), namely distinguishing a nodule region from a non-nodule region, finding out the boundary of the nodule region, and refining the shapes of the segmented nodules, namely filling holes and removing the connection with the non-nodule region by using corrosion and expansion morphological operators;

and 3, step 3: automatically segmenting all thyroid nodule pictures (namely 10000 pictures) by using the model obtained in the step 2 to obtain an ROI (region of interest), namely all benign and malignant nodules;

the fourth process specifically comprises the following steps: dividing the ROI automatically segmented in the third process into p groups on average, normalizing the data, namely extracting the characteristics of the nodules after automatically segmenting the nodules, and performing linear transformation on the characteristics to map a result value to [0,1 ];

the fifth process specifically comprises the following steps: the method for extracting features from all ROIs (the method for extracting features in a specific process and three automatic segmentation processes is the same, except that the object only aims at a nodule region, three convolutional layers are reduced in a network structure compared with the automatic segmentation process, 3 fully-connected layers are added, the number of neuron nodes is 64, 64 and 1, the sizes of convolutional cores are respectively 13x13 in the first layer, 5x5 in the second layer and the third layer, and 3x3 in the rest layers, step sizes are respectively 2 in the first three convolutional layers and 1 in the rest layers, the sizes of downsampling layers are 3x3 and 2, and features are extracted from non-nodule regions and nodule regions at the same time in the automatic segmentation part); the specific convolutional neural network structure is shown in fig. 2;

then, a new Softmax classification is utilized, namely, only two classes of Softmax classifiers are utilized to solve the optimal value of a loss function, namely, optimization J (theta), wherein the class number p of the Softmax classifiers is equal to 2 (namely, benign nodules and malignant nodules); the probability of belonging to a benign nodule or a malignant nodule can be obtained by a gradient descent method, and the specific process is the same as the method of the automatic segmentation process in the third process (only a classification label is predicted according to the probabilities, and then the benign and malignant diagnosis is carried out on one nodule);

the sixth process specifically comprises the following steps: and repeating the experiment of the fifth process, namely selecting p-1 group of data for training each time for p group of data, and performing the rest tests to finally obtain the optimal parameters of the recognition model, thereby obtaining the auxiliary diagnosis system for automatically recognizing the benign and malignant thyroid nodules based on the deep convolutional neural network. The thyroid nodule image to be identified is input into the auxiliary diagnosis system, and the benign and malignant diagnosis of the nodule can be obtained.

Fig. 3 and 4 are images showing the original image of thyroid nodule and the mask image of corresponding nodule region used in the experiment; fig. 5 and 6 show an original picture of thyroid nodules and an effect picture of automatically segmenting a nodule region mask by using CNN.

Finally, it should be noted that the above-mentioned list is only a specific embodiment of the present invention. It is obvious that the present invention is not limited to the above embodiments, but many variations are possible. All modifications which can be derived or suggested by a person skilled in the art from the disclosure of the present invention are to be considered within the scope of the invention.

Claims (1)

1. The method for automatically identifying the benign and malignant thyroid nodules based on the deep convolutional neural network is characterized by comprising the following processes:

firstly, reading B ultrasonic data of thyroid nodules;

secondly, preprocessing the thyroid nodule image;

thirdly, selecting an image, automatically learning and segmenting a nodular part and a non-nodular part by using a Convolutional Neural Network (CNN), wherein the nodular part is an interested Region (ROI), and thinning the shape of the nodular part;

fourthly, dividing the ROI extracted in the third step into p groups on average, extracting the characteristics of the ROI by using CNN, and normalizing;

fifthly, selecting p-1 group data in the fourth step as a training set, testing the rest group, and training a recognition model through CNN for testing;

sixthly, repeating the step five, performing p times of cross tests to obtain the optimal parameters of the recognition model, and finally determining an auxiliary diagnosis system for automatically recognizing the benign and malignant thyroid nodules based on the deep convolutional neural network;

the first process specifically comprises the following steps: reading thyroid nodule images including at least 5000 images of benign nodules and at least 5000 images of malignant nodules;

the second process specifically comprises the following steps: carrying out image graying on the thyroid nodule image read in the first process, removing marks made by doctors for measuring nodule related quantity in the ultrasonic image by utilizing gray values of surrounding pixel points, denoising by utilizing Gaussian filtering, and finally carrying out equalization enhancement on contrast by utilizing a gray histogram to obtain a preprocessed enhanced image;

the third process specifically comprises the following steps:

step 1: selecting 10000 enhanced images preprocessed by the second process, wherein the enhanced images comprise 5000 good and malignant nodules;

step 2: for each picture, firstly, manually cutting out a nodule part and a non-nodule part, and then training an automatic segmentation model through CNN;

the CNN is a network structure consisting of 13 convolutional layers and 2 downsampling layers; the sizes of convolution kernels of the convolutional layers are respectively: the first layer was 13x13, the second and third layers were 5x5, and the remaining layers were 3x 3; the step sizes of the convolutional layers are respectively as follows: the first two convolutional layers are 2, the remainder are 1; the size of the down-sampling layers is 3x3, and the step size is 2;

the specific method for training the automatic segmentation model through the CNN comprises the following steps:

(1) the method comprises the following steps of automatically learning features through a CNN convolution layer and a down-sampling layer, and extracting the features, wherein the method comprises the following specific steps:

step A: on a convolution layer, the feature maps of the upper layer are convoluted by a convolution kernel which can be learned, and then an output feature map can be obtained through an activation function; each output is the value of a convolution kernel convolving one input or combining multiple convolved inputs:

$x_j^l=f({\mathrm{\Σ}}_{i\∈M_j}{x}_i^{l-1}*k_{ij}^l+b_j^l)$

wherein the symbol denotes the convolution operator; the l represents the number of layers; the i represents the ith neuron node of the l-1 layer; the j represents the jth neuron node of the l layer; the M is_jRepresenting a set of selected input maps; the above-mentionedRefers to the output of the l-1 layer as the input of the l layer; f is an activation function, here a sigmoid functionAs an activation function, e denotes the euler number 2.718281828, e^xIs an exponential function; the k is a convolution operator; b is a bias; each output map is given an additional offset b, but the convolution kernel that convolves each input map is different for a particular output map;

this step also requires a gradient calculation to update the sensitivity, which is used to indicate how much b changes, how much the error will change:

wherein l represents the number of layers; the j represents the jth neuron node of the l layer; o represents each element multiplication; the sensitivity of the representation output neuron, i.e. the rate of change of the bias b; s is^l＝W^lx^l-1+b^l，x^l-1Refers to the output of the l-1 layer, W is the weight, b is the offset; f is an activation function, here a sigmoid functionAs an activation function, e denotes the euler number 2.718281828, e^xIs an exponential function; f' (x) is the derivative function of f (x); the above-mentionedRepresenting the weight shared by each layer; the up (.) represents an upsample operation;

then sum all nodes in the sensitivity map in layer l, fast calculate the gradient of bias b:

$\frac{\∂E}{\∂b_j^l}={\mathrm{\Σ}}_{u,v}{\left({\mathrm{\δ}}_j^l\right)}_{u,v}$

wherein l represents the number of layers; the j represents the jth neuron node of the l layer; b represents a bias; the sensitivity of the representation output neuron, i.e. the rate of change of the bias b; the u, v represents the (u, v) position of the output maps; said E is an error function, whereThe C represents the dimension of the label, and if the problem is two-classification, the label can be marked as y_h∈ {0,1}, where C is 1, it can also be written as y_h∈ { (0,1), (1,0) }, when C ═ 2; saidThe h dimension of the label corresponding to the nth sample is represented; the above-mentionedAn h output representing a network output corresponding to the n sample;

and finally, calculating the weight of the convolution kernel by using a BP algorithm:

${\mathrm{\ΔW}}^l=-\mathrm{\η}\frac{\∂E}{\∂W^l}$

wherein W is a weight parameter; said E is an error function, andthe C represents the dimension of the label, and if the problem is two-classification, the label can be marked as y_h∈ {0,1}, where C is 1, it can also be written as y_h∈ { (0,1), (1,0) }, when C ═ 2; saidThe h dimension of the label corresponding to the nth sample is represented; the above-mentionedThe h output representing the net output corresponding to the n sample, η being the learning rate, i.e. the step size, since the weights of many connections are shared, for a given weight, it is necessary to gradient the point for all connections associated with the weight and then sum the gradients:

$\frac{\∂E}{\∂k_{ij}^l}={\mathrm{\Σ}}_{u,v}{\left({\mathrm{\δ}}_j^l\right)}_{u,v}{\left(p_i^{l-1}\right)}_{u,v}$

wherein l represents the number of layers; the i represents the ith neuron node of the l layer; the j represents the jth neuron node of the l layer; b represents the bias, which represents the sensitivity of the output neuron, i.e. the rate of change of bias b; said u, v represents the output ma(u, v) position of ps; said E is an error function, whereThe C represents the dimension of the label, and if the problem is two-classification, the label can be marked as y_h∈ {0,1}, where C is 1, it can also be written as y_h∈ { (0,1), (1,0) }, when C ═ 2; saidThe h dimension of the label corresponding to the nth sample is represented; the above-mentionedAn h output representing a network output corresponding to the n sample; the above-mentionedIs a convolution kernel; the above-mentionedIs thatWhen convolved withThe value of the (u, v) position of the output convolution map is determined by the (u, v) position of the previous layer, which is the region block in all the pictures with the same size as the convolution kernel, and the element-by-element multiplication patchThe result of element-by-element multiplication;

and B: the down-sampling layer has N input maps, and has N output maps, and only if each output map is reduced, then has:

$x_j^l=f\left({\mathrm{\β}}_j^{l}down\right(x_j^{l-1})+b_j^l)$

wherein f is an activation function, here a sigmoid function is takenAs an activation function, e denotes the euler number 2.718281828, e^xIs an exponential function; the above-mentionedThe down (.) represents a down-sampling function, all pixels of different nxn blocks of the input image are summed, so that the output image is reduced by n times in two dimensions, and each output map corresponds to a weight parameter β belonging to the output map and an additive bias b;

parameters β and b are updated by the gradient descent method:

$\frac{\∂E}{\∂b_j}={\mathrm{\Σ}}_{u,v}{\left({\mathrm{\δ}}_j^l\right)}_{u,v}$

wherein the conv2 is a two-dimensional convolution operator; the rot180 is rotated 180 degrees; the 'full' means that complete convolution is performed; the l represents the number of layers; the i represents the ith neuron node of the l layer; the j represents the jth neuron node of the l layer; b represents a bias; the sensitivity of the representation output neuron, i.e. the rate of change of the bias b; the u, v represents the (u, v) position of the output maps; said E is an error function, the expression is as above, i.e.The C represents the dimension of the label, and if the problem is two-classification, the label can be marked as y_h∈ {0,1}, where C is 1, it can also be written as y_h∈ { (0,1), (1,0) }, when C ═ 2; saidThe h dimension of the label corresponding to the nth sample is represented; the above-mentionedRepresenting the h output of the network output corresponding to the n sample, β being a weight parameter, down representing a down-sampling function, andis the convolution kernel of layer l + 1; the above-mentionedIs the jth neuron node of the output of layer l-1; s is^l＝W^lx^l-1+b^lWhere W is a weight parameter, b is an offset,is s^lThe jth component of (a);

and C: the CNN automatically learns the combination of the feature maps, and the jth feature map combination is as follows:

$x_j^l=f\left(\underset{i=1}{\overset{N_{in}}{\mathrm{\Σ}}}{\mathrm{\α}}_{ij}\right(x_i^{l-1}*k_i^l)+b_j^l)$

s.t.∑_iα_ij＝1，and 0≤α_ij≤1.

wherein the symbol denotes the convolution operator; the l represents the number of layers; the i represents the ith neuron node of the l layer; the j represents the jth neuron node of the l layer; f is an activation function, here a sigmoid functionAs an activation function, e denotes the euler number 2.718281828, e^xIs an exponential function; the above-mentionedIs the ith component of the l-1 layer output; said N is_inRepresenting the number of maps input; the above-mentionedIs a convolution kernel; the above-mentionedIs an offset, said α_ijRepresenting the output map of the l-1 layer as the input of the l layer, and obtaining the weight or contribution of the ith input map of the jth output map by the l-1 layer;

(2) automatically identifying nodules by using the features extracted in the step (1) and combining Softmax, and determining a model for automatic segmentation; in particular, if a sample is given, the Softmax recognition procedure outputs a probability value indicating the probability that the sample belongs to a class of several classes, and the loss function is:

$J\left(\mathrm{\θ}\right)=-\frac{1}{m}\[\underset{i=1}{\overset{m}{\Σ}}\underset{j=1}{\overset{c}{\Σ}}1\{y^{\left(i\right)}=j\}log\frac{e^{{\mathrm{\θ}}_j^T{x}^{\left(i\right)}}}{{\mathrm{\Σ}}_{l=1}^c{e}^{{\mathrm{\θ}}_l^{T}x\left(i\right)}}\]+\frac{\mathrm{\λ}}{2}\underset{i=1}{\overset{c}{\Σ}}\underset{j=0}{\overset{n}{\Σ}}{\mathrm{\θ}}_{ij}^2$

wherein m represents a total of m samples; the c represents that the samples can be divided into c types in total; the above-mentionedIs a matrix, each row is the parameter corresponding to a category, namely weight and bias; said 1{. is an indicative function, i.e. when the value in the brace is true, the result of the function is 1, otherwiseThe result was 0; the lambda is a parameter for balancing the fidelity term and the regular term, wherein lambda is a positive number; the J (theta) refers to a loss function of the system; said e represents the Euler number 2.718281828, e^xIs an exponential function; the T is a transpose operator in the representation matrix calculation; log represents the natural logarithm, i.e., the logarithm based on the euler number; n represents the dimension of the weight and bias parameters; x is the number of⁽ⁱ⁾Is the ith dimension of the input vector; y is⁽ⁱ⁾Is the ith dimension of each sample label; then the gradient is used to solve:

${\▿}_{{\mathrm{\θ}}_j}J\left(\mathrm{\θ}\right)=-\frac{1}{m}\[\underset{i=1}{\overset{m}{\mathrm{\Σ}}}{x}^{\left(i\right)}\left(1\right\{y^{\left(i\right)}=j\}-p(y^{\left(i\right)}=j|x^{\left(i\right)};\mathrm{\θ}\left)\right)\]+{\mathrm{\λ\θ}}_j$

wherein,the m represents a total of m samples; the above-mentionedIs a matrix, each row is the parameter corresponding to a category, namely weight and bias; said 1{ } is an indicative function, i.e. when the value in the brace is true, the result of the function is 1, otherwise the result is 0; the lambda is a parameter for balancing the fidelity term and the regular term, wherein lambda is a positive number; the J (theta) refers to a loss function of the system;is the J (θ) derivative function; said e represents the Euler number 2.718281828, e^xIs an exponential function; the T is a transpose operator in the representation matrix calculation; log represents the natural logarithm, i.e., the logarithm based on the euler number; x is the number of⁽ⁱ⁾Is the ith dimension of the input vector; y is⁽ⁱ⁾Is the ith dimension of each sample label;

(3) automatically segmenting nodules of all thyroid glands by using CNN (CNN), namely distinguishing a nodule region from a non-nodule region, finding out the boundary of the nodule region, and refining the shapes of the segmented nodules, namely filling holes and removing the connection with the non-nodule region by using corrosion and expansion morphological operators;

and 3, step 3: automatically segmenting all thyroid nodule pictures by using the model obtained in the step 2 to obtain an ROI (region of interest), namely all benign and malignant nodules;

the fourth process specifically comprises the following steps: dividing the ROI automatically segmented in the third process into p groups on average, normalizing the data, namely extracting the characteristics of the nodules after automatically segmenting the nodules, and performing linear transformation on the characteristics to map a result value to [0,1 ];

the fifth process specifically comprises the following steps: using a CNN training recognition model to extract features of all ROIs;

then, a new Softmax classification is used, namely only two Softmax classifiers are used for solving the optimal value of a loss function, namely the optimization J (theta), wherein the class number c of the Softmax classifiers is equal to 2; the probability of belonging to a benign nodule or a malignant nodule can be obtained by a gradient descent method, and the specific process is the same as the method of the automatic segmentation process in the third process;

the sixth process specifically comprises the following steps: repeating the process five, namely selecting p-1 group of data for training each time for p group of data, and performing the rest tests to finally obtain the optimal parameters of the recognition model, thereby obtaining the auxiliary diagnosis system for automatically recognizing the benign and malignant thyroid nodules based on the deep convolutional neural network;

the thyroid nodule image to be identified is input into the auxiliary diagnosis system, and the benign and malignant diagnosis of the nodule can be obtained.