CN111192334A - Trainable compressed sensing module and image segmentation method - Google Patents

Trainable compressed sensing module and image segmentation method Download PDF

Info

Publication number
CN111192334A
CN111192334A CN202010002908.1A CN202010002908A CN111192334A CN 111192334 A CN111192334 A CN 111192334A CN 202010002908 A CN202010002908 A CN 202010002908A CN 111192334 A CN111192334 A CN 111192334A
Authority
CN
China
Prior art keywords
trainable
sensing module
network
network model
matrix
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010002908.1A
Other languages
Chinese (zh)
Other versions
CN111192334B (en
Inventor
向德辉
田海鸿
陈新建
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou University
Original Assignee
Suzhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou University filed Critical Suzhou University
Priority to CN202010002908.1A priority Critical patent/CN111192334B/en
Publication of CN111192334A publication Critical patent/CN111192334A/en
Application granted granted Critical
Publication of CN111192334B publication Critical patent/CN111192334B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/002—Image coding using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/004—Predictors, e.g. intraframe, interframe coding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10104—Positron emission tomography [PET]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30061—Lung
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a trainable compression sensing module and an image segmentation method, which comprise the following steps: constructing a network model with the trainable compression sensing module; and taking the three-dimensional PET image with the non-small cell lung tumor as a training data set to be brought into the network model to train the network, and obtaining the trained network. The method can realize information compression in the training process, remove redundant characteristic graphs and enhance effective characteristic graphs, and has the advantages of more accurate segmentation result, higher segmentation speed and less used parameter quantity.

Description

Trainable compressed sensing module and image segmentation method
Technical Field
The invention relates to the technical field of image processing, in particular to a trainable compression sensing module and an image segmentation method.
Background
In recent 50 years, lung cancer has become the most common malignant tumor in the world, and the incidence rate is high and top in the list of malignant tumors, and the death rate is the first. According to statistics, the incidence rate of lung tumor is the first of all tumors in men. PET (positron emission tomography) is capable of detecting the active processes and characteristics of metabolism of biological tissues from the level of molecules and cells, with a higher brightness on PET images for tissues with vigorous metabolism (e.g. tumors, heart, liver). PET images are therefore widely used for clinical lung tumor diagnosis. Accurate automatic segmentation can provide important assistance to the diagnosis of the doctor. Current lung tumor segmentation methods can be divided into non-learning based methods and learning based methods. Non-learning based methods typically rely on statistical distributions of intensities.
However, non-learning based segmentation methods have limited ability to cope with variability in tumor shape. Convolutional neural networks have rapidly proven to be the most advanced tool for processing various medical images in recent years. UNet and UNet-based improved networks perform well on medical image pixel-level segmentation tasks. However, when the segmentation target is very similar in brightness to the surrounding tissue structure, the segmentation effect of the convolutional neural network is poor. Since the lung tumor in the PET image is very bright and the brightness of other organs (e.g., heart, spine and liver) is similar to the tumor, when the convolutional neural network segmentation is used, the false positive result is very high.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a trainable compressed sensing module and an image segmentation method, which can realize information compression in the training process, remove redundant feature maps and enhance effective feature maps.
In order to solve the above technical problem, the present invention provides a trainable compressed sensing module, including the following steps:
obtaining three-dimensional matrix input of compressed sensing module
Figure BDA0002354159680000021
The input of the compression sensing module is from the output of the previous layer of the convolutional layer, wherein H is the height of the characteristic diagram, W is the width of the characteristic diagram, and C is the number of characteristic channels of the current layer;
inputting the three-dimensional matrix of the compressed sensing module
Figure BDA0002354159680000022
Making two-dimensional matrix deformationObtaining a deformed two-dimensional matrix
Figure BDA0002354159680000023
Two-dimensional matrix using singular value decomposition algorithm
Figure BDA0002354159680000024
Decomposition into X ═ U ∑ VT, wherein ,
Figure BDA0002354159680000025
Figure BDA0002354159680000026
is a left-singular orthogonal vector matrix that,
Figure BDA0002354159680000027
is a right singular orthogonal vector matrix and,
Figure BDA0002354159680000028
is a matrix of singular values;
setting a weight matrix
Figure BDA0002354159680000029
The weight matrix
Figure BDA00023541596800000210
The elements on the diagonal of (1) are trainable and updated, the values of the elements outside the diagonal are zero, and a new Y matrix is obtained
Figure BDA00023541596800000211
Using set weight matrices
Figure BDA00023541596800000212
Selecting singular values of sigma to realize two-dimensional rectangle X ═ U sigma VTThe information compression of (2) removes redundant features and enhances the available information.
The invention discloses an image segmentation method based on the trainable compressed sensing module, which comprises the following steps:
constructing a network model with the trainable compression sensing module;
acquiring a three-dimensional PET image training data set with non-small cell lung tumor, bringing the training data set into a network model to train a network, and acquiring the trained network.
Preferably, the constructing of the trainable compression induction module network model specifically includes:
the method comprises the steps of utilizing a full convolutional network as a basic framework of a network model, wherein the full convolutional network comprises three convolutional layers, three trainable compression sensing modules, three times of downsampling and three times of upsampling, and each convolutional layer of the full convolutional network comprises a normalization function, a convolution function and an activation function.
Preferably, the system further comprises a deep layer supervision mechanism, and the deep layer supervision mechanism specifically comprises the following steps:
adding an auxiliary branch after the trainable compression sensing module, wherein the auxiliary branch amplifies some lower-level and medium-level feature vectors by using deconvolution;
calculating the obtained feature vectors with the same size and the same complete size by utilizing a softmax activation function to obtain an additional dense prediction graph;
for the prediction graph of the auxiliary branch and the corresponding manual labeling graph, calculating a segmentation error between the prediction graph and the corresponding manual labeling graph by using a cross entropy cost function;
the cross-entropy losses of all the auxiliary branches and the cross-entropy loss of the last layer are combined to encourage gradient back-propagation and more efficient updating of parameters in each iteration.
Preferably, the method further comprises the following steps:
order to
Figure BDA0002354159680000031
The weight parameter of the first trainable compressed sensing module network model of the network model, wherein L is 1,2, …, L;
all trainable compression sensing modules are weighted by
Figure BDA0002354159680000032
With p (x)i(ii) a Φ) represents pixel x after the output of the last layer has passed through the softmax functioniThe cross entropy loss can then be expressed as:
Figure BDA0002354159680000033
wherein ,
Figure BDA0002354159680000034
representing a training data set, pixel yiIs associated with pixel xiThe element belongs to the segmentation label of the pixel corresponding to the X, the second term of the formula is normalized by weight, and lambda is an adjustable hyper-parameter.
Preferably, in the "deep supervision mechanism",
by using
Figure BDA0002354159680000041
Weight representing the first d trainable compressed sensing modules of the main network, p (x)i；Φd) The output representing the d-th branch is also after the softmax function pixel xiA predicted probability of (d);
the deep supervision mechanism
Figure BDA0002354159680000042
And optimizing the formula to obtain the optimal weight phi, thereby supervising and training the compressed sensing module to extract the key effective characteristic diagram.
Preferably, the output of the last layer of the Focal local supervision network model is used.
Preferably, the "output of the last layer using the Focal local supervision network model" specifically includes:
network loss function
Figure BDA0002354159680000043
wherein pk＝yip(xi) is the prediction probability of the kth class, α is a balance factor used to balance positive and negative samples, gamma is a weight factor, gamma is>0 can reduce the loss of easily-divided samples;
total loss function
Figure BDA0002354159680000044
wherein ,
Figure BDA0002354159680000047
cross entropy loss for the last layer output of the network model;
Figure BDA0002354159680000048
is the cross entropy loss, η, of the middle auxiliary branchdIs that
Figure BDA0002354159680000045
Is a balanced weight of
Figure BDA0002354159680000046
D is the number of branches with deep supervision.
Preferably, the "acquiring a three-dimensional PET image training data set with a non-small cell lung tumor" specifically includes: and (3) cutting each three-dimensional PET image into two-dimensional slice images along the Z-axis direction as the input of a convolutional neural network, and performing data amplification by using overturning, rotating and translating in the length and width directions.
The invention has the beneficial effects that:
1. the invention provides a trainable compressed sensing module CSM which can realize information compression in a training process, remove redundant feature maps and enhance effective feature maps;
2. the full convolution neural network provided by the invention can extract key features through the trainable CSM and further strengthen the key features in a convolution mode, so that a more excellent segmentation effect is obtained;
3. the invention provides a deep monitoring mechanism for monitoring weight parameters in a trainable compression sensing module CSM, which can guide the CSM to extract features in the training process;
4. compared with the existing method, the method provided by the invention has the advantages of more accurate segmentation result, higher segmentation speed and less used parameter amount.
Drawings
FIG. 1 is a block diagram of a trainable compressed sensing module CSM of the present invention;
FIG. 2 is a graph comparing effects, wherein (a) is a visual graph without the addition of a CSM module; (b) is a visual effect graph after adding a CSM module.
FIG. 3 is a block diagram of the overall network architecture of the present invention;
FIG. 4 is a PET image of a human body at various angles, wherein (a) is a sectional view of a coronal plane of a three-dimensional PET image; (b) a cross-sectional view of a horizontal plane of a three-dimensional PET image at the time of drawing; (c) the figure shows a section of a sagittal plane of a three-dimensional PET image;
FIG. 5 is a comparison graph of different segmentation methods.
Detailed Description
The present invention is further described below in conjunction with the following figures and specific examples so that those skilled in the art may better understand the present invention and practice it, but the examples are not intended to limit the present invention.
Example one
Referring to fig. 1, the invention proposes a trainable compressed sensing module CSM based on the idea of principal component analysis PCA, comprising the following steps:
step one, acquiring three-dimensional matrix input of a compression induction module
Figure BDA0002354159680000051
The input of the compression sensing module is from the output of the previous convolution layer, wherein H is the height of the characteristic diagram, W is the width of the characteristic diagram, and C is the channel number of the characteristic diagram;
step two, inputting the three-dimensional matrix of the compressed sensing module
Figure BDA0002354159680000061
Performing two-dimensional matrix deformation to obtain a deformed two-dimensional matrix
Figure BDA0002354159680000062
Where N × W is the total number of pixels on each feature map;
step three, utilizing a singular value decomposition algorithm to divide the two-dimensional matrix into two-dimensional matrixes
Figure BDA0002354159680000063
Decomposition into X ═ U ∑ VT, wherein ,
Figure BDA0002354159680000064
is a left-singular orthogonal vector matrix that,
Figure BDA0002354159680000065
is a right singular orthogonal vector matrix and,
Figure BDA0002354159680000066
is a singular value matrix whose diagonal elements are all non-negative real numbers and whose values of the elements except the diagonal are all zero. Defining the elements on the diagonal as
Figure BDA0002354159680000067
wherein λiRepresenting the ith singular value. The C singular values are arranged on the diagonal in descending order. The covariance matrix can be written as:
XXT＝U∑VTV∑TUT＝U∑∑TUT(1)
where T denotes a matrix transpose. SigmaTIs a diagonal matrix. Thus the singular value decomposition of X is equivalent to XXTEigenspace decomposition of (a). Singular value of X XXTIs the square root of the eigenvalue of (a), the left singular vector of X is XXTThe feature vector of (2).
Step four, utilizing the matrix Yr＝U∑rFor X ═ U ∑ VTPerforming dimensionality reduction, wherein YrObtained after using a singular value decomposition algorithmTwo-dimensional matrix, r representing the first r number of singular values, ΣrPresentation selection
Figure BDA0002354159680000068
The first r columns of the singular value matrix;
since eigenvalues obtained by singular value decomposition are arranged in descending order, a weight matrix is set
Figure BDA0002354159680000069
The weight matrix
Figure BDA00023541596800000610
The elements on the diagonal of (1) are trainable and updated, the elements outside the diagonal have values of zero, and the elements outside the diagonal have values of zero. Thus, the new Y matrix can be defined as:
Figure BDA00023541596800000611
using defined weight matrices
Figure BDA00023541596800000612
The CSM module can realize information compression in the training process, remove redundant features and enhance effective information. After network training, although the number of the convolution characteristic maps of each layer is not changed, redundant information in each layer is removed. The feature maps of each layer become sparse and uncorrelated with each other.
Each CSM module may be understood as an information compression algorithm for transforming the originally related feature map into a set of linearly uncorrelated feature maps, called principal components. This is consistent with the signature that is irrelevant for our goal. The CSM provided by the invention has the characteristic extraction effect diagram as shown in FIG. 2.
Example two
Referring to fig. 3, the invention discloses an image segmentation method, based on the trainable compressed sensing module, including the following steps:
step one, constructing a trainable compression induction module network model; the method specifically comprises the following steps: the full convolutional network FCN is used as a basic framework of a network model, wherein the full convolutional network comprises three convolutional layers, three trainable compressed sensing modules CSM, three downsamplings and three upsamplings, and each convolutional layer of the full convolutional network comprises normalization (batch normalization), convolution (convolution kernel size is 3 × 3, sliding step size is 1) and an activation function (ReLU). The downsampling of the full convolution network adopts a convolution mode with the sliding step length of 2 and the convolution kernel size of 3 multiplied by 3. A trainable Compressed Sensing Module (CSM) is added behind each convolution layer to remove redundant features and extract valid features.
The invention also comprises a deep supervision mechanism, which specifically comprises the following steps: auxiliary branches are added behind the trainable compression sensing module, and the auxiliary branches amplify some feature vectors of lower levels and medium levels by deconvolution (the convolution kernel size is 3 x 3, and the sliding step length is 2, 4 and 8 respectively); calculating the obtained feature vectors with the same size and the same complete size by utilizing a softmax activation function to obtain an additional dense prediction graph; for the prediction graph of the auxiliary branch and the corresponding manual labeling graph, calculating a segmentation error between the prediction graph and the corresponding manual labeling graph by using a cross entropy cost function; the cross-entropy penalties of all the auxiliary branches and the cross-entropy penalty of the last layer are combined to excite gradient back propagation so as to more efficiently update the parameters in each iteration.
Order to
Figure BDA0002354159680000071
The weight parameter of the first trainable compressed sensing module network model of the network model, wherein L is 1,2, …, L; all trainable compression sensing modules are weighted by
Figure BDA0002354159680000072
With p (x)i(ii) a Φ) represents pixel x after the output of the last layer has passed through the softmax functioniThe cross entropy loss can then be expressed as:
Figure BDA0002354159680000081
wherein ,
Figure BDA0002354159680000082
representing a training data set, pixel yiIs associated with pixel xiThe element belongs to the segmentation label of the pixel corresponding to the X, the second term of the formula is normalized by weight, and lambda is an adjustable hyper-parameter.
On the other hand, in the "deep supervision mechanism", use
Figure BDA0002354159680000083
Weight representing the first d trainable compressed sensing modules of the main network, p (x)i；Φd) The output representing the d-th branch is also after the softmax function pixel xiA predicted probability of (d);
the deep supervision mechanism
Figure BDA0002354159680000084
By optimizing the formula, the optimal weight phi can be obtained, so that the trainable compressed sensing module CSM provided by the invention is supervised to extract a key effective characteristic diagram.
since the segmentation target monitors the output of the last layer of the network using Focal local less than a fraction of the slice, as shown in equation (5), better performance is obtained by adjusting the hyper-parameters α and γ.
Figure BDA0002354159680000085
wherein pk＝yip(xi) is the prediction probability of the kth class, α is a balance factor used to balance positive and negative samples, gamma is a weight factor, gamma is>0 can reduce the loss of easily dividable samples.
The overall loss function consists of three parts: the cross entropy Loss of the last layer output, the cross entropy Loss of the middle auxiliary branch and the Focal Loss. The total loss function is shown in equation 6.
Figure BDA0002354159680000086
wherein ,
Figure BDA00023541596800000811
cross entropy loss for the last layer output of the network model;
Figure BDA00023541596800000812
is the cross entropy loss, η, of the middle auxiliary branchdIs that
Figure BDA0002354159680000087
Is a balanced weight of
Figure BDA0002354159680000088
D is the number of branches with deep supervision.
Figure BDA0002354159680000089
Figure BDA00023541596800000810
And step two, acquiring a three-dimensional PET image training data set with the non-small cell lung tumor, and bringing the training data set into a network model to train a network to obtain the trained network. As shown in fig. 4, PET images are taken at various angles of the human body.
Data used in the present invention is collected from a patient with a non-small cell lung tumor (NSCLC). The data set consisted of 54 three-dimensional (3D) PET images with physician labeling. The image size of each 3D PET image was 512X 60, and the voxel size was 0.234X 1mm3. In the present invention, 49 of them were used for training, 5 were used for testing, and each 3D pet image was sliced into two-dimensional (2D) slice images along the Z-axis direction as an input of the convolutional neural network. Taking into account data balance, using flipping, rotation, and translation in the length and width directionsAnd (5) data amplification. And finally, performing verification by using 13-fold cross validation.
The invention uses a Keras ImageDataGenerator for data amplification, with specific parameters being flip, rotation, width/height translation (True, 0.2, 0.2). The number of pictures per network was set to 4, and the network was trained using a random gradient descent (SGD) with a momentum of 0.9 and a weight decay of 0.0001. Using a "multivariate" learning rate strategy, in which the learning rate is multiplied by
Figure BDA0002354159680000091
power is 0.9, and initial learning rate is set to 4e-3。
The experimental results of the invention are as follows:
in order to quantitatively evaluate the performance of the method provided by the invention, the segmentation result is compared with a manual labeling chart according to the following four indexes: DSC coefficient (Dice similarity coefficient), Precision (Precision), true positive fraction TPF (true positive fraction) and false positive fraction FPF (false positive fraction). DSC calculates the overlap between the segmentation results and the manual annotation map and is defined as:
Figure BDA0002354159680000092
where TP is the number of divided pixels that are true positive, FP is the number of divided pixels that are false positive, and FN is the number of divided pixels that are false negative. The TPF, FPF and accuracy index calculation formula is as follows:
Figure BDA0002354159680000093
Figure BDA0002354159680000094
Figure BDA0002354159680000101
the present invention compares the segmentation results with others' methods, as shown in table 1, where Param represents the neural network parameters and M represents the megabits.
As shown in Table 1, the results of the present invention are compared with the results of the prior art methods.
TABLE 1
Methods DSC(％) Precision(％) TPF(％) FPF(％) Param
DenseNet 57.77±17.31 57.59±26.55 87.47±15.65 0.29±0.24 2.4M
CGAN 54.10±10.01 75.40±14.58 51.57±16.08 5.66±13.39 142M
Segcaps 58.98±23.21 50.03±23.43 97.89±3.01 3.29±3.46 1.4M
UNet 58.04±18.56 60.10±20.08 90.65±10.71 0.12±0.11 31M
SFC-FCN 79.63±7.99 86.83±7.14 92.05±5.81 0.02±0.01 1M
Fig. 5 shows a graph of the segmentation results of different methods. The first column is the original PET picture. The second column is the corresponding manual callout. Next, a graph of the segmentation results of CGAN, DenseNet, SegCaps, UNet follows. The last ones are graphs of the segmentation results of the method of the invention.
The invention has the following beneficial effects: (1) a trainable Compressed Sensing Module (CSM) is proposed; the method can realize information compression in the training process, remove redundant feature maps and enhance effective feature maps; (2) the proposed full convolution neural network can extract key features through a trainable Compressed Sensing Module (CSM) and further strengthen the key features through a convolution mode, so that a more excellent segmentation effect is obtained; (3) a deep supervision mechanism is provided for supervising weight parameters in a trainable Compressed Sensing Module (CSM), and the CSM can be guided to carry out feature extraction in the training process; (4) compared with the existing method, the method provided by the invention has the advantages of more accurate segmentation result, higher segmentation speed and less used parameter amount.
The above-mentioned embodiments are merely preferred embodiments for fully illustrating the present invention, and the scope of the present invention is not limited thereto. The equivalent substitution or change made by the technical personnel in the technical field on the basis of the invention is all within the protection scope of the invention. The protection scope of the invention is subject to the claims.

Claims (10)

1. A trainable compressed sensing module comprising the steps of:
obtaining three-dimensional matrix input of compressed sensing module
Figure FDA0002354159670000011
The input of the compression sensing module is from the output of the previous convolution layer, wherein H is the height of the characteristic diagram, W is the width of the characteristic diagram, and C is the channel number of the characteristic diagram;
inputting the three-dimensional matrix of the compressed sensing module
Figure FDA0002354159670000012
Performing two-dimensional matrix deformation to obtain a deformed two-dimensional matrix
Figure FDA0002354159670000013
Where N × W is the total number of pixels on each feature map;
two-dimensional matrix using singular value decomposition algorithm
Figure FDA0002354159670000014
Decomposition into X ═ U ∑ VT, wherein ,
Figure FDA0002354159670000015
Figure FDA0002354159670000016
is the left singular quadrature momentThe number of the arrays is determined,
Figure FDA0002354159670000017
is a right singular orthogonal vector matrix and,
Figure FDA0002354159670000018
is a matrix of singular values;
setting a weight matrix
Figure FDA0002354159670000019
The weight matrix
Figure FDA00023541596700000110
The elements on the diagonal of (1) are trainable and updated, the values of the elements outside the diagonal are zero, and a new Y matrix is obtained
Figure FDA00023541596700000111
Using set weight matrices
Figure FDA00023541596700000112
Selecting singular values of sigma to realize two-dimensional rectangle X ═ U sigma VTThe information compression of (2) removes redundant features and enhances the available information.
2. An image segmentation method based on sparse feature maps to reconstruct a full convolution network, comprising the trainable compressed sensing module of claim 1, comprising the steps of:
constructing a network model with the trainable compression sensing module;
acquiring a three-dimensional PET image training data set with non-small cell lung tumor, bringing the training data set into a network model to train a network, and acquiring the trained network.
3. The image segmentation method according to claim 2, wherein constructing the trainable compression sensing module network model specifically comprises:
the method comprises the steps of utilizing a full convolutional network as a basic framework of a network model, wherein the full convolutional network comprises three convolutional layers, three trainable compression sensing modules, three times of downsampling and three times of upsampling, and each convolutional layer of the full convolutional network comprises a normalization function, a convolution function and an activation function.
4. The image segmentation method according to claim 3, further comprising a deep supervision mechanism, the deep supervision mechanism comprising in particular the steps of:
adding an auxiliary branch after the trainable compression sensing module, wherein the auxiliary branch amplifies some lower-level and medium-level feature vectors by using deconvolution;
calculating the obtained feature vectors with the same size and the same complete size by utilizing a softmax activation function to obtain an additional dense prediction graph;
for the prediction graph of the auxiliary branch and the corresponding manual labeling graph, calculating a segmentation error between the prediction graph and the corresponding manual labeling graph by using a cross entropy cost function;
the cross-entropy losses of all the auxiliary branches and the cross-entropy loss of the last layer are combined to encourage gradient back-propagation and more efficient updating of parameters in each iteration.
5. The image segmentation method of claim 4, further comprising:
order to
Figure FDA0002354159670000021
The weight parameter of the first trainable compressed sensing module network model of the network model, wherein L is 1,2, …, L;
all trainable compression sensing modules are weighted by
Figure FDA0002354159670000022
With p (x)i(ii) a Phi) representsPixel x after the output of the last layer passes through the softmax functioniThe cross entropy loss can then be expressed as:
Figure FDA0002354159670000023
wherein ,
Figure FDA0002354159670000024
representing a training data set, pixel yiIs associated with pixel xiThe element belongs to the segmentation label of the pixel corresponding to the X, the second term of the formula is normalized by weight, and lambda is an adjustable hyper-parameter.
6. The image segmentation method according to claim 4, wherein in the deep supervision mechanism,
by using
Figure FDA0002354159670000031
Weight representing the first d trainable compressed sensing modules of the main network, p (x)i；Φd) The output representing the d-th branch is also after the softmax function pixel xiA predicted probability of (d);
the deep supervision mechanism
Figure FDA0002354159670000032
And optimizing the formula to obtain the optimal weight phi, thereby supervising and training the compressed sensing module to extract the key effective characteristic diagram.
7. The image segmentation method of claim 4 wherein the output of the last layer of the Focal local supervised network model is used.
8. The image segmentation method according to claim 7, wherein the "output of the last layer using a Focal local supervised network model" specifically includes:
network loss function
Figure FDA0002354159670000033
wherein pk＝yip(xi) is the prediction probability of the kth class, α is a balance factor used to balance positive and negative samples, gamma is a weight factor, gamma is>0 can reduce the loss of easily-divided samples;
total loss function
Figure FDA0002354159670000034
wherein ,
Figure FDA0002354159670000035
cross entropy loss for the last layer output of the network model;
Figure FDA0002354159670000036
is the cross entropy loss, η, of the middle auxiliary branchdIs that
Figure FDA0002354159670000037
Is a balanced weight of
Figure FDA0002354159670000038
D is the number of branches with deep supervision.
9. The image segmentation method as set forth in claim 2, wherein the "acquiring a three-dimensional PET image training dataset with a non-small cell lung tumor" specifically comprises: and (3) cutting each three-dimensional PET image into two-dimensional slice images along the Z-axis direction as the input of a convolutional neural network, and performing data amplification by using overturning, rotating and translating in the length and width directions.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the method of any of claims 1 to 9 are implemented when the program is executed by the memory.
CN202010002908.1A 2020-01-02 2020-01-02 Trainable compressed sensing module and image segmentation method Active CN111192334B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010002908.1A CN111192334B (en) 2020-01-02 2020-01-02 Trainable compressed sensing module and image segmentation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010002908.1A CN111192334B (en) 2020-01-02 2020-01-02 Trainable compressed sensing module and image segmentation method

Publications (2)

Publication Number Publication Date
CN111192334A true CN111192334A (en) 2020-05-22
CN111192334B CN111192334B (en) 2023-06-06

Family

ID=70710647

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010002908.1A Active CN111192334B (en) 2020-01-02 2020-01-02 Trainable compressed sensing module and image segmentation method

Country Status (1)

Country Link
CN (1) CN111192334B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112734762A (en) * 2020-12-31 2021-04-30 西华师范大学 Dual-path UNet network tumor segmentation method based on covariance self-attention mechanism

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103596010A (en) * 2013-11-20 2014-02-19 天津大学 Video coding and decoding system based on dictionary learning and compressed sensing
CN107610192A (en) * 2017-09-30 2018-01-19 西安电子科技大学 Adaptive observation compressed sensing image reconstructing method based on deep learning
WO2018120740A1 (en) * 2016-12-29 2018-07-05 深圳光启合众科技有限公司 Picture classification method, device and robot
CN108765255A (en) * 2018-05-31 2018-11-06 东南大学 Angular quantification index modulation image watermark System and method for based on compressed sensing technology
CN109168002A (en) * 2018-07-26 2019-01-08 西安电子科技大学 Vision signal measurement field estimation method based on compressed sensing and convolutional neural networks

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103596010A (en) * 2013-11-20 2014-02-19 天津大学 Video coding and decoding system based on dictionary learning and compressed sensing
WO2018120740A1 (en) * 2016-12-29 2018-07-05 深圳光启合众科技有限公司 Picture classification method, device and robot
CN108256544A (en) * 2016-12-29 2018-07-06 深圳光启合众科技有限公司 Picture classification method and device, robot
CN107610192A (en) * 2017-09-30 2018-01-19 西安电子科技大学 Adaptive observation compressed sensing image reconstructing method based on deep learning
CN108765255A (en) * 2018-05-31 2018-11-06 东南大学 Angular quantification index modulation image watermark System and method for based on compressed sensing technology
CN109168002A (en) * 2018-07-26 2019-01-08 西安电子科技大学 Vision signal measurement field estimation method based on compressed sensing and convolutional neural networks

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112734762A (en) * 2020-12-31 2021-04-30 西华师范大学 Dual-path UNet network tumor segmentation method based on covariance self-attention mechanism

Also Published As

Publication number Publication date
CN111192334B (en) 2023-06-06

Similar Documents

Publication Publication Date Title
Yu et al. Tensorizing GAN with high-order pooling for Alzheimer’s disease assessment
Li et al. Deep convolutional neural networks for imaging data based survival analysis of rectal cancer
Ma et al. Thyroid diagnosis from SPECT images using convolutional neural network with optimization
US11170502B2 (en) Method based on deep neural network to extract appearance and geometry features for pulmonary textures classification
US10366491B2 (en) Deep image-to-image recurrent network with shape basis for automatic vertebra labeling in large-scale 3D CT volumes
CN112270666A (en) Non-small cell lung cancer pathological section identification method based on deep convolutional neural network
Zhou et al. Multi-classification of skin diseases for dermoscopy images using deep learning
CN113436173B (en) Abdominal multi-organ segmentation modeling and segmentation method and system based on edge perception
CN115147600A (en) GBM multi-mode MR image segmentation method based on classifier weight converter
Li et al. Brain tumor segmentation using 3D generative adversarial networks
Ristea et al. CyTran: a cycle-consistent transformer with multi-level consistency for non-contrast to contrast CT translation
Li et al. Double attention U-Net for brain tumor MR image segmentation
Agarwal et al. Weakly supervised lesion co-segmentation on CT scans
CN114708212A (en) Heart image segmentation method based on SEA-Unet
CN114820450A (en) CT angiography image classification method suitable for Li's artificial liver treatment
CN114663445A (en) Three-dimensional heart image segmentation method based on multi-scale edge perception
Gao et al. Transformer based tooth classification from cone-beam computed tomography for dental charting
Zhang et al. MLP-based classification of COVID-19 and skin diseases
Sengun et al. Automatic liver segmentation from CT images using deep learning algorithms: a comparative study
CN111192334B (en) Trainable compressed sensing module and image segmentation method
CN112465118A (en) Low-rank generation type countermeasure network construction method for medical image generation
Wu et al. Human identification with dental panoramic images based on deep learning
Barhoumi et al. Efficient scopeformer: Towards scalable and rich feature extraction for intracranial hemorrhage detection
CN115100155A (en) Method and system for establishing radiation pneumonitis prediction model
Wang et al. Renal lesion classification in kidney CT images by seven-layer convolution neural network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant