WO2021017372A1 - 一种基于生成对抗网络的医学图像分割方法、系统及电子设备 - Google Patents

一种基于生成对抗网络的医学图像分割方法、系统及电子设备 Download PDF

Info

Publication number
WO2021017372A1
WO2021017372A1 PCT/CN2019/125428 CN2019125428W WO2021017372A1 WO 2021017372 A1 WO2021017372 A1 WO 2021017372A1 CN 2019125428 W CN2019125428 W CN 2019125428W WO 2021017372 A1 WO2021017372 A1 WO 2021017372A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
level
segmented
samples
pixel
Prior art date
Application number
PCT/CN2019/125428
Other languages
English (en)
French (fr)
Inventor
王书强
吴昆�
陈卓
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Publication of WO2021017372A1 publication Critical patent/WO2021017372A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30008—Bone
    • G06T2207/30012—Spine; Backbone
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker

Definitions

  • This application belongs to the technical field of medical image processing, and particularly relates to a medical image segmentation method, system and electronic equipment based on a generative confrontation network.
  • Medical imaging technology includes two parts: medical imaging and medical image processing.
  • Common medical imaging techniques mainly include MRI (Magnetic Resonance Imaging), computed tomography (CT), positron emission computed tomography (PET), ultrasound imaging (US) and X-ray imaging.
  • CT computed tomography
  • PET positron emission computed tomography
  • US ultrasound imaging
  • X-ray imaging Different imaging technologies have their own advantages in the diagnosis and treatment of different diseases. In specific clinical applications, it has gradually formed the selection of corresponding imaging technologies for the diagnosis and treatment of specific diseases.
  • magnetic resonance imaging can have excellent resolution for soft tissue imaging without ionizing radiation and other hazards, and has a wide range of applications in the diagnosis and treatment of brain and uterus.
  • the basic process of applying deep learning to medical image processing is shown in Figure 1.
  • Generative Adversarial Networks is a deep learning model and one of the most promising methods for unsupervised learning on complex distributions in recent years.
  • the model generates quite good output through mutual game learning of (at least) two modules in the framework: Generative Model and Discriminative Model.
  • Generative Model Existing image segmentation models based on generative adversarial networks can be applied to cross-category object image segmentation, but in the field of medical images, this model has problems such as insufficient feature extraction and large amount of calculation for adversarial training.
  • This application provides a medical image segmentation method, system, and electronic device based on a generative confrontation network, which aims to solve at least one of the above technical problems in the prior art to a certain extent.
  • a medical image segmentation method based on generative adversarial network including the following steps:
  • Step a Collect pixel-level labeled samples of other medical images and image-level labeled samples of medical images to be segmented;
  • Step b training a capsule network-based generative confrontation network through the pixel-level labeled samples of the other medical images and the image-level labeled samples of the medical images to be segmented, the generative confrontation network including a generator and a discriminator;
  • Step c The generator performs pixel-level feature extraction on the pixel-level annotation samples of other medical images, and processes the image-level annotation samples of the medical image to be segmented through the pixel-level features to generate the pixel-level medical image to be segmented Labeling samples, and generating segmentation prediction samples of the medical image to be segmented based on the pixel-level labeling samples;
  • Step d Input the segmented prediction samples generated by the generator and the real labeled samples of the image to be segmented into the discriminator for "generation-adversarial" training, to determine the authenticity of the segmented prediction samples, and generate according to the error function Optimize the detector and discriminator to obtain a trained generative confrontation network;
  • Step e input the image-level annotated medical image to be segmented into the trained generation confrontation network, and output the pixel-level segmented image of the medical image to be segmented through the generation confrontation network.
  • the technical solution adopted by the embodiment of the application further includes: in the step c, the generator includes a capsule network module and a region positioning network, and the generator generates a segmentation prediction sample of the medical image to be segmented specifically includes:
  • Step b1 Pre-train the capsule network module with pixel-level labeled samples of other medical images to obtain non-semantic label samples, and process the image-level labeled samples of the image to be segmented through the non-semantic label samples to distinguish the image to be segmented Image-level annotation of the background and effective segmentation area of the sample;
  • Step b2 input the image-level annotation samples of the image to be segmented into the pre-trained capsule network module, and output the reconstructed image of the image-level annotation samples of the image to be segmented through the capsule network module;
  • Step b3 The regional positioning network uses the feature extraction of the convolutional layer to generate the feature map containing the position information of the image-level labeled samples of the image to be segmented, and uses the global average pooling layer to combine the weights (w 1 , w 2 ..., w n ) Perform a weighted average with the feature map to obtain the regional positioning feature map of the image-level labeled samples of the image to be segmented;
  • Step b4 Perform a self-diffusion algorithm according to the reconstructed image and the regional positioning feature map, determine the regional pixel point segmentation line, and obtain segmentation prediction samples of the image-level labeled samples of the image to be segmented.
  • the technical solution adopted by the embodiment of the application further includes: in the step b2, the capsule network module includes a convolutional layer, a PrimaryCaps layer, a DigitCaps layer, and a decoding layer, and the capsule network module uses the output vector of a single capsule neuron , Record the direction and position information of the edge pixels of the image-level label sample segmentation area of the image to be segmented, use the nonlinear activation function of the vector to extract the classification probability value, determine the segmentation area and background of the image-level label sample of the image to be segmented, and calculate the edge Loss and output the reconstructed image of the image-level labeled samples of the image to be segmented.
  • the technical solution adopted in the embodiment of the present application further includes: in the step b4, the executing the self-diffusion algorithm based on the reconstructed image and the regional positioning feature map specifically includes: using a random walk in the region with a larger activation value in the regional positioning feature map
  • the self-diffusion algorithm diffuses the pixels, uses the input points of the regional positioning feature map, calculates the Gaussian distance from each pixel on the image to the input point, and selects the optimal path from it, obtains the segmentation line of the regional pixel, and finally generates the segmentation prediction sample.
  • the discriminator includes a cascaded Cascade module, a Capsule network module, and a parameter optimization module, and the "generate-confrontation" training performed by the discriminator specifically includes :
  • Step d1 Extract the incorrectly labeled pixels in the segmentation prediction sample and the key pixels whose confidence is lower than the set threshold and the corresponding ground truth through the cascaded Cascade module, and filter the pixels that are correctly labeled and whose confidence is higher than the set threshold ;
  • Step d2 Process the extracted key pixels and the corresponding ground truth through the Capsule network module, and generate errors;
  • Step d3 The parameter optimization module uses the error generated by the Capsule network module to optimize the network parameters of the generator and the discriminator; wherein, for a given segmentation prediction sample ⁇ I f ,L f* ⁇ and the corresponding real labeled sample ⁇ I f ,L f ⁇ , the overall error function of the network is:
  • a medical image segmentation system based on a generative confrontation network, including a sample collection module and a generative confrontation network,
  • Sample collection module used to collect pixel-level labeled samples of other medical images and image-level labeled samples of medical images to be segmented;
  • the generative confrontation network includes a generator and a discriminator.
  • the generator performs pixel-level feature extraction on pixel-level annotation samples of other medical images, and processes the image-level annotation samples of the medical image to be segmented through the pixel-level features to generate Generating a pixel-level labeled sample of the medical image to be segmented, and generating a segmentation prediction sample of the medical image to be segmented based on the pixel-level labeled sample;
  • the segmentation prediction samples generated by the generator and the real labeled samples of the image to be segmented are input to the discriminator for "generation-antagonism" training, to determine the authenticity of the segmentation prediction samples, and the generator and the discrimination are determined according to the error function Optimize the device to obtain a trained Generative Adversarial Network;
  • the image-level annotated medical image to be segmented is input into a trained generation confrontation network, and the pixel-level segmented image of the medical image to be segmented is output through the generation confrontation network.
  • the generator includes a pre-training module, a capsule network module, a regional positioning network module, and a sample generation module:
  • Pre-training module used to pre-train the capsule network module with pixel-level labeled samples of other medical images to obtain non-semantic label samples, and process the image-level labeled samples of the image to be segmented through the non-semantic label samples to distinguish the The background and effective segmentation area of the image-level annotated sample of the image to be segmented;
  • Capsule network module used to input the image-level annotation samples of the image to be segmented into the pre-trained capsule network module, and output the reconstructed image of the image-level annotation samples of the image to be segmented through the capsule network module;
  • Regional positioning network used to use the feature extraction of the convolutional layer to generate the feature map containing the location information of the image-level labeled samples of the image to be segmented, and use the global average pooling layer to combine the weights (w 1 ,w 2 ...,w n ) Perform a weighted average with the feature map to obtain the regional positioning feature map of the image-level labeled samples of the image to be segmented;
  • Sample generation module used to execute a self-diffusion algorithm according to the reconstructed image and the regional positioning feature map, determine the regional pixel point segmentation line, and obtain the segmentation prediction samples of the image-level labeled samples of the image to be segmented.
  • the technical solution adopted in the embodiment of the application further includes: the capsule network module includes a convolutional layer, a PrimaryCaps layer, a DigitCaps layer, and a decoding layer, and the capsule network module uses the output vector of a single capsule neuron to record the image of the image to be segmented
  • the direction and location information of the edge pixels of the segmented area of the sample is used to extract the probability value of the classification using the nonlinear activation function of the vector, the segmentation area and background of the image-level labeling sample of the image to be segmented are determined, and the edge loss is calculated and output of the image to be segmented
  • the reconstructed image of the image-level annotated sample is a convolutional layer, a PrimaryCaps layer, a DigitCaps layer, and a decoding layer
  • the capsule network module uses the output vector of a single capsule neuron to record the image of the image to be segmented
  • the direction and location information of the edge pixels of the segmented area of the sample is used
  • the technical solution adopted in the embodiment of the present application further includes: the sample generation module executes the self-diffusion algorithm according to the reconstructed image and the regional positioning feature map, specifically including: using the self-diffusion algorithm of random walk in the region with the larger activation value in the regional positioning feature map Diffusion pixel points, using the input points of the regional positioning feature map, calculate the Gaussian distance from each pixel on the image to the input point, and select the optimal path from it to obtain the segmentation line of the regional pixel points, and finally generate segmentation prediction samples.
  • the discriminator includes a cascade Cascade module, a Capsule network module, and a parameter optimization module:
  • Cascade Cascade module used to extract incorrectly labeled pixels and key pixels whose confidence is lower than a set threshold and the corresponding ground truth in the segmentation prediction sample, and filter pixels that are correctly labeled and whose confidence is higher than the set threshold;
  • Capsule network module used to process the extracted key pixels and the corresponding ground truth, and generate errors
  • Parameter optimization module used to optimize the network parameters of the generator and the discriminator using the error generated by the Capsule network module; among them, for a given segmentation prediction sample ⁇ I f ,L f* ⁇ and the corresponding real labeled sample ⁇ I f ,L f ⁇ , the overall error function of the network is:
  • an electronic device including:
  • At least one processor At least one processor
  • a memory communicatively connected with the at least one processor; wherein,
  • the memory stores instructions that can be executed by the one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the aforementioned medical image segmentation method based on generating a confrontation network.
  • Step a Collect pixel-level labeled samples of other medical images and image-level labeled samples of medical images to be segmented;
  • Step b Train a capsule network-based generative confrontation network through the pixel-level labeled samples of the other medical images and the image-level labeled samples of the medical images to be segmented, the generative confrontation network including a generator and a discriminator;
  • Step c The generator performs pixel-level feature extraction on the pixel-level annotation samples of other medical images, and processes the image-level annotation samples of the medical image to be segmented through the pixel-level features to generate the pixel-level medical image to be segmented Labeling samples, and generating segmentation prediction samples of the medical image to be segmented based on the pixel-level labeling samples;
  • Step d Input the segmented prediction samples generated by the generator and the real labeled samples of the image to be segmented into the discriminator for "generation-adversarial" training, to determine the authenticity of the segmented prediction samples, and generate according to the error function Optimize the detector and discriminator to obtain a trained generative confrontation network;
  • Step e input the image-level annotated medical image to be segmented into the trained generation confrontation network, and output the pixel-level segmented image of the medical image to be segmented through the generation confrontation network.
  • the beneficial effects produced by the embodiments of the present application are: the medical image segmentation method, system and electronic equipment based on the generation of confrontation network in the embodiments of the present application optimize the deep convolutional neural network through the fusion capsule mechanism, and integrate the Capsule
  • the idea of network and cascade waterfall in the case of small medical image samples, generate new training image samples, realize the semantic segmentation of low-quality medical image data with only image-level labels, and transfer the learned segmentation knowledge from pixels
  • Fully annotated data of level annotation is transferred to image-level weakly annotated data, thereby improving the feature expression ability of the model, expanding the applicability of medical image annotation samples, effectively reducing the dependence of the segmentation model on pixel-level annotation data, and having network information
  • With less redundancy and sufficient feature extraction under the premise of a small number of pixel-level labeled samples, it can not only improve the efficiency of generated samples and real samples against training, but also can effectively achieve high-precision pixel-level image segmentation.
  • Figure 1 is a basic flow chart of deep learning applied to medical image processing
  • FIG. 2 is a flowchart of a medical image segmentation method based on a generative confrontation network according to an embodiment of the present application
  • FIG. 3 is a schematic structural diagram of a generative confrontation network according to an embodiment of the present application.
  • Figure 4 is a schematic diagram of the structure of a capsule network module
  • Figure 5 is a schematic diagram of the network structure of the regional positioning network
  • FIG. 6 is a schematic structural diagram of a medical image segmentation system based on a generative confrontation network according to an embodiment of the present application
  • FIG. 7 is a schematic diagram of a hardware device structure of a medical image segmentation method based on a generative confrontation network provided by an embodiment of the present application.
  • the medical image segmentation method based on the generative confrontation network in the embodiment of the application improves the generation of the confrontation network through the fusion capsule mechanism.
  • the generator can perform pixel-level analysis of different types of high-quality images.
  • the feature is extracted, and the capsule model is used for structured feature representation to realize the generation of pixel-level labeled samples;
  • a suitable discriminator is constructed to determine the authenticity of the generated pixel-level labeled samples, and an appropriate error optimization function is designed.
  • the discrimination results are fed back to the models of the generator and the discriminator respectively.
  • the sample generation and discrimination capabilities of the generator and the discriminator are improved respectively.
  • the trained generator is used to generate pixel-level labeled samples to achieve Image-level annotation of pixel-level segmentation of medical images.
  • this application only takes the medical image segmentation of cervical spondylotic myelopathy (CSM) as an example for detailed description.
  • CSM cervical spondylotic myelopathy
  • the disease source is not limited to a single case, and can be extended to image segmentation scenes of multiple cases. For example, brain MRI image segmentation.
  • For image segmentation of different cases only the training samples of the corresponding cases need to be collected in the data collection stage, and the training samples are replaced in the model generator.
  • FIG. 2 is a flowchart of a medical image segmentation method based on a generative confrontation network according to an embodiment of the present application.
  • the medical image segmentation method based on the generative confrontation network in the embodiment of the present application includes the following steps:
  • Step 100 Collect pixel-level annotation samples of other medical images and CSM image-level annotation samples respectively;
  • other medical images include medical images of lungs and other parts.
  • the method of acquiring the labeled samples is specifically: collecting a small amount of fully labeled CSM image samples ⁇ I f , L f , T f ⁇ and a total of 500 DTI samples (diffusion tensor imaging, diffusion tensor imaging), the image size is 28x28, CSM group 60 cases, including 27 males and 33 females, aged 20-71 years old, average 45 years old, including pixel-level labeled samples and image-level labeled samples ⁇ L f ,T f ⁇ ; 8000 DTI samples of pixel-level labeled samples ⁇ I O ,L O ⁇ of other medical images (such as human lungs), the image size is 28x28.
  • the DTI samples of the CSM image and other medical images are obtained, and the region of interest (ROI) is determined on the DTI sample, and the ROI is placed in the center of the spinal cord lesion to ensure the uniformity of the DTI sample size , To avoid the influence of spinal fluid and artifacts.
  • ROI region of interest
  • FIG. 3 is a schematic structural diagram of a generative confrontation network according to an embodiment of the present application.
  • the generative confrontation network in the embodiment of the present application includes a generator and a discriminator, where the generator is responsible for generating pixel-level annotation data of medical images, and the discriminator is responsible for fine-graining the generated annotations.
  • the generator includes two parts: a capsule network module and a regional positioning network.
  • the pixel-level labeled samples ⁇ I O , L O ⁇ of other medical images are used as the pre-training samples of the capsule network module, and the CSM image-level labeled samples ⁇ L f , T f ⁇ is used as the training sample of the regional positioning network.
  • Step 200 Pre-train the capsule network module with pixel-level labeled samples of other medical images to obtain non-semantic labeled samples, and process CSM image-level labeled samples through non-semantic labeled samples to distinguish the background and background of CSM image-level labeled samples Effective segmentation area;
  • the capsule network module adopts a transferable semantic segmentation model, which can transfer the learned segmentation knowledge from the pixel-level annotation of the fully annotated data to the image-level annotation of the weakly annotated data.
  • this application uses a trained model to match the new data with the data of the target domain to obtain pixel-level segmentation features, so that the sample size is small.
  • high-precision image segmentation can also be achieved.
  • the pre-training process of the capsule network module specifically includes:
  • Step 201 processing pixel-level annotation samples of other medical images to turn them into non-semantic segmented image annotations
  • the non-semantic segmented image annotation that is, the annotation only distinguishes the effective segmentation area and background of the sample, but does not distinguish the shape of the sample.
  • the network trained by such training data learns the knowledge of distinguishing objects and backgrounds, which are broad high-level features, and for images, there are distinctions between objects and backgrounds, so broad high-level features have strong
  • the versatility can be easily transferred to other different tasks, so that the knowledge learned from high-quality pixel-level medical images can be transferred to low-quality image-level medical images, while high-quality The image does not need to be directly related to the low-quality image.
  • Step 202 Process the CSM image-level annotation samples according to the non-semantic segmentation image annotations of other medical images to generate non-semantic label samples ⁇ I O , L O ⁇ of the CSM image-level annotation samples, and obtain the pixel level by filtering the semantic information The non-semantic labeling L O ;
  • step 202 the purpose of obtaining pixel-level non-semantic annotations is to distinguish the effective segmentation area and background of the data, so that it is easier to transfer the learned knowledge in the strongly and weakly annotated data.
  • Step 300 Input the CSM image-level annotation samples ⁇ L f , T f ⁇ into the pre-trained capsule network module, and the capsule network module outputs the reconstructed image of the CSM image-level annotation samples;
  • the capsule network module uses the output vector of a single capsule neuron to record the direction and position information of the edge pixels of the CSM image-level annotation sample segmentation area, and uses the nonlinear activation function of the vector to extract the probability value of the classification to determine the CSM image-level annotation
  • the segmented area and background of the sample calculate the edge loss and output the reconstructed image of the CSM image-level annotated sample.
  • This application uses the capsule network module to instantiate the parameter information of the pixels and output the activation vector to record the position and angle information of the segmented area, which can effectively improve the sharpness of the boundary area of the segmented area.
  • the structure of the capsule network module is shown in Figure 4.
  • the model includes a convolutional layer, a PrimaryCaps layer, a DigitCaps layer and a decoding layer.
  • Each capsule represents a function and outputs an activated vector.
  • the length of the vector represents the probability that the region segmentation line sought by the capsule is correct.
  • the functions of each layer of the capsule network module are:
  • Convolutional layer Through convolution operation on CSM image-level labeled samples ⁇ L f , T f ⁇ , the primary features such as the morphology of the spinal cord and the compression position are obtained. Taking a CSM image with an input size of 28x28 as an example, the convolutional layer has 256 9x9x1 convolution kernels with a step length of 1, using the Relu activation function, after feature extraction of the convolutional layer, and outputting a feature tensor of 20x20x256.
  • PrimaryCaps layer contains 32 primary capsules, accepts the primary features obtained by the convolutional layer, and each capsule generates a combination of features and performs vectorized expression. Each dimension of the vector represents information such as the direction and position of the feature.
  • Each main capsule applies 8 9x9x256 convolution kernels to an input tensor of 20x20x256. Since there are 32 main capsules, the output is a 6x6x8x32 feature tensor.
  • DigitCaps layer Each digital capsule corresponds to the vector output by the PrimaryCaps layer. Each digital capsule accepts a 6x6x8x32 tensor as input, and uses dynamic routing to nest the output variables of each main capsule into a multi-layer digital capsule, and activate the vector
  • the key feature of is to map the 8-dimensional input space to the 16-dimensional capsule output space through an 8x16 weight matrix.
  • the decoding layer is the last fully connected layer of the network, including the Relu function and the Sigmoid function. It accepts the correct 16-dimensional vector output from the DigitCaps layer and uses it as input to learn the multiple features expressed by the capsule output vector, calculate edge loss, and learn reconstruction A 28x28 size image with the same pixels as the input image.
  • the loss function of the capsule network module is as follows:
  • O L ((I f ,T f ); ⁇ L ) represents the output of the capsule network module, ⁇ L represents the weight and parameters of the network training, and J b represents the binary cross-entropy loss function of the elements in the brackets .
  • S400 Perform pixel-level label prediction on CSM image-level labeled samples ⁇ I f , T f ⁇ through a regional positioning network, and output a regional positioning feature map;
  • step 400 the network structure of the regional positioning network is shown in FIG. 5.
  • the regional positioning network uses the feature extraction of the convolutional layer to generate a feature map containing location information, and uses a global average pooling layer to perform a weighted average of the weights (w 1 , w 2 ..., w n ) with the feature map to obtain regional positioning features Figure.
  • the region with larger activation value in the regional positioning feature map is most likely to be the segmentation position of the cervical spinal cord injury region.
  • the regional positioning network makes full use of the primary features obtained by the training sample after the convolutional layer operation to locate the hot spots of the feature map. Since the convolutional neural network needs to train many parameters, the repeated use of the feature map can reduce the amount of network parameters, so that The training efficiency of the model is higher.
  • Step 500 The sample generation module executes the self-diffusion algorithm according to the reconstructed image output by the capsule network and the regional positioning feature map output by the regional positioning network, determines the segmentation line of the regional pixel points, and obtains a coarser segmentation prediction sample ⁇ I f ,L f* ⁇ ;
  • step 500 in order to output the segmentation map M containing semantic information as segmentation samples labeled at the pixel level, the idea of random walk (Random Walk) is used in the area with the larger activation value, and the pixels are diffused through the self-diffusion algorithm of random walk.
  • the input point of the regional positioning feature map is calculated, the Gaussian distance from each pixel on the image to the input point is calculated, and the optimal path is selected from it, and the segmentation line of the regional pixel points is obtained, and the classification points are diffused from each larger activation value, and finally Generate coarser segmentation prediction samples ⁇ I f ,L f* ⁇ .
  • Zi ,j represents the Gaussian distance between two adjacent superpixels.
  • Step 600 Input the segmented prediction samples ⁇ I f ,L f* ⁇ output by the generator and the real labeled samples ⁇ I f ,L f ⁇ together into the discriminator for "generation-adversarial" training, and optimize the generator;
  • the discriminator uses the capsule network module to record the direction and position information of the segmentation area, improves the sharpness of the boundary area of the segmentation area, and uses the cascade mode to extract the key area pixels in the image that are difficult to correctly classify, and filter out simple and clear Use the processed image to perform “generation-adversarial” training until a Nash equilibrium is formed between the generator and the discriminator, and the discriminator cannot distinguish that the image comes from the segmented prediction sample generated by the generator ⁇ I f ,L f* ⁇ is still the real labeled sample ⁇ I f ,L f ⁇ to complete the training of the generative confrontation network.
  • the discriminator includes a cascaded Cascade module, a Capsule network module, and a parameter optimization module; the specific functions of each module are as follows:
  • Cascade Cascade module used to extract the key pixels of the segmentation prediction sample; in the process of image segmentation, the difficulty of marking each pixel is different, and the flat background area can be easily distinguished, but the object But it is difficult to distinguish the boundary pixels from the background area.
  • the previous network structure put all these pixels into the network for processing, resulting in unnecessary redundancy of the network.
  • this application adopts the idea of cascade Cascade, treats pixels differently, and focuses on key pixel areas that are difficult to classify.
  • the segmentation prediction samples generated by the generator are labeled incorrectly and the key pixels whose confidence is lower than a certain threshold and the corresponding ground truth are extracted, and the pixels that are correctly labeled and with high confidence are filtered out. In this way, the pixels input to the next stage of training are only key pixels that are not easy to distinguish, which can reduce redundant information in the network and improve the work efficiency of the network.
  • the Capsule network module is responsible for processing the extracted key pixels and the corresponding ground truth, and produce errors; specifically, the functions of the Capsule network module include:
  • Step 610 local feature extraction; input the key pixels and the corresponding ground truth as input into the corresponding convolutional layers; then use several convolution layers to convolve the input key pixels and the corresponding ground truth to extract
  • the low-level features in the segmentation prediction sample ⁇ I f ,L f* ⁇ are obtained; the activation function of the convolutional layer is the ReLU function.
  • Step 611 High-dimensional feature extraction; by constructing the PrimaryCaps layer, the extracted low-level features are input to the PrimaryCaps layer to obtain high-dimensional feature vectors containing spatial location information; the DigitCaps layer is constructed, and the output variables in the PrimaryCaps layer are embedded using dynamic routing. The set is mapped to the DigitCaps layer to construct the current high-level features that can best characterize all input features and input them to the next layer;
  • the calculation method between the PrimaryCaps layer and the DigitCaps layer involved in step 611 is as follows:
  • the feature vector extracted from the key pixel and the corresponding Ground truth after convolution in the convolutional layer is u i
  • the low-level feature vector u i is used as the input of the PrimaryCaps layer
  • the weight matrix Wij is multiplied to obtain the prediction vector among them:
  • the weighted sum S j can be obtained by linear combination between the prediction vectors, and the weight coefficient is c ij , where:
  • the length of the S j vector is limited by the compression function to obtain the output vector V j , where:
  • the first half is the scaling scale of the input vector S j
  • the second half is the unit vector of S j .
  • the coefficient c ij is constant
  • the calculation formula of c ij is:
  • Equation (7) b ij is a constant, b ij b ij values of the previous iteration by the value of V j and The sum of the products is obtained, that is, the update method of b ij is:
  • Step 612 The discriminator puts the high-level feature vector V output by the DigitCaps layer into the decoding layer, and finally outputs the judgment result of the authenticity of the image through several fully connected layers; specifically: if the output result is 0, the judgment is false, which means The input image is judged to be a fake image; if the output result is 1, the judgement is true, which means that the input image successfully confuses the discriminator.
  • Parameter optimization module Use the error generated by the Capsule network module to optimize the network parameters of the generator and the discriminator, so that the generator can output more optimized segmentation results.
  • Equation (9) ⁇ S ⁇ p, respectively and a parameter indicating the generator and a discriminator, J b represents the Mutual entropy loss function, O s O p, respectively, and the output of the generator and a discriminator.
  • the parameter optimization process includes two parts:
  • Step 620 Fix the generator parameter ⁇ S and optimize the discriminator parameter ⁇ p ; in the confrontation training process, first fix the generator parameter ⁇ S , and use the segmentation prediction sample generated by the generator to send to the discriminator, and the discriminator judges Authenticity, and use the discriminator error function to adjust the discriminator parameter ⁇ p through the back propagation algorithm to improve its own discrimination ability.
  • the error function corresponding to the discriminator is:
  • the parameters of the discriminator are continuously optimized, and the discriminative ability is continuously enhanced. It is more and more easy to distinguish the generated images of the generator, thus entering the next stage.
  • Step 621 Fix the discriminator parameter ⁇ p and optimize the generator parameter ⁇ S ; the network brings the discriminator’s discriminating result into the generator error function, and adjusts the generator parameter ⁇ S through the backpropagation algorithm to make the generator generate more High-quality segmentation results, so that the generator generates more accurate results to confuse the discriminator.
  • the error function corresponding to the generator is:
  • Step 700 Input the image-level annotated CSM image into the trained generation confrontation network, and output the pixel-level segmentation image of the CSM image through the generation confrontation network.
  • FIG. 6 is a schematic structural diagram of a medical image segmentation system based on a generative confrontation network according to an embodiment of the present application.
  • the medical image segmentation system based on the generative confrontation network in the embodiment of the present application includes a sample acquisition module and a generation confrontation network.
  • the generation confrontation network is trained through the image samples collected by the sample acquisition module.
  • the generation confrontation network includes a generator and a discriminator.
  • the generator uses The capsule model performs structured feature representation, and then realizes the generation of pixel-level labeled samples.
  • the discriminator is used to determine the authenticity of the generated pixel-level labeled samples, and design an appropriate error optimization function, and the discrimination results are fed back to the generator and the discriminator.
  • the sample generation ability and discrimination ability of the generator and the discriminator are improved respectively, and finally the trained generator is used to generate pixel-level labeled samples to achieve pixel-level segmentation of image-level labeled medical images.
  • Sample collection module used to separately collect pixel-level labeled samples of other medical images and CSM image-level labeled samples; among them, other medical images include medical images of lungs and other parts, and the specific method of obtaining labeled samples is: collect a small amount of fully labeled
  • a total of 500 DTI samples (diffusion tensor imaging) of CSM image samples ⁇ I f ,L f ,T f ⁇ , image size is 28x28
  • CSM group 60 cases including 27 males and 33 females, age 20 ⁇ 71 years old, average 45 years old, including pixel-level labeled samples and image-level labeled samples ⁇ L f ,T f ⁇ ; other medical images (such as human lungs) pixel-level labeled samples ⁇ I O ,L O ⁇ 8000
  • One DTI sample, the image size is 28x28.
  • the DTI samples of the CSM image and other medical images are obtained, and the region of interest (ROI) is determined on the DTI sample, and the ROI is placed in the center of the spinal cord lesion to ensure the uniformity of the DTI sample size , To avoid the influence of spinal fluid and artifacts.
  • ROI region of interest
  • the generator includes a pre-training module, a capsule network module, a regional positioning network module, and a sample generation module.
  • the functions of each module are as follows:
  • Pre-training module used to pre-train the capsule network module with pixel-level labeled samples of other medical images to obtain non-semantic labeled samples, and process CSM image-level labeled samples through non-semantic labeled samples to distinguish CSM image-level labeled samples
  • the background and effective segmentation area of, among them, the capsule network module adopts a transferable semantic segmentation model, which can transfer the learned segmentation knowledge from the pixel-level annotation of fully annotated data to the image-level annotation of weakly annotated data.
  • this application uses a trained model to match the new data with the data of the target domain to obtain pixel-level segmentation features, so that the sample size is small. In the same case, high-precision image segmentation can also be achieved.
  • the pre-training process of the capsule network module specifically includes:
  • the CSM image-level annotation samples are processed to generate non-semantic label samples ⁇ I O , L O ⁇ of the CSM image-level annotation samples, and the semantic information is filtered to obtain the pixel-level Non-semantic labeling L O ; among them, the purpose of obtaining pixel-level non-semantic labeling is to distinguish the effective segmentation area and background of the data, so that it is easier to transfer the learned knowledge in the strong and weak labeled data.
  • Capsule network module used to input CSM image-level labeled samples ⁇ L f ,T f ⁇ into the pre-trained capsule network module, and the capsule network module outputs reconstructed images of CSM image-level labeled samples; among them, the capsule network module uses a single capsule
  • the output vector of the neuron records the direction and position information of the edge pixels of the segmentation area of the CSM image-level annotation sample, and uses the nonlinear activation function of the vector to extract the classification probability value, determines the segmentation area and background of the CSM image-level annotation sample, and calculates the edge loss And output the reconstructed image of CSM image-level annotated samples.
  • This application uses the capsule network module to instantiate the parameter information of the pixels and output the activation vector to record the position and angle information of the segmented area, which can effectively improve the sharpness of the boundary area of the segmented area.
  • the capsule network module includes a convolutional layer, a PrimaryCaps layer, a DigitCaps layer and a decoding layer.
  • Each capsule represents a function and outputs an activation vector.
  • the length of the vector represents the probability that the region segmentation line sought by the capsule is correct.
  • the functions of each layer of the capsule network module are:
  • Convolutional layer Through convolution operation on CSM image-level labeled samples ⁇ L f , T f ⁇ , the primary features such as the morphology of the spinal cord and the compression position are obtained. Taking a CSM image with an input size of 28x28 as an example, the convolutional layer has 256 9x9x1 convolution kernels with a step length of 1, using the Relu activation function, after feature extraction of the convolutional layer, and outputting a feature tensor of 20x20x256.
  • PrimaryCaps layer contains 32 primary capsules, accepts the primary features obtained by the convolutional layer, and each capsule generates a combination of features and performs vectorized expression. Each dimension of the vector represents information such as the direction and position of the feature.
  • Each main capsule applies 8 9x9x256 convolution kernels to an input tensor of 20x20x256. Since there are 32 main capsules, the output is a 6x6x8x32 feature tensor.
  • DigitCaps layer Each digital capsule corresponds to the vector output by the PrimaryCaps layer. Each digital capsule accepts a 6x6x8x32 tensor as input, and uses dynamic routing to nest the output variables of each main capsule into a multi-layer digital capsule, and activate the vector
  • the key feature of is to map the 8-dimensional input space to the 16-dimensional capsule output space through an 8x16 weight matrix.
  • the decoding layer is the last fully connected layer of the network, including the Relu function and the Sigmoid function. It accepts the correct 16-dimensional vector output from the DigitCaps layer and uses it as input to learn the multiple features expressed by the capsule output vector, calculate edge loss, and learn reconstruction A 28x28 size image with the same pixels as the input image.
  • the loss function of the capsule network module is as follows:
  • O L ((I f ,T f ); ⁇ L ) represents the output of the capsule network module, ⁇ L represents the weight and parameters of the network training, and J b represents the binary cross-entropy loss function of the elements in the brackets .
  • Regional positioning network module used to perform pixel-level label prediction on CSM image-level labeled samples ⁇ I f , T f ⁇ , and output a regional positioning feature map; among them, the regional positioning network uses the feature extraction of the convolutional layer to generate the location information
  • the feature map, and the global average pooling layer is used, and the weights (w 1 , w 2 ..., w n ) and the feature map are weighted and averaged to obtain a regional positioning feature map.
  • the region with larger activation value in the regional positioning feature map is most likely to be the segmentation position of the cervical spinal cord injury region.
  • the regional positioning network makes full use of the primary features obtained by the training sample after the convolutional layer operation to locate the hot spots of the feature map. Since the convolutional neural network needs to train many parameters, the repeated use of the feature map can reduce the amount of network parameters, so that The training efficiency of the model is higher.
  • Sample generation module used to execute the self-diffusion algorithm based on the reconstructed image output by the capsule network module and the regional positioning feature map output by the regional positioning network to determine the segmentation line of the regional pixel points to obtain a coarser segmentation prediction sample ⁇ I f ,L f* ⁇ ;
  • the idea of random walk is used in the area with the larger activation value, and the pixels are diffused through the self-diffusion algorithm of random walk, using The input point of the regional positioning feature map is calculated, the Gaussian distance from each pixel on the image to the input point is calculated, and the optimal path is selected from it, and the segmentation line of the regional pixel points is obtained, and the classification points are diffused from each larger activation value, and finally Generate coarser segmentation prediction samples ⁇ I f ,L f* ⁇ .
  • Zi ,j represents the Gaussian distance between two adjacent superpixels.
  • the segmentation prediction samples ⁇ I f ,L f* ⁇ generated by the generator and the real labeled samples ⁇ I f ,L f ⁇ are input to the discriminator for adversarial training.
  • the discriminator uses the capsule network to record the direction and position information of the segmentation area to improve the segmentation The sharpness of the regional boundary area, and use the cascade mode to extract the key area pixels that are difficult to correctly classify in the image, filter out the simple and clear flat area pixels, and use the processed image for "generation-adversarial" training until the generator Nash equilibrium is formed between the discriminator and the discriminator.
  • the discriminator cannot distinguish whether the image comes from the segmentation prediction sample ⁇ I f ,L f* ⁇ generated by the generator or the real labeled sample ⁇ I f ,L f ⁇ , completing the training of the generated confrontation network .
  • the discriminator includes a cascade Cascade module, a Capsule network module, and a parameter optimization module; the specific functions of each module are as follows:
  • Cascade Cascade module used to extract the key pixels of the segmentation prediction sample; in the process of image segmentation, the difficulty of marking each pixel is different, and the flat background area can be easily distinguished, but the object But it is difficult to distinguish the boundary pixels from the background area.
  • the previous network structure put all these pixels into the network for processing, resulting in unnecessary redundancy of the network.
  • this application adopts the idea of cascade Cascade, treats pixels differently, and focuses on key pixel areas that are difficult to classify.
  • the segmentation prediction sample generated by the generator the incorrectly labeled pixels and the pixels whose confidence is lower than a certain threshold are extracted, and the correctly labeled and highly confident pixels are filtered out. In this way, the pixels input to the next stage of training are only key pixels that are not easy to distinguish, which can reduce redundant information in the network and improve the work efficiency of the network.
  • the Capsule network module is responsible for processing the extracted key pixels and generating errors; specifically, the functions of the Capsule network module include:
  • the segmentation prediction sample ⁇ I f ,L f* ⁇ output by the generator is filtered by the cascade Cascade module to extract the key pixels and the corresponding ground truth, and input them as input to the corresponding convolution In the layer; then use several convolutional layers to convolve the input key pixels and the corresponding ground truth separately to extract the low-level features in the segmentation prediction sample ⁇ I f ,L f* ⁇ ; the activation function of the convolution layer It is the ReLU function.
  • High-dimensional feature extraction by constructing the PrimaryCaps layer, the extracted low-level features are input to the PrimaryCaps layer to obtain high-dimensional feature vectors containing spatial location information; the DigitCaps layer is constructed, and the output variables in the PrimaryCaps layer are nested using dynamic routing Map it to the DigitCaps layer, construct the current high-level features that best characterize all input features, and input them to the next layer;
  • the feature vector extracted from the key pixel and the corresponding Ground truth after convolution in the convolutional layer is u i
  • the low-level feature vector u i is used as the input of the PrimaryCaps layer
  • the weight matrix Wij is multiplied to obtain the prediction vector among them:
  • the weighted sum S j can be obtained by linear combination between the prediction vectors, and the weight coefficient is c ij , where:
  • the length of the S j vector is limited by the compression function to obtain the output vector V j , where:
  • the first half is the scaling scale of the input vector S j
  • the second half is the unit vector of S j .
  • the coefficient c ij is constant
  • the calculation formula of c ij is:
  • Equation (7) b ij is a constant, b ij b ij values of the previous iteration by the value of V j and The sum of the products is obtained, that is, the update method of b ij is:
  • the discriminator puts the high-level feature vector V output by the DigitCaps layer into the decoding layer, and finally outputs the judgment result of the authenticity of the image through several fully connected layers; specifically: if the output result is 0, the judgment is false, indicating the input The image is judged to be a fake image; if the output result is 1, the judgement is true, which means that the input image successfully confuses the discriminator.
  • Parameter optimization module Used to optimize the network parameters of the generator and the discriminator using the error generated by the Capsule network module, so that the generator can output more optimized segmentation results.
  • Equation (9) ⁇ S ⁇ p, respectively and a parameter indicating the generator and a discriminator, J b represents the Mutual entropy loss function, O s O p, respectively, and the output of the generator and a discriminator.
  • the parameter optimization process includes two parts:
  • the parameters of the discriminator are continuously optimized, and the discriminative ability is continuously enhanced. It is more and more easy to distinguish the generated images of the generator, thus entering the next stage.
  • the error function corresponding to the generator is:
  • the image-level annotated CSM image to be segmented is input into the trained generation confrontation network, and the pixel-level segmentation image of the CSM image to be segmented is output through the generation confrontation network.
  • FIG. 7 is a schematic diagram of a hardware device structure of a medical image segmentation method based on a generative confrontation network provided by an embodiment of the present application.
  • the device includes one or more processors and memory. Taking a processor as an example, the device may also include: an input system and an output system.
  • the processor, the memory, the input system, and the output system may be connected by a bus or other methods.
  • the connection by a bus is taken as an example.
  • the memory can be used to store non-transitory software programs, non-transitory computer executable programs, and modules.
  • the processor executes various functional applications and data processing of the electronic device by running non-transitory software programs, instructions, and modules stored in the memory, that is, realizing the processing methods of the foregoing method embodiments.
  • the memory may include a program storage area and a data storage area, where the program storage area can store an operating system and an application program required by at least one function; the data storage area can store data and the like.
  • the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid state storage devices.
  • the storage may optionally include storage remotely arranged with respect to the processor, and these remote storages may be connected to the processing system through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
  • the input system can receive input digital or character information, and generate signal input.
  • the output system may include display devices such as a display screen.
  • the one or more modules are stored in the memory, and when executed by the one or more processors, the following operations of any of the foregoing method embodiments are performed:
  • Step a Collect pixel-level labeled samples of other medical images and image-level labeled samples of medical images to be segmented;
  • Step b Train a capsule network-based generative confrontation network through the pixel-level labeled samples of the other medical images and the image-level labeled samples of the medical images to be segmented, the generative confrontation network including a generator and a discriminator;
  • Step c The generator performs pixel-level feature extraction on the pixel-level annotation samples of other medical images, and processes the image-level annotation samples of the medical image to be segmented through the pixel-level features to generate the pixel-level medical image to be segmented Labeling samples, and generating segmentation prediction samples of the medical image to be segmented based on the pixel-level labeling samples;
  • Step d Input the segmented prediction samples generated by the generator and the real labeled samples of the image to be segmented into the discriminator for "generation-adversarial" training, to determine the authenticity of the segmented prediction samples, and generate according to the error function Optimize the detector and discriminator to obtain a trained generative confrontation network;
  • Step e input the image-level annotated medical image to be segmented into the trained generation confrontation network, and output the pixel-level segmented image of the medical image to be segmented through the generation confrontation network.
  • the embodiments of the present application provide a non-transitory (non-volatile) computer storage medium, the computer storage medium stores computer executable instructions, and the computer executable instructions can perform the following operations:
  • Step a Collect pixel-level labeled samples of other medical images and image-level labeled samples of medical images to be segmented;
  • Step b Train a capsule network-based generative confrontation network through the pixel-level labeled samples of the other medical images and the image-level labeled samples of the medical images to be segmented, the generative confrontation network including a generator and a discriminator;
  • Step c The generator performs pixel-level feature extraction on the pixel-level annotation samples of other medical images, and processes the image-level annotation samples of the medical image to be segmented through the pixel-level features to generate the pixel-level medical image to be segmented Labeling samples, and generating segmentation prediction samples of the medical image to be segmented based on the pixel-level labeling samples;
  • Step d Input the segmented prediction samples generated by the generator and the real labeled samples of the image to be segmented into the discriminator for "generation-adversarial" training, to determine the authenticity of the segmented prediction samples, and generate according to the error function Optimize the detector and discriminator to obtain a trained generative confrontation network;
  • Step e input the image-level annotated medical image to be segmented into the trained generation confrontation network, and output the pixel-level segmented image of the medical image to be segmented through the generation confrontation network.
  • the embodiment of the present application provides a computer program product, the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program includes program instructions, when the program instructions are executed by a computer To make the computer do the following:
  • Step a Collect pixel-level labeled samples of other medical images and image-level labeled samples of medical images to be segmented;
  • Step b Train a capsule network-based generative confrontation network through the pixel-level labeled samples of the other medical images and the image-level labeled samples of the medical images to be segmented, the generative confrontation network including a generator and a discriminator;
  • Step c The generator performs pixel-level feature extraction on the pixel-level annotation samples of other medical images, and processes the image-level annotation samples of the medical image to be segmented through the pixel-level features to generate the pixel-level medical image to be segmented Labeling samples, and generating segmentation prediction samples of the medical image to be segmented based on the pixel-level labeling samples;
  • Step d Input the segmented prediction samples generated by the generator and the real labeled samples of the image to be segmented into the discriminator for "generation-adversarial" training, to determine the authenticity of the segmented prediction samples, and generate according to the error function Optimize the detector and discriminator to obtain a trained generative confrontation network;
  • Step e input the image-level annotated medical image to be segmented into the trained generation confrontation network, and output the pixel-level segmented image of the medical image to be segmented through the generation confrontation network.
  • the medical image segmentation method, system, and electronic device based on the generative adversarial network in the embodiments of the application optimize the deep convolutional neural network through the fusion capsule mechanism, and integrate the Capsule network and the cascade waterfall idea, in the case of small medical image samples
  • This improves the feature expression ability of the model, expands the applicability of medical image annotation samples, and effectively reduces the dependence of the segmentation model on pixel-level annotation data.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

一种基于生成对抗网络的医学图像分割方法、系统及电子设备。首先，研究生成器如何对不同类别的高质量图像的像素级特征进行提取，并利用胶囊模型进行结构化特征表示，进而实现像素级标注样本的生成；其次构建合适的判别器，用于判别生成像素级标注样本的真伪，并设计合适的误差优化函数，将判别结果分别反馈到生成器和判别器的模型当中，通过不断的对抗训练，分别提高生成器和判别器的样本生成能力和判别能力，最后利用训练好的生成器生成像素级标注样本，实现图像级标注医学图像的像素级分割。有效地降低了分割模型对像素级标注数据的依赖，既能提高生成样本与真实样本对抗训练的效率，又可以有效实现高精度的像素级图像分割。

Description

一种基于生成对抗网络的医学图像分割方法、系统及电子设备 技术领域
本申请属于医学图像处理技术领域，特别涉及一种基于生成对抗网络的医学图像分割方法、系统及电子设备。
背景技术
随着医学影像技术的蓬勃发展，医学影像在临床医疗中有着广泛和深入的应用。据统计，全球每年有几千万病例通过医学影像进行辅助诊断和治疗。在基于医学影像诊断和治疗的传统方法中，医师对医学影像数据进行阅读、识别，并对疾病的诊断和治疗做出判断。这种诊疗方式非常低效，且个体差异大，医生凭个人的经验很容易漏诊和误诊，长时间阅片会导致医生疲劳，阅片准确率下降。随着人工智能的兴起，通过用机器预先对影像数据的筛选和判断，标注重点可疑区域，再交由医生进行诊断和治疗，可以大大减轻医生的工作量，且结果全面、稳定和高效。因此，人工智能在医学影像领域内具有重要的应用前景。
医学影像技术包括医学成像和医学图像处理两部分。常见的医学成像技术主要有MRI(Magnetic Resonance Imaging,磁共振成像)、计算机断层扫描成像(CT)、正电子发射型计算机断层显像(PET)、超声成像(US)和X射线成像。不同的成像技术，在不同的疾病诊断和治疗的应用上各有优势，在具体的临床应用中逐渐形成了针对特定疾病的诊疗目的选择相应的成像技术。比如，磁共振成像能够对软组织成像具有极好的分辨率，没有电离辐射等危害，在大脑和子宫等部位的诊疗上具有广泛的应用。深度学习应用于医学图像处理的基本流程如图1所示。
而在传统的基于生成对抗网络的医学图像分割任务中，为了能够充分的训练神经网络，达到高准确率的结果，需要准备大量的相关的医学影像数据，并且需要对这些医学影像数据进行人工的逐像素的标注。例如研究人脑的肿瘤区域分割，就需要人为的去标注对应的脑部肿瘤图像。医学疾病多种多样，对应的医学图像也是多种多样，利用深度学习进行医学影像的分割，每一种疾病对应的医学影像都需要进行人为的手工标注，而这样会耗费大量的人力物力。即使是最大的公共数据集，也只能提供有限的语义类别的像素级标注样本。高质量的数据在医学图像数据集中稀缺，严重限制语义分割模型的精确度。
生成式对抗网络(GAN,Generative Adversarial Networks)是一种深度学习模型，是近年来复杂分布上无监督学习最具前景的方法之一。模型通过框架中(至少)两个模块：生成模型(Generative Model)和判别模型(Discriminative Model)的互相博弈学习产生相当好的输出。现有的基于生成对抗网络的图像分割模型可应用于跨类别的物体图像分割，但是在医学图像领域，该模型则存在特征提取不够充分、对抗训练的计算量大等问题。
发明内容
本申请提供了一种基于生成对抗网络的医学图像分割方法、系统及电子设备，旨在至少在一定程度上解决现有技术中的上述技术问题之一。
为了解决上述问题，本申请提供了如下技术方案：
一种基于生成对抗网络的医学图像分割方法，包括以下步骤：
步骤a：分别采集其他医学图像的像素级标注样本以及待分割医学图像的图像级标注样本；
步骤b：通过所述其他医学图像的像素级标注样本以及待分割医学图像的 图像级标注样本训练基于胶囊网络的生成对抗网络，所述生成对抗网络包括生成器和判别器；
步骤c：所述生成器对其他医学图像的像素级标注样本进行像素级特征提取，通过所述像素级特征对待分割医学图像的图像级标注样本进行处理，生成所述待分割医学图像的像素级标注样本，并基于所述像素级标注样本生成所述待分割医学图像的分割预测样本；
步骤d：将所述生成器生成的分割预测样本和待分割图像的真实标注样本一起输入到判别器进行“生成-对抗”训练，判别所述分割预测样本的真伪，并根据误差函数对生成器和判别器进行优化，得到训练好的生成对抗网络；
步骤e：将图像级标注的待分割医学图像输入训练好的生成对抗网络，通过所述生成对抗网络输出待分割医学图像的像素级分割图像。
本申请实施例采取的技术方案还包括：在所述步骤c中，所述生成器包括胶囊网络模块和区域定位网络，所述生成器生成待分割医学图像的分割预测样本具体包括：
步骤b1：通过其他医学图像的像素级标注样本对胶囊网络模块进行预训练，得到无语义标签样本，通过所述无语义标签样本对待分割图像的图像级标注样本进行处理，区分所述待分割图像的图像级标注样本的背景和有效分割区域；
步骤b2：将所述待分割图像的图像级标注样本输入完成预训练后的胶囊网络模块，通过所述胶囊网络模块输出待分割图像的图像级标注样本的重建图像；
步骤b3：区域定位网络利用卷积层的特征提取生成待分割图像的图像级标注样本的包含位置信息的特征图，并采用全局平均池化层，将权重(w 1,w 2…,w n)与特征图进行加权平均，得到待分割图像的图像级标注样本的区域定位特征图；
步骤b4：根据所述重建图像和区域定位特征图执行自扩散算法，确定区域像素点分割线，得到待分割图像的图像级标注样本的分割预测样本。
本申请实施例采取的技术方案还包括：在所述步骤b2中，所述胶囊网络模块包括卷积层、PrimaryCaps层、DigitCaps层和解码层，所述胶囊网络模块采用单个胶囊神经元的输出向量，记录待分割图像的图像级标注样本分割区域边缘像素的方向和位置信息，采用矢量的非线性激活函数提取分类的概率值，确定待分割图像的图像级标注样本的分割区域与背景，计算边缘损失并输出待分割图像的图像级标注样本的重建图像。
本申请实施例采取的技术方案还包括：在所述步骤b4中，所述根据重建图像和区域定位特征图执行自扩散算法具体包括：在区域定位特征图中激活值越大的区域运用随机漫步的自扩散算法扩散像素点，利用区域定位特征图的输入点，计算图像上每个像素到输入点的高斯距离，并从中选择最优的路径，获得区域像素点的分割线，最终生成分割预测样本。
本申请实施例采取的技术方案还包括：在所述步骤d中，所述判别器包括级联Cascade模块、Capsule网络模块和参数优化模块，所述判别器进行的“生成-对抗”训练具体包括：
步骤d1：通过级联Cascade模块提取所述分割预测样本中标注错误的像素以及置信度低于设定阈值的关键像素以及对应的ground truth，并过滤标注正确且置信度高于设定阈值的像素；
步骤d2：通过Capsule网络模块将提取到的关键像素以及对应的ground truth进行处理，并产生误差；
步骤d3：所述参数优化模块利用Capsule网络模块产生的误差对生成器与判别器的网络参数进行优化；其中，对于给定的分割预测样本{I f,L f*}和对应的真 实标注样本{I f,L f}，网络的整体误差函数为：
Figure PCTCN2019125428-appb-000001
上述公式中，θ S和θ p分别表示生成器和判别器的参数，J b表示二元交叉熵损失函数，O s和O p分别表示生成器和判别器的输出，当输入来自真实标注样本{I f,L f}和分割预测样本{I f,L f*}时，通过输出1和0来标注像素点类别的真伪。
本申请实施例采取的另一技术方案为：一种基于生成对抗网络的医学图像分割系统，包括样本采集模块和生成对抗网络，
样本采集模块：用于分别采集其他医学图像的像素级标注样本以及待分割医学图像的图像级标注样本；
通过所述其他医学图像的像素级标注样本以及待分割医学图像的图像级标注样本训练基于胶囊网络的生成对抗网络；
所述生成对抗网络包括生成器和判别器，所述生成器对其他医学图像的像素级标注样本进行像素级特征提取，通过所述像素级特征对待分割医学图像的图像级标注样本进行处理，生成所述待分割医学图像的像素级标注样本，并基于所述像素级标注样本生成所述待分割医学图像的分割预测样本；
将所述生成器生成的分割预测样本和待分割图像的真实标注样本一起输入到判别器进行“生成-对抗”训练，判别所述分割预测样本的真伪，并根据误差函数对生成器和判别器进行优化，得到训练好的生成对抗网络；
将图像级标注的待分割医学图像输入训练好的生成对抗网络，通过所述生成对抗网络输出待分割医学图像的像素级分割图像。
本申请实施例采取的技术方案还包括：所述生成器包括预训练模块、胶囊 网络模块、区域定位网络模块和样本生成模块：
预训练模块：用于通过其他医学图像的像素级标注样本对胶囊网络模块进行预训练，得到无语义标签样本，通过所述无语义标签样本对待分割图像的图像级标注样本进行处理，区分所述待分割图像的图像级标注样本的背景和有效分割区域；
胶囊网络模块：用于将所述待分割图像的图像级标注样本输入完成预训练后的胶囊网络模块，通过所述胶囊网络模块输出待分割图像的图像级标注样本的重建图像；
区域定位网络：用于利用卷积层的特征提取生成待分割图像的图像级标注样本的包含位置信息的特征图，并采用全局平均池化层，将权重(w 1,w 2…,w n)与特征图进行加权平均，得到待分割图像的图像级标注样本的区域定位特征图；
样本生成模块：用于根据所述重建图像和区域定位特征图执行自扩散算法，确定区域像素点分割线，得到待分割图像的图像级标注样本的分割预测样本。
本申请实施例采取的技术方案还包括：所述胶囊网络模块包括卷积层、PrimaryCaps层、DigitCaps层和解码层，所述胶囊网络模块采用单个胶囊神经元的输出向量，记录待分割图像的图像级标注样本分割区域边缘像素的方向和位置信息，采用矢量的非线性激活函数提取分类的概率值，确定待分割图像的图像级标注样本的分割区域与背景，计算边缘损失并输出待分割图像的图像级标注样本的重建图像。
本申请实施例采取的技术方案还包括：所述样本生成模块根据重建图像和区域定位特征图执行自扩散算法具体包括：在区域定位特征图中激活值越大的区域运用随机漫步的自扩散算法扩散像素点，利用区域定位特征图的输入点， 计算图像上每个像素到输入点的高斯距离，并从中选择最优的路径，获得区域像素点的分割线，最终生成分割预测样本。
本申请实施例采取的技术方案还包括：所述判别器包括级联Cascade模块、Capsule网络模块和参数优化模块：
级联Cascade模块：用于提取所述分割预测样本中标注错误的像素以及置信度低于设定阈值的关键像素以及对应的ground truth，并过滤标注正确且置信度高于设定阈值的像素；
Capsule网络模块：用于将提取到的关键像素以及对应的ground truth进行处理，并产生误差；
参数优化模块：用于利用Capsule网络模块产生的误差对生成器与判别器的网络参数进行优化；其中，对于给定的分割预测样本{I f,L f*}和对应的真实标注样本{I f,L f}，网络的整体误差函数为：
Figure PCTCN2019125428-appb-000002
上述公式中，θ S和θ p分别表示生成器和判别器的参数，J b表示二元交叉熵损失函数，O s和O p分别表示生成器和判别器的输出，当输入来自真实标注样本{I f,L f}和分割预测样本{I f,L f*}时，通过输出1和0来标注像素点类别的真伪。
本申请实施例采取的又一技术方案为：一种电子设备，包括：
至少一个处理器；以及
与所述至少一个处理器通信连接的存储器；其中，
所述存储器存储有可被所述一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行上述的基于生成对抗网络 的医学图像分割方法的以下操作：
步骤a：分别采集其他医学图像的像素级标注样本以及待分割医学图像的图像级标注样本；
步骤b：通过所述其他医学图像的像素级标注样本以及待分割医学图像的图像级标注样本训练基于胶囊网络的生成对抗网络，所述生成对抗网络包括生成器和判别器；
步骤c：所述生成器对其他医学图像的像素级标注样本进行像素级特征提取，通过所述像素级特征对待分割医学图像的图像级标注样本进行处理，生成所述待分割医学图像的像素级标注样本，并基于所述像素级标注样本生成所述待分割医学图像的分割预测样本；
步骤d：将所述生成器生成的分割预测样本和待分割图像的真实标注样本一起输入到判别器进行“生成-对抗”训练，判别所述分割预测样本的真伪，并根据误差函数对生成器和判别器进行优化，得到训练好的生成对抗网络；
步骤e：将图像级标注的待分割医学图像输入训练好的生成对抗网络，通过所述生成对抗网络输出待分割医学图像的像素级分割图像。
相对于现有技术，本申请实施例产生的有益效果在于：本申请实施例的基于生成对抗网络的医学图像分割方法、系统及电子设备通过融合胶囊机制对深度卷积神经网络进行优化，融合Capsule网络和级联瀑布的思想，在医学图像样本量小的情况下，生成新的训练图像样本，实现对低质量仅有图像级别标签的医学影像数据的语义分割，将学习到的分割知识从像素级标注的全标注数据，转移到图像级别的弱标注数据，从而提高了模型特征表达能力，扩展医学图像标注样本的可适用性，有效地降低分割模型对像素级标注数据的依赖，具有网络信息冗余少，特征提取充分的特点，在少量像素级标注样本的前提下，既能 提高生成样本与真实样本对抗训练的效率，又可以有效实现高精度的像素级图像分割。
附图说明
图1是深度学习应用于医学图像处理的基本流程图；
图2是本申请实施例的基于生成对抗网络的医学图像分割方法的流程图；
图3是本申请实施例的生成对抗网络的结构示意图；
图4是胶囊网络模块的结构示意图；
图5是区域定位网络的网络结构示意图；
图6是本申请实施例的基于生成对抗网络的医学图像分割系统的结构示意图；
图7是本申请实施例提供的基于生成对抗网络的医学图像分割方法的硬件设备结构示意图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本申请，并不用于限定本申请。
为了解决现有技术存在的不足，本申请实施例的基于生成对抗网络的医学图像分割方法通过融合胶囊机制对生成对抗网络进行改进，首先，研究生成器如何对不同类别的高质量图像的像素级特征进行提取，并利用胶囊模型进行结构化特征表示，进而实现像素级标注样本的生成；其次构建合适的判别器，用于判别生成像素级标注样本的真伪，并设计合适的误差优化函数，将判别结果分别反馈到生成器和判别器的模型当中，通过不断的对抗训练，分别提高生成 器和判别器的样本生成能力和判别能力，最后利用训练好的生成器生成像素级标注样本，实现图像级标注医学图像的像素级分割。以下实施例中，本申请仅以脊髓型颈椎病(cervical spondylotic myelopathy,CSM)的医学图像分割为例进行详细阐述，病源种类不局限于单一病例，可延伸到多种病例的图像分割场景中，例如脑部MRI图像分割等。针对不同病例的图像分割，只需在数据采集阶段采集对应病例的训练样本，并在模型的生成器中将训练样本进行替换即可。
请参阅图2，是本申请实施例的基于生成对抗网络的医学图像分割方法的流程图。本申请实施例的基于生成对抗网络的医学图像分割方法包括以下步骤：
步骤100：分别采集其他医学图像的像素级标注样本以及CSM图像级标注样本；
步骤100中，其他医学图像包括肺部等部位的医学图像，标注样本的获取方式具体为：采集少量的全标注CSM图像样本{I f,L f,T f}的共500张DTI样本(diffusion tensor imaging，弥散张量成像)，图像尺寸为28x28，CSM组60例，其中男27例女33例，年龄20～71岁，平均45岁，包含像素级标注样本和图像级标注样本{L f,T f}；其他医学图像(如人的肺部)的像素级标注样本{I O,L O}的8000张DTI样本，图像尺寸为28x28。通过校正变形及选定阈值后分别获取到CSM图像和其他医学图像的DTI样本，在DTI样本上确定感兴趣分割区域(ROI)，并将ROI放置于脊髓病变区中央，保证DTI样本尺寸的统一，避免脊髓液和伪影的影响。
请参阅图3，是本申请实施例的生成对抗网络的结构示意图。本申请实施例的生成对抗网络包括生成器和判别器，其中，生成器负责生成医学图像的像素级标注数据，而判别器则负责精细化生成的标注。生成器包括胶囊网络模块和区域定位网络两个部分，将其他医学图像的像素级标注样本{I O,L O}作为胶 囊网络模块的预训练样本，并将CSM图像级标注样本{L f,T f}作为区域定位网络的训练样本。
步骤200：通过其他医学图像的像素级标注样本对胶囊网络模块进行预训练，得到无语义标签样本，并通过无语义标签样本对CSM图像级标注样本进行处理，区分CSM图像级标注样本的背景和有效分割区域；
步骤200中，胶囊网络模块采用可迁移的语义分割模型，该模型可以将学习到的分割知识，从全标注数据的像素级标注转移到弱标注数据的图像级标注中。在实际的应用中，高质量的医学图像获取十分困难，因此，本申请通过已训练好的模型使得新的数据与目标域的数据进行匹配，获取像素级分割的特征，从而在样本量较少的情况下同样能够实现高精度的图像分割。胶囊网络模块的预训练过程具体包括：
步骤201：对其他医学图像的像素级标注样本进行处理，将其变成无语义的分割图像标注；
步骤201中，无语义的分割图像标注即标注仅仅区分样本的有效分割区域与背景，而并不区分样本的形状。这样的训练数据训练出的网络，学习到的是区分物体与背景的知识，属于宽泛的高阶特征，而对于图像来说，都有物体与背景的区分，因此宽泛的高阶特征具有较强的通用性，可以很容易的迁移至其他不同的任务中，如此就可以将从高质量像素级别标注的医学图像中学习到的知识，迁移到低质量图像级别标注的医学图像中，而高质量的图像与低质量的图像并不需要有直接的关联性。
步骤202：根据其他医学图像的无语义分割图像标注对CSM图像级标注样本进行处理，生成CSM图像级标注样本的无语义标签样本{I O,L O}，并通过过滤语义信息，获取像素级别的无语义标注L O；
步骤202中，获取像素级别的无语义标注的目的在于区分数据的有效分割区域和背景，从而更容易在强弱标注的数据中迁移所学习到的知识。
步骤300：将CSM图像级标注样本{L f,T f}输入完成预训练后的胶囊网络模块，胶囊网络模块输出CSM图像级标注样本的重建图像；
步骤300中，胶囊网络模块采用单个胶囊神经元的输出向量，记录CSM图像级标注样本分割区域边缘像素的方向和位置信息，采用矢量的非线性激活函数提取分类的概率值，确定CSM图像级标注样本的分割区域与背景，计算边缘损失并输出CSM图像级标注样本的重建图像。本申请利用胶囊网络模块实例化像素的参数信息，输出激活向量记录分割区域的位置和角度信息，能有效地提高分割区域边界区域的锐化程度。
胶囊网络模块的结构如图4所示。该模型包括卷积层、PrimaryCaps层、DigitCaps层和解码层，每个胶囊代表一个函数，输出激活的向量，向量长度代表胶囊所寻找的区域分割线正确的概率。胶囊网络模块每层的功能分别为：
卷积层：通过对CSM图像级标注样本{L f,T f}进行卷积操作获得脊髓间盘形态、受压位置等初级特征。以输入尺寸为28x28的CSM图像为例，卷积层有256个步长为1的9x9x1的卷积核，使用Relu激活函数，经过卷积层的特征提取，输出20x20x256的特征张量。
PrimaryCaps层：包含32个主胶囊，接受卷积层获得的初级特征，每个胶囊生成特征的组合并进行向量化表达，向量的每个维度表示特征的方向、位置等信息。每个主胶囊将8个9x9x256的卷积核应用到20x20x256的输入张量中，由于有32个主胶囊，输出为6x6x8x32的特征张量。
DigitCaps层：每个数字胶囊对应PrimaryCaps层输出的向量，每个数字胶囊接受一个6x6x8x32的张量作为输入，采用动态路由将每个主胶囊的输出变 量嵌套映射到多层数字胶囊中，激活向量的关键特征，通过8x16的权重矩阵将8维输入空间映射到16维胶囊输出空间。
解码层：解码层即为网络最后的全连接层，包含Relu函数和Sigmoid函数，接受DigitCaps层输出的正确的16维向量，并作为输入学习胶囊输出向量表达的多重特征，计算边缘损失，学习重建一张与输入图像相同像素的28x28大小的图像。
本申请实施例中，胶囊网络模块的损失函数如下：
对于给定的CSM图像级标注样本{I f,T f}，对应的损失函数为：
Figure PCTCN2019125428-appb-000003
公式(1)中，O L((I f,T f)；θ L)表示胶囊网络模块的输出，θ L表示网络训练的权重及参数，J b表示括号内元素的二元交叉熵损失函数。
将胶囊网络模块应用到CSM图像级标注样本{I f,T f}，输出无语义的粗分割图M＝O L((I f,T f)；θ L)。
S400：通过区域定位网络对CSM图像级标注样本{I f,T f}进行像素级标签预测，并输出区域定位特征图；
步骤400中，区域定位网络的网络结构如图5所示。区域定位网络利用卷积层的特征提取生成包含位置信息的特征图，并采用全局平均池化层，将权重(w 1,w 2…,w n)与特征图进行加权平均，得到区域定位特征图。区域定位特征图中激活值越大的区域最可能是脊髓颈椎的损伤区域分割位置。区域定位网络充分利用训练样本在卷积层操作后获得的初级特征定位特征图的热点区域，由于卷积神经网络需要训练的参数很多，对特征图的重复利用，能够降低网络的参数量，使得模型的训练效率更高。
步骤500：样本生成模块根据胶囊网络输出的重建图像和区域定位网络输 出的区域定位特征图执行自扩散算法，确定区域像素点分割线，得到较为粗糙的分割预测样本{I f,L f*}；
步骤500中，为了将包含语义信息的分割图M输出为像素级标注的分割样本，在激活值越大的区域运用随机漫步(Random Walk)思想，通过随机漫步的自扩散算法扩散像素点，利用区域定位特征图的输入点，计算图像上每个像素到输入点的高斯距离，并从中选择最优的路径，获得区域像素点的分割线，从每个激活值较大的分类点扩散，最终生成较为粗糙的分割预测样本{I f,L f*}。
给定一个CSM图像级标注样本{I f,T f}，将它转换成超像素p＝{p1，p2，...，pN}，这些图像由一个无向图模型G进行描述，其中每个节点对应一个特定的超像素，然后在无向图模型G上执行自扩散算法。以粗分割图M为基础，定义类别的自扩散过程的目标函数：
Figure PCTCN2019125428-appb-000004
公式(2)中，q＝[q1，q2，...，qN]表示所有超像素p的标签向量，如果pi∈A，则qi固定为1，否则其初始值为0。
Z i,j＝exp(-||F(p i)-F(p j)||/2σ 2)      (3)
公式(3)中，Z i,j表示两个相邻超像素之间的高斯距离。
通过以上操作，就可以利用少量的高质量有像素级别标注的图像来实现对仅有图像级别标注的CSM图像进行语义分割。
步骤600：将生成器输出的分割预测样本{I f,L f*}和真实标注样本{I f,L f}一起输入到判别器进行“生成-对抗”训练，对生成器进行优化；
步骤600中，判别器利用胶囊网络模块记录分割区域的方向和位置信息，提高分割区域边界区域的锐化程度，并利用级联模式提取出图像中难以正确分类的关键区域像素，过滤掉简单明确的平坦区域像素，利用处理过的图像进行 “生成-对抗”训练，直至生成器与判别器之间形成纳什均衡，判别器无法区分图像是来自于生成器生成的分割预测样本{I f,L f*}还是真实标注样本{I f,L f}，完成生成对抗网络的训练。
如图3所示，本申请实施例中，判别器包括级联Cascade模块、Capsule网络模块和参数优化模块；各模块具体功能如下：
级联Cascade模块：用于负责提取分割预测样本的关键像素；在图像分割的过程中，每一个像素标注的难易程度是不一样的，平坦的背景区域可以很容易的被区分出来，但是物体与背景区域的边界像素却难以区分。以往的网络结构将这些像素统统放入网络中进行处理，造成网络不必要的冗余。而本申请采用级联Cascade的思想，对像素区别对待，重点处理难以分类的重点像素区域。将生成器生成的分割预测样本中标注错误以及置信度低于一定阈值的关键像素以及对应的ground truth提取出来，而将标注正确且置信度很高的像素过滤掉。这样，输入到下一阶段训练的像素仅为不容易区分的关键像素，可以减少网络中的冗余信息，提高网络的工作效率。
Capsule网络模块负责将提取到的关键像素以及对应的ground truth进行处理，并产生误差；具体的，Capsule网络模块的功能包括：
步骤610：局部特征提取；将关键像素以及对应的ground truth作为输入分别输入到对应的卷积层中；然后利用若干卷积层，对输入的关键像素与对应的ground truth分别进行卷积，提取出分割预测样本{I f,L f*}中的低级特征；其中卷积层的激活函数为ReLU函数。
步骤611：高维特征提取；通过构建PrimaryCaps层，将提取的低级特征作输入到PrimaryCaps层，获取包含空间位置信息的高维特征向量；构建DigitCaps层，采用动态路由将PrimaryCaps层中的输出变量嵌套映射到 DigitCaps层中，构建出当前最能够表征所有输入特征的高级特征，并将其输入到下一层；
步骤611中涉及到的PrimaryCaps层与DigitCaps层之间的计算方式如下：
设关键像素与对应的Ground truth经过卷积层卷积后提取出的特征向量为u i，将低级特征向量u i作为PrimaryCaps层的输入，与权重矩阵W ij相乘得到预测向量
Figure PCTCN2019125428-appb-000005
其中：
Figure PCTCN2019125428-appb-000006
而预测向量之间可以通过线性组合的方式得到加权和S j，权重系数为c ij，其中：
Figure PCTCN2019125428-appb-000007
获得加权和S j后，通过压缩函数将S j向量长度限定，获得输出向量V j，其中：
Figure PCTCN2019125428-appb-000008
公式(6)中，前半部分为输入向量S j的缩放尺度，后半部分为S j的单位向量。而在计算S j的过程中，系数c ij为常量，c ij的计算公式为：
Figure PCTCN2019125428-appb-000009
公式(7)中，b ij为常量，b ij的数值通过上次迭代的b ij的值与V j和
Figure PCTCN2019125428-appb-000010
的积求和得到，即，b ij的更新方式为：
Figure PCTCN2019125428-appb-000011
步骤612：判别器将DigitCaps层输出的高级特征向量V放入解码层中，通过若干全连接层，最终输出图像真伪的判别结果；具体为：若输出结果为0，则判别为假，表示输入图像被判别为伪造图像；若输出结果为1，则判别为真， 表示输入图像成功混淆了判别器。
参数优化模块：利用Capsule网络模块产生的误差对生成器与判别器的网络参数进行优化，使生成器可以输出更优化的分割结果。
对于给定的分割预测样本{I f,L f*}和对应的真实标注样本{I f,L f}，网络的整体误差函数如下所示：
Figure PCTCN2019125428-appb-000012
公式(9)中，θ S和θ p分别表示生成器和判别器的参数，J b表示二元交叉熵损失函数，O s和O p分别表示生成器和判别器的输出。当输入来自真实标注样本{I f,L f}和分割预测样本{I f,L f*}时，输出1和0来标注像素点类别的真伪。
本申请实施例中，参数优化的过程包括两部分：
步骤620：固定生成器参数θ S，优化判别器参数θ p；在对抗训练过程中，首先固定生成器参数θ S，利用生成器生成的分割预测样本送入到判别器中，由判别器判断真伪，并利用判别器误差函数，通过反向传播算法调整判别器参数θ p，提高自身鉴别能力。而判别器对应的误差函数为：
Figure PCTCN2019125428-appb-000013
在训练的过程中，判别器的参数不断优化，判别能力不断增强，越来越容易区分出生成器的生成图像，从而进入到下一个阶段。
步骤621：固定判别器参数θ p，优化生成器参数θ S；网络将判别器的判别结果带入到生成器误差函数中，通过反向传播算法调整生成器参数θ S，使得生成器生成更高质量的分割结果，这样，生成器生成更加精确的结果来迷惑判别器。而生成器对应的误差函数为：
Figure PCTCN2019125428-appb-000014
重复上述两个优化步骤，最后，生成器与判别器之间形成纳什均衡，判别器无法区分图像是来自于生成器输出的分割预测样本{I f,L f*}还是真实标注样本{I f,L f}，则生成对抗网络训练完成。
步骤700：将图像级标注的CSM图像输入训练好的生成对抗网络，通过生成对抗网络输出CSM图像的像素级分割图像。
请参阅图6，是本申请实施例的基于生成对抗网络的医学图像分割系统的结构示意图。本申请实施例的基于生成对抗网络的医学图像分割系统包括样本采集模块和生成对抗网络，通过样本采集模块采集的图像样本训练生成对抗网络，生成对抗网络包括包括生成器和判别器，生成器利用胶囊模型进行结构化特征表示，进而实现像素级标注样本的生成，判别器用于判别生成像素级标注样本的真伪，并设计合适的误差优化函数，将判别结果分别反馈到生成器和判别器的模型当中，通过不断的对抗训练，分别提高生成器和判别器的样本生成能力和判别能力，最后利用训练好的生成器生成像素级标注样本，实现图像级标注医学图像的像素级分割。具体的：
样本采集模块：用于分别采集其他医学图像的像素级标注样本以及CSM图像级标注样本；其中，其他医学图像包括肺部等部位的医学图像，标注样本的获取方式具体为：采集少量的全标注CSM图像样本{I f,L f,T f}的共500张DTI样本(diffusion tensor imaging，弥散张量成像)，图像尺寸为28x28，CSM组60例，其中男27例女33例，年龄20～71岁，平均45岁，包含像素级标注样本和图像级标注样本{L f,T f}；其他医学图像(如人的肺部)的像素级标注样本{I O,L O}的8000张DTI样本，图像尺寸为28x28。通过校正变形及选定阈值后分别获取到CSM图像和其他医学图像的DTI样本，在DTI样本上确定感兴 趣分割区域(ROI)，并将ROI放置于脊髓病变区中央，保证DTI样本尺寸的统一，避免脊髓液和伪影的影响。
生成器包括预训练模块、胶囊网络模块、区域定位网络模块和样本生成模块，各模块功能具体如下：
预训练模块：用于通过其他医学图像的像素级标注样本对胶囊网络模块进行预训练，得到无语义标签样本，并通过无语义标签样本对CSM图像级标注样本进行处理，区分CSM图像级标注样本的背景和有效分割区域；其中，胶囊网络模块采用可迁移的语义分割模型，该模型可以将学习到的分割知识，从全标注数据的像素级标注转移到弱标注数据的图像级标注中。在实际的应用中，高质量的医学图像获取十分困难，因此，本申请通过已训练好的模型使得新的数据与目标域的数据进行匹配，获取像素级分割的特征，从而在样本量较少的情况下同样能够实现高精度的图像分割。胶囊网络模块的预训练过程具体包括：
1、对其他医学图像的像素级标注样本进行处理，将其变成无语义的分割图像标注；其中，无语义的分割图像标注即标注仅仅区分样本的有效分割区域与背景，而并不区分样本的形状。这样的训练数据训练出的网络，学习到的是区分物体与背景的知识，属于宽泛的高阶特征，而对于图像来说，都有物体与背景的区分，因此宽泛的高阶特征具有较强的通用性，可以很容易的迁移至其他不同的任务中，如此就可以将从高质量像素级别标注的医学图像中学习到的知识，迁移到低质量图像级别标注的医学图像中，而高质量的图像与低质量的图像并不需要有直接的关联性。
2、根据其他医学图像的无语义分割图像标注对CSM图像级标注样本进行处理，生成CSM图像级标注样本的无语义标签样本{I O,L O}，并通过过滤语义信息，获取像素级别的无语义标注L O；其中，获取像素级别的无语义标注的 目的在于区分数据的有效分割区域和背景，从而更容易在强弱标注的数据中迁移所学习到的知识。
胶囊网络模块：用于将CSM图像级标注样本{L f,T f}输入完成预训练后的胶囊网络模块，胶囊网络模块输出CSM图像级标注样本的重建图像；其中，胶囊网络模块采用单个胶囊神经元的输出向量，记录CSM图像级标注样本分割区域边缘像素的方向和位置信息，采用矢量的非线性激活函数提取分类的概率值，确定CSM图像级标注样本的分割区域与背景，计算边缘损失并输出CSM图像级标注样本的重建图像。本申请利用胶囊网络模块实例化像素的参数信息，输出激活向量记录分割区域的位置和角度信息，能有效地提高分割区域边界区域的锐化程度。
胶囊网络模块包括卷积层、PrimaryCaps层、DigitCaps层和解码层，每个胶囊代表一个函数，输出激活的向量，向量长度代表胶囊所寻找的区域分割线正确的概率。胶囊网络模块每层的功能分别为：
卷积层：通过对CSM图像级标注样本{L f,T f}进行卷积操作获得脊髓间盘形态、受压位置等初级特征。以输入尺寸为28x28的CSM图像为例，卷积层有256个步长为1的9x9x1的卷积核，使用Relu激活函数，经过卷积层的特征提取，输出20x20x256的特征张量。
PrimaryCaps层：包含32个主胶囊，接受卷积层获得的初级特征，每个胶囊生成特征的组合并进行向量化表达，向量的每个维度表示特征的方向、位置等信息。每个主胶囊将8个9x9x256的卷积核应用到20x20x256的输入张量中，由于有32个主胶囊，输出为6x6x8x32的特征张量。
DigitCaps层：每个数字胶囊对应PrimaryCaps层输出的向量，每个数字胶囊接受一个6x6x8x32的张量作为输入，采用动态路由将每个主胶囊的输出变 量嵌套映射到多层数字胶囊中，激活向量的关键特征，通过8x16的权重矩阵将8维输入空间映射到16维胶囊输出空间。
解码层：解码层即为网络最后的全连接层，包含Relu函数和Sigmoid函数，接受DigitCaps层输出的正确的16维向量，并作为输入学习胶囊输出向量表达的多重特征，计算边缘损失，学习重建一张与输入图像相同像素的28x28大小的图像。
本申请实施例中，胶囊网络模块的损失函数如下：
对于给定的CSM图像级标注样本{I f,T f}，对应的损失函数为：
Figure PCTCN2019125428-appb-000015
公式(1)中，O L((I f,T f)；θ L)表示胶囊网络模块的输出，θ L表示网络训练的权重及参数，J b表示括号内元素的二元交叉熵损失函数。
将胶囊网络模块应用到CSM图像级标注样本{I f,T f}，输出无语义的粗分割图M＝O L((I f,T f)；θ L)。
区域定位网络模块：用于对CSM图像级标注样本{I f,T f}进行像素级标签预测，并输出区域定位特征图；其中，区域定位网络利用卷积层的特征提取生成包含位置信息的特征图，并采用全局平均池化层，将权重(w 1,w 2…,w n)与特征图进行加权平均，得到区域定位特征图。区域定位特征图中激活值越大的区域最可能是脊髓颈椎的损伤区域分割位置。区域定位网络充分利用训练样本在卷积层操作后获得的初级特征定位特征图的热点区域，由于卷积神经网络需要训练的参数很多，对特征图的重复利用，能够降低网络的参数量，使得模型的训练效率更高。
样本生成模块：用于根据胶囊网络模块输出的重建图像和区域定位网络输出的区域定位特征图执行自扩散算法，确定区域像素点分割线，得到较为粗糙 的分割预测样本{I f,L f*}；其中，为了将包含语义信息的分割图M输出为像素级标注的分割样本，在激活值越大的区域运用随机漫步(Random Walk)思想，通过随机漫步的自扩散算法扩散像素点，利用区域定位特征图的输入点，计算图像上每个像素到输入点的高斯距离，并从中选择最优的路径，获得区域像素点的分割线，从每个激活值较大的分类点扩散，最终生成较为粗糙的分割预测样本{I f,L f*}。
给定一个CSM图像级标注样本{I f,T f}，将它转换成超像素p＝{p1，p2，...，pN}，这些图像由一个无向图模型G进行描述，其中每个节点对应一个特定的超像素，然后在无向图模型G上执行自扩散算法。以粗分割图M为基础，定义类别的自扩散过程的目标函数：
Figure PCTCN2019125428-appb-000016
公式(2)中，q＝[q1，q2，...，qN]表示所有超像素p的标签向量，如果pi∈A，则qi固定为1，否则其初始值为0。
Z i,j＝exp(-||F(p i)-F(p j)||/2σ 2)     (3)
公式(3)中，Z i,j表示两个相邻超像素之间的高斯距离。
通过以上操作，就可以利用少量的高质量有像素级别标注的图像来实现对仅有图像级别标注的CSM图像进行语义分割。
将生成器生成的分割预测样本{I f,L f*}与真实标注样本{I f,L f}输入判别器进行对抗训练，判别器利用胶囊网络记录分割区域的方向和位置信息，提高分割区域边界区域的锐化程度，并利用级联模式提取出图像中难以正确分类的关键区域像素，过滤掉简单明确的平坦区域像素，利用处理过的图像进行“生成-对抗”训练，直至生成器与判别器之间形成纳什均衡，判别器无法区分图像是来自于生成器生成的分割预测样本{I f,L f*}还是真实标注样本{I f,L f}，完成生成 对抗网络的训练。
具体的，判别器包括级联Cascade模块、Capsule网络模块和参数优化模块；各模块具体功能如下：
级联Cascade模块：用于负责提取分割预测样本的关键像素；在图像分割的过程中，每一个像素标注的难易程度是不一样的，平坦的背景区域可以很容易的被区分出来，但是物体与背景区域的边界像素却难以区分。以往的网络结构将这些像素统统放入网络中进行处理，造成网络不必要的冗余。而本申请采用级联Cascade的思想，对像素区别对待，重点处理难以分类的重点像素区域。将生成器生成的分割预测样本中标注错误的像素以及置信度低于一定阈值的像素提取出来，而将标注正确且置信度很高的像素过滤掉。这样，输入到下一阶段训练的像素仅为不容易区分的关键像素，可以减少网络中的冗余信息，提高网络的工作效率。
Capsule网络模块负责将提取到的关键像素进行处理，并产生误差；具体的，Capsule网络模块的功能包括：
1、局部特征提取；将生成器输出的分割预测样本{I f,L f*}经过级联Cascade模块过滤，提取出关键像素以及对应的ground truth，将其作为输入分别输入到对应的卷积层中；然后利用若干卷积层，对输入的关键像素与对应的ground truth分别进行卷积，提取出分割预测样本{I f,L f*}中的低级特征；其中卷积层的激活函数为ReLU函数。
2、高维特征提取；通过构建PrimaryCaps层，将提取的低级特征作输入到PrimaryCaps层，获取包含空间位置信息的高维特征向量；构建DigitCaps层，采用动态路由将PrimaryCaps层中的输出变量嵌套映射到DigitCaps层中，构建出当前最能够表征所有输入特征的高级特征，并将其输入到下一层；
上述中，PrimaryCaps层与DigitCaps层之间的计算方式如下：
设关键像素与对应的Ground truth经过卷积层卷积后提取出的特征向量为u i，将低级特征向量u i作为PrimaryCaps层的输入，与权重矩阵W ij相乘得到预测向量
Figure PCTCN2019125428-appb-000017
其中：
Figure PCTCN2019125428-appb-000018
而预测向量之间可以通过线性组合的方式得到加权和S j，权重系数为c ij，其中：
Figure PCTCN2019125428-appb-000019
获得加权和S j后，通过压缩函数将S j向量长度限定，获得输出向量V j，其中：
Figure PCTCN2019125428-appb-000020
公式(6)中，前半部分为输入向量S j的缩放尺度，后半部分为S j的单位向量。而在计算S j的过程中，系数c ij为常量，c ij的计算公式为：
Figure PCTCN2019125428-appb-000021
公式(7)中，b ij为常量，b ij的数值通过上次迭代的b ij的值与V j和
Figure PCTCN2019125428-appb-000022
的积求和得到，即，b ij的更新方式为：
Figure PCTCN2019125428-appb-000023
3、判别器将DigitCaps层输出的高级特征向量V放入解码层中，通过若干全连接层，最终输出图像真伪的判别结果；具体为：若输出结果为0，则判别为假，表示输入图像被判别为伪造图像；若输出结果为1，则判别为真，表示输入图像成功混淆了判别器。
参数优化模块：用于利用Capsule网络模块产生的误差对生成器与判别器 的网络参数进行优化，使生成器可以输出更优化的分割结果。
对于给定的分割预测样本{I f,L f*}和对应的真实标注样本{I f,L f}，网络的整体误差函数如下所示：
Figure PCTCN2019125428-appb-000024
公式(9)中，θ S和θ p分别表示生成器和判别器的参数，J b表示二元交叉熵损失函数，O s和O p分别表示生成器和判别器的输出。当输入来自真实标注样本{I f,L f}和分割预测样本{I f,L f*}时，输出1和0来标注像素点类别的真伪。
本申请实施例中，参数优化的过程包括两部分：
1、固定生成器参数θ S，优化判别器参数θ p；在对抗训练过程中，首先固定生成器参数θ S，利用生成器生成的分割预测样本送入到判别器中，由判别器判断真伪，并利用判别器误差函数，通过反向传播算法调整判别器参数θ p，提高自身鉴别能力。而判别器对应的误差函数为：
Figure PCTCN2019125428-appb-000025
在训练的过程中，判别器的参数不断优化，判别能力不断增强，越来越容易区分出生成器的生成图像，从而进入到下一个阶段。
2、固定判别器参数θ p，优化生成器参数θ S；网络将判别器的判别结果带入到生成器误差函数中，通过反向传播算法调整生成器参数θ S，使得生成器生成更高质量的分割结果，这样，生成器生成更加精确的结果来迷惑判别器。而生成器对应的误差函数为：
Figure PCTCN2019125428-appb-000026
重复上述两个优化步骤，最后，生成器与判别器之间形成纳什均衡，判别器无法区分图像是来自于生成器输出的分割预测样本{I f,L f*}还是真实标注样 本{I f,L f}，则生成对抗网络训练完成。
将图像级标注的待分割CSM图像输入训练好的生成对抗网络，通过生成对抗网络输出待分割CSM图像的像素级分割图像。
图7是本申请实施例提供的基于生成对抗网络的医学图像分割方法的硬件设备结构示意图。如图3所示，该设备包括一个或多个处理器以及存储器。以一个处理器为例，该设备还可以包括：输入系统和输出系统。
处理器、存储器、输入系统和输出系统可以通过总线或者其他方式连接，图3中以通过总线连接为例。
存储器作为一种非暂态计算机可读存储介质，可用于存储非暂态软件程序、非暂态计算机可执行程序以及模块。处理器通过运行存储在存储器中的非暂态软件程序、指令以及模块，从而执行电子设备的各种功能应用以及数据处理，即实现上述方法实施例的处理方法。
存储器可以包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需要的应用程序；存储数据区可存储数据等。此外，存储器可以包括高速随机存取存储器，还可以包括非暂态存储器，例如至少一个磁盘存储器件、闪存器件、或其他非暂态固态存储器件。在一些实施例中，存储器可选包括相对于处理器远程设置的存储器，这些远程存储器可以通过网络连接至处理系统。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
输入系统可接收输入的数字或字符信息，以及产生信号输入。输出系统可包括显示屏等显示设备。
所述一个或者多个模块存储在所述存储器中，当被所述一个或者多个处理器执行时，执行上述任一方法实施例的以下操作：
步骤a：分别采集其他医学图像的像素级标注样本以及待分割医学图像的图像级标注样本；
步骤b：通过所述其他医学图像的像素级标注样本以及待分割医学图像的图像级标注样本训练基于胶囊网络的生成对抗网络，所述生成对抗网络包括生成器和判别器；
步骤c：所述生成器对其他医学图像的像素级标注样本进行像素级特征提取，通过所述像素级特征对待分割医学图像的图像级标注样本进行处理，生成所述待分割医学图像的像素级标注样本，并基于所述像素级标注样本生成所述待分割医学图像的分割预测样本；
步骤d：将所述生成器生成的分割预测样本和待分割图像的真实标注样本一起输入到判别器进行“生成-对抗”训练，判别所述分割预测样本的真伪，并根据误差函数对生成器和判别器进行优化，得到训练好的生成对抗网络；
步骤e：将图像级标注的待分割医学图像输入训练好的生成对抗网络，通过所述生成对抗网络输出待分割医学图像的像素级分割图像。
上述产品可执行本申请实施例所提供的方法，具备执行方法相应的功能模块和有益效果。未在本实施例中详尽描述的技术细节，可参见本申请实施例提供的方法。
本申请实施例提供了一种非暂态(非易失性)计算机存储介质，所述计算机存储介质存储有计算机可执行指令，该计算机可执行指令可执行以下操作：
步骤a：分别采集其他医学图像的像素级标注样本以及待分割医学图像的图像级标注样本；
步骤b：通过所述其他医学图像的像素级标注样本以及待分割医学图像的图像级标注样本训练基于胶囊网络的生成对抗网络，所述生成对抗网络包括生成器和判别器；
步骤c：所述生成器对其他医学图像的像素级标注样本进行像素级特征提 取，通过所述像素级特征对待分割医学图像的图像级标注样本进行处理，生成所述待分割医学图像的像素级标注样本，并基于所述像素级标注样本生成所述待分割医学图像的分割预测样本；
步骤d：将所述生成器生成的分割预测样本和待分割图像的真实标注样本一起输入到判别器进行“生成-对抗”训练，判别所述分割预测样本的真伪，并根据误差函数对生成器和判别器进行优化，得到训练好的生成对抗网络；
步骤e：将图像级标注的待分割医学图像输入训练好的生成对抗网络，通过所述生成对抗网络输出待分割医学图像的像素级分割图像。
本申请实施例提供了一种计算机程序产品，所述计算机程序产品包括存储在非暂态计算机可读存储介质上的计算机程序，所述计算机程序包括程序指令，当所述程序指令被计算机执行时，使所述计算机执行以下操作：
步骤a：分别采集其他医学图像的像素级标注样本以及待分割医学图像的图像级标注样本；
步骤b：通过所述其他医学图像的像素级标注样本以及待分割医学图像的图像级标注样本训练基于胶囊网络的生成对抗网络，所述生成对抗网络包括生成器和判别器；
步骤c：所述生成器对其他医学图像的像素级标注样本进行像素级特征提取，通过所述像素级特征对待分割医学图像的图像级标注样本进行处理，生成所述待分割医学图像的像素级标注样本，并基于所述像素级标注样本生成所述待分割医学图像的分割预测样本；
步骤d：将所述生成器生成的分割预测样本和待分割图像的真实标注样本一起输入到判别器进行“生成-对抗”训练，判别所述分割预测样本的真伪，并根据误差函数对生成器和判别器进行优化，得到训练好的生成对抗网络；
步骤e：将图像级标注的待分割医学图像输入训练好的生成对抗网络，通过所述生成对抗网络输出待分割医学图像的像素级分割图像。
本申请实施例的基于生成对抗网络的医学图像分割方法、系统及电子设备通过融合胶囊机制对深度卷积神经网络进行优化，融合Capsule网络和级联瀑布的思想，在医学图像样本量小的情况下，生成新的训练图像样本，实现对低质量仅有图像级别标签的医学影像数据的语义分割，将学习到的分割知识从像素级标注的全标注数据，转移到图像级别的弱标注数据，从而提高了模型特征表达能力，扩展医学图像标注样本的可适用性，有效地降低分割模型对像素级标注数据的依赖，具有网络信息冗余少，特征提取充分的特点，在少量像素级标注样本的前提下，既能提高生成样本与真实样本对抗训练的效率，又可以有效实现高精度的像素级图像分割。
以上所述仅是本发明的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本发明的保护范围。

Claims (11)

  1. 一种基于生成对抗网络的医学图像分割方法，其特征在于，包括以下步骤：
    步骤a：分别采集其他医学图像的像素级标注样本以及待分割医学图像的图像级标注样本；
    步骤b：通过所述其他医学图像的像素级标注样本以及待分割医学图像的图像级标注样本训练基于胶囊网络的生成对抗网络，所述生成对抗网络包括生成器和判别器；
    步骤c：所述生成器对其他医学图像的像素级标注样本进行像素级特征提取，通过所述像素级特征对待分割医学图像的图像级标注样本进行处理，生成所述待分割医学图像的像素级标注样本，并基于所述像素级标注样本生成所述待分割医学图像的分割预测样本；
    步骤d：将所述生成器生成的分割预测样本和待分割图像的真实标注样本一起输入到判别器进行“生成-对抗”训练，判别所述分割预测样本的真伪，并根据误差函数对生成器和判别器进行优化，得到训练好的生成对抗网络；
    步骤e：将图像级标注的待分割医学图像输入训练好的生成对抗网络，通过所述生成对抗网络输出待分割医学图像的像素级分割图像。
  2. 根据权利要求1所述的基于生成对抗网络的医学图像分割方法，其特征在于，在所述步骤c中，所述生成器包括胶囊网络模块和区域定位网络，所述生成器生成待分割医学图像的分割预测样本具体包括：
    步骤b1：通过其他医学图像的像素级标注样本对胶囊网络模块进行预训练，得到无语义标签样本，通过所述无语义标签样本对待分割图像的图像级标注样本 进行处理，区分所述待分割图像的图像级标注样本的背景和有效分割区域；
    步骤b2：将所述待分割图像的图像级标注样本输入完成预训练后的胶囊网络模块，通过所述胶囊网络模块输出待分割图像的图像级标注样本的重建图像；
    步骤b3：区域定位网络利用卷积层的特征提取生成待分割图像的图像级标注样本的包含位置信息的特征图，并采用全局平均池化层，将权重(w 1,w 2…,w n)与特征图进行加权平均，得到待分割图像的图像级标注样本的区域定位特征图；
    步骤b4：根据所述重建图像和区域定位特征图执行自扩散算法，确定区域像素点分割线，得到待分割图像的图像级标注样本的分割预测样本。
  3. 根据权利要求2所述的基于生成对抗网络的医学图像分割方法，其特征在于，在所述步骤b2中，所述胶囊网络模块包括卷积层、PrimaryCaps层、DigitCaps层和解码层，所述胶囊网络模块采用单个胶囊神经元的输出向量，记录待分割图像的图像级标注样本分割区域边缘像素的方向和位置信息，采用矢量的非线性激活函数提取分类的概率值，确定待分割图像的图像级标注样本的分割区域与背景，计算边缘损失并输出待分割图像的图像级标注样本的重建图像。
  4. 根据权利要求2所述的基于生成对抗网络的医学图像分割方法，其特征在于，在所述步骤b4中，所述根据重建图像和区域定位特征图执行自扩散算法具体包括：在区域定位特征图中激活值越大的区域运用随机漫步的自扩散算法扩散像素点，利用区域定位特征图的输入点，计算图像上每个像素到输入点的高斯距离，并从中选择最优的路径，获得区域像素点的分割线，最终生成分割预测样本。
  5. 根据权利要求1至4任一项所述的基于生成对抗网络的医学图像分割方法，其特征在于，在所述步骤d中，所述判别器包括级联Cascade模块、Capsule网络模块和参数优化模块，所述判别器进行的“生成-对抗”训练具体包括：
    步骤d1：通过级联Cascade模块提取所述分割预测样本中标注错误的像素以及置信度低于设定阈值的关键像素以及对应的ground truth，并过滤标注正确且置信度高于设定阈值的像素；
    步骤d2：通过Capsule网络模块将提取到的关键像素以及对应的ground truth进行处理，并产生误差；
    步骤d3：所述参数优化模块利用Capsule网络模块产生的误差对生成器与判别器的网络参数进行优化；其中，对于给定的分割预测样本{I f,L f*}和对应的真实标注样本{I f,L f}，网络的整体误差函数为：
    Figure PCTCN2019125428-appb-100001
    上述公式中，θ S和θ p分别表示生成器和判别器的参数，J b表示二元交叉熵损失函数，O s和O p分别表示生成器和判别器的输出，当输入来自真实标注样本{I f,L f}和分割预测样本{I f,L f*}时，通过输出1和0来标注像素点类别的真伪。
  6. 一种基于生成对抗网络的医学图像分割系统，其特征在于，包括样本采集模块和生成对抗网络，
    样本采集模块：用于分别采集其他医学图像的像素级标注样本以及待分割医学图像的图像级标注样本；
    通过所述其他医学图像的像素级标注样本以及待分割医学图像的图像级标注样本训练基于胶囊网络的生成对抗网络；
    所述生成对抗网络包括生成器和判别器，所述生成器对其他医学图像的像素级标注样本进行像素级特征提取，通过所述像素级特征对待分割医学图像的图像级标注样本进行处理，生成所述待分割医学图像的像素级标注样本，并基于所述像素级标注样本生成所述待分割医学图像的分割预测样本；
    将所述生成器生成的分割预测样本和待分割图像的真实标注样本一起输入 到判别器进行“生成-对抗”训练，判别所述分割预测样本的真伪，并根据误差函数对生成器和判别器进行优化，得到训练好的生成对抗网络；
    将图像级标注的待分割医学图像输入训练好的生成对抗网络，通过所述生成对抗网络输出待分割医学图像的像素级分割图像。
  7. 根据权利要求6所述的基于生成对抗网络的医学图像分割系统，其特征在于，所述生成器包括预训练模块、胶囊网络模块、区域定位网络模块和样本生成模块：
    预训练模块：用于通过其他医学图像的像素级标注样本对胶囊网络模块进行预训练，得到无语义标签样本，通过所述无语义标签样本对待分割图像的图像级标注样本进行处理，区分所述待分割图像的图像级标注样本的背景和有效分割区域；
    胶囊网络模块：用于将所述待分割图像的图像级标注样本输入完成预训练后的胶囊网络模块，通过所述胶囊网络模块输出待分割图像的图像级标注样本的重建图像；
    区域定位网络：用于利用卷积层的特征提取生成待分割图像的图像级标注样本的包含位置信息的特征图，并采用全局平均池化层，将权重(w 1,w 2…,w n)与特征图进行加权平均，得到待分割图像的图像级标注样本的区域定位特征图；
    样本生成模块：用于根据所述重建图像和区域定位特征图执行自扩散算法，确定区域像素点分割线，得到待分割图像的图像级标注样本的分割预测样本。
  8. 根据权利要求7所述的基于生成对抗网络的医学图像分割系统，其特征在于，所述胶囊网络模块包括卷积层、PrimaryCaps层、DigitCaps层和解码层，所述胶囊网络模块采用单个胶囊神经元的输出向量，记录待分割图像的图像级标注样本分割区域边缘像素的方向和位置信息，采用矢量的非线性激活函数提取分 类的概率值，确定待分割图像的图像级标注样本的分割区域与背景，计算边缘损失并输出待分割图像的图像级标注样本的重建图像。
  9. 根据权利要求7所述的基于生成对抗网络的医学图像分割系统，其特征在于，所述样本生成模块根据重建图像和区域定位特征图执行自扩散算法具体包括：在区域定位特征图中激活值越大的区域运用随机漫步的自扩散算法扩散像素点，利用区域定位特征图的输入点，计算图像上每个像素到输入点的高斯距离，并从中选择最优的路径，获得区域像素点的分割线，最终生成分割预测样本。
  10. 根据权利要求6至9任一项所述的基于生成对抗网络的医学图像分割系统，其特征在于，所述判别器包括级联Cascade模块、Capsule网络模块和参数优化模块：
    级联Cascade模块：用于提取所述分割预测样本中标注错误的像素以及置信度低于设定阈值的关键像素以及对应的ground truth，并过滤标注正确且置信度高于设定阈值的像素；
    Capsule网络模块：用于将提取到的关键像素以及对应的ground truth进行处理，并产生误差；
    参数优化模块：用于利用Capsule网络模块产生的误差对生成器与判别器的网络参数进行优化；其中，对于给定的分割预测样本{I f,L f*}和对应的真实标注样本{I f,L f}，网络的整体误差函数为：
    Figure PCTCN2019125428-appb-100002
    上述公式中，θ S和θ p分别表示生成器和判别器的参数，J b表示二元交叉熵损失函数，O s和O p分别表示生成器和判别器的输出，当输入来自真实标注样本{I f,L f}和分割预测样本{I f,L f*}时，通过输出1和0来标注像素点类别的真伪。
  11. 一种电子设备，包括：
    至少一个处理器；以及
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行上述1至5任一项所述的基于生成对抗网络的医学图像分割方法的以下操作：
    步骤a：分别采集其他医学图像的像素级标注样本以及待分割医学图像的图像级标注样本；
    步骤b：通过所述其他医学图像的像素级标注样本以及待分割医学图像的图像级标注样本训练基于胶囊网络的生成对抗网络，所述生成对抗网络包括生成器和判别器；
    步骤c：所述生成器对其他医学图像的像素级标注样本进行像素级特征提取，通过所述像素级特征对待分割医学图像的图像级标注样本进行处理，生成所述待分割医学图像的像素级标注样本，并基于所述像素级标注样本生成所述待分割医学图像的分割预测样本；
    步骤d：将所述生成器生成的分割预测样本和待分割图像的真实标注样本一起输入到判别器进行“生成-对抗”训练，判别所述分割预测样本的真伪，并根据误差函数对生成器和判别器进行优化，得到训练好的生成对抗网络；
    步骤e：将图像级标注的待分割医学图像输入训练好的生成对抗网络，通过所述生成对抗网络输出待分割医学图像的像素级分割图像。
PCT/CN2019/125428 2019-08-01 2019-12-14 一种基于生成对抗网络的医学图像分割方法、系统及电子设备 WO2021017372A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910707712.X 2019-08-01
CN201910707712.XA CN110503654B (zh) 2019-08-01 2019-08-01 一种基于生成对抗网络的医学图像分割方法、系统及电子设备

Publications (1)

Publication Number Publication Date
WO2021017372A1 true WO2021017372A1 (zh) 2021-02-04

Family

ID=68586980

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/125428 WO2021017372A1 (zh) 2019-08-01 2019-12-14 一种基于生成对抗网络的医学图像分割方法、系统及电子设备

Country Status (2)

Country Link
CN (1) CN110503654B (zh)
WO (1) WO2021017372A1 (zh)

Cited By (47)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112950569A (zh) * 2021-02-25 2021-06-11 平安科技（深圳）有限公司 黑色素瘤图像识别方法、装置、计算机设备及存储介质
CN112991304A (zh) * 2021-03-23 2021-06-18 武汉大学 一种基于激光定向能量沉积监测系统的熔池溅射检测方法
CN113052369A (zh) * 2021-03-15 2021-06-29 北京农业智能装备技术研究中心 智能农机作业管理方法及系统
CN113066094A (zh) * 2021-03-09 2021-07-02 中国地质大学（武汉） 一种基于生成对抗网络的地理栅格智能化局部脱敏方法
CN113112454A (zh) * 2021-03-22 2021-07-13 西北工业大学 一种基于任务动态学习部分标记的医学图像分割方法
CN113130050A (zh) * 2021-04-20 2021-07-16 皖南医学院第一附属医院（皖南医学院弋矶山医院） 一种医学信息显示方法及显示系统
CN113171118A (zh) * 2021-04-06 2021-07-27 上海深至信息科技有限公司 一种基于生成式对抗网络的超声检查操作引导方法
CN113239978A (zh) * 2021-04-22 2021-08-10 科大讯飞股份有限公司 医学图像预处理模型与分析模型的相关方法和装置
CN113378472A (zh) * 2021-06-23 2021-09-10 合肥工业大学 一种基于生成对抗网络的混合边界电磁逆散射成像方法
CN113470046A (zh) * 2021-06-16 2021-10-01 浙江工业大学 一种面向医学图像超像素灰度纹理采样特征的图注意力网络分割方法
CN113469084A (zh) * 2021-07-07 2021-10-01 西安电子科技大学 基于对比生成对抗网络的高光谱图像分类方法
CN113553954A (zh) * 2021-07-23 2021-10-26 上海商汤智能科技有限公司 行为识别模型的训练方法及装置、设备、介质和程序产品
CN113628159A (zh) * 2021-06-16 2021-11-09 维库（厦门）信息技术有限公司 一种基于深度学习网络全自动训练方法、装置及存储介质
CN113705371A (zh) * 2021-08-10 2021-11-26 武汉理工大学 一种水上视觉场景分割方法及装置
CN113706546A (zh) * 2021-08-23 2021-11-26 浙江工业大学 一种基于轻量级孪生网络的医学图像分割方法及装置
CN113763394A (zh) * 2021-08-24 2021-12-07 同济大学 一种基于医疗风险的医学图像分割控制方法
CN113902674A (zh) * 2021-09-02 2022-01-07 北京邮电大学 医学影像分割方法和电子设备
CN113920127A (zh) * 2021-10-27 2022-01-11 华南理工大学 一种训练数据集独立的单样本图像分割方法和系统
CN113935977A (zh) * 2021-10-22 2022-01-14 河北工业大学 一种基于生成对抗网络的太阳能电池板缺陷生成方法
CN113936165A (zh) * 2021-09-07 2022-01-14 上海商涌科技有限公司 Ct图像的处理方法、终端及计算机存储介质
CN114066964A (zh) * 2021-11-17 2022-02-18 江南大学 一种基于深度学习的水产实时尺寸检测方法
CN114140368A (zh) * 2021-12-03 2022-03-04 天津大学 一种基于生成式对抗网络的多模态医学图像合成方法
CN114186735A (zh) * 2021-12-10 2022-03-15 沭阳鸿行照明有限公司 基于人工智能的消防应急照明灯布局优化方法
CN114240950A (zh) * 2021-11-23 2022-03-25 电子科技大学 一种基于深度神经网络的脑部肿瘤图像生成和分割方法
CN114331875A (zh) * 2021-12-09 2022-04-12 上海大学 一种基于对抗边缘学习的印刷工艺中图像出血位预测方法
CN114494322A (zh) * 2022-02-11 2022-05-13 合肥工业大学 一种基于图像融合技术的多模态图像分割方法
CN114549554A (zh) * 2022-02-22 2022-05-27 山东融瓴科技集团有限公司 基于风格不变性的空气污染源分割方法
CN114549842A (zh) * 2022-04-22 2022-05-27 山东建筑大学 基于不确定性知识域自适应的半监督图像分割方法及系统
CN114677515A (zh) * 2022-04-25 2022-06-28 电子科技大学 基于类间相似性的弱监督语义分割方法
CN114821229A (zh) * 2022-04-14 2022-07-29 江苏集萃清联智控科技有限公司 基于条件生成对抗网络的水下声学数据集增广方法及系统
CN114818734A (zh) * 2022-05-25 2022-07-29 清华大学 基于目标-属性-关系的对抗场景语义分析方法以及装置
CN114882047A (zh) * 2022-04-19 2022-08-09 厦门大学 一种基于半监督与Transformers的医学图像分割方法及系统
CN114897782A (zh) * 2022-04-13 2022-08-12 华南理工大学 基于生成式对抗网络的胃癌病理切片图像分割预测方法
CN115081920A (zh) * 2022-07-08 2022-09-20 华南农业大学 考勤签到调度管理方法、系统、设备及存储介质
CN115187467A (zh) * 2022-05-31 2022-10-14 北京昭衍新药研究中心股份有限公司 一种基于生成对抗网络的增强型虚拟图像数据生成方法
CN115272136A (zh) * 2022-09-27 2022-11-01 广州卓腾科技有限公司 基于大数据的证件照眼镜反光消除方法、装置、介质及设备
CN115439846A (zh) * 2022-08-09 2022-12-06 北京邮电大学 图像的分割方法、装置、电子设备及介质
CN115546239A (zh) * 2022-11-30 2022-12-30 珠海横琴圣澳云智科技有限公司 基于边界注意力与距离变换的目标分割方法和装置
CN115880440A (zh) * 2023-01-31 2023-03-31 中国科学院自动化研究所 一种基于生成对抗网络的磁粒子三维重建成像方法
WO2023097640A1 (zh) * 2021-12-03 2023-06-08 宁德时代新能源科技股份有限公司 用于生成包含特定特征的图像样本的方法和系统
CN117094986A (zh) * 2023-10-13 2023-11-21 中山大学深圳研究院 基于小样本的自适应缺陷检测方法及终端设备
CN117093548A (zh) * 2023-10-20 2023-11-21 公诚管理咨询有限公司 一种招投标管理稽核系统
CN117152138A (zh) * 2023-10-30 2023-12-01 陕西惠宾电子科技有限公司 一种基于无监督学习的医学图像肿瘤目标检测方法
CN117523318A (zh) * 2023-12-26 2024-02-06 宁波微科光电股份有限公司 一种抗光干扰的地铁屏蔽门异物检测方法、装置及介质
CN117726642A (zh) * 2024-02-07 2024-03-19 中国科学院宁波材料技术与工程研究所 一种用于光学相干断层扫描图像的高反射病灶分割方法和装置
CN118015021A (zh) * 2024-04-07 2024-05-10 安徽农业大学 基于滑动窗口的主动域自适应跨模态医学图像分割方法
CN117726642B (zh) * 2024-02-07 2024-05-31 中国科学院宁波材料技术与工程研究所 一种用于光学相干断层扫描图像的高反射病灶分割方法和装置

Families Citing this family (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110503654B (zh) * 2019-08-01 2022-04-26 中国科学院深圳先进技术研究院 一种基于生成对抗网络的医学图像分割方法、系统及电子设备
CN111067522A (zh) * 2019-12-16 2020-04-28 中国科学院深圳先进技术研究院 大脑成瘾结构图谱评估方法及装置
CN111160441B (zh) * 2019-12-24 2024-03-26 上海联影智能医疗科技有限公司 分类方法、计算机设备和存储介质
CN111275686B (zh) * 2020-01-20 2023-05-26 中山大学 用于人工神经网络训练的医学图像数据的生成方法及装置
CN111383215A (zh) * 2020-03-10 2020-07-07 图玛深维医疗科技(北京)有限公司 一种基于生成对抗网络的病灶检测模型的训练方法
CN111383217B (zh) * 2020-03-11 2023-08-29 深圳先进技术研究院 大脑成瘾性状评估的可视化方法、装置及介质
WO2021179205A1 (zh) * 2020-03-11 2021-09-16 深圳先进技术研究院 医学图像分割方法、医学图像分割装置及终端设备
CN111429464B (zh) * 2020-03-11 2023-04-25 深圳先进技术研究院 医学图像分割方法、医学图像分割装置及终端设备
CN111292322B (zh) * 2020-03-19 2024-03-01 中国科学院深圳先进技术研究院 医学图像处理方法、装置、设备及存储介质
CN111436936B (zh) * 2020-04-29 2021-07-27 浙江大学 基于mri的ct图像重建方法
CN111598900B (zh) * 2020-05-18 2022-08-09 腾讯医疗健康(深圳)有限公司 一种图像区域分割模型训练方法、分割方法和装置
CN111798471B (zh) * 2020-07-27 2024-04-02 中科智脑(北京)技术有限公司 图像语义分割网络的训练方法
CN111932555A (zh) * 2020-07-31 2020-11-13 商汤集团有限公司 一种图像处理方法及装置、计算机可读存储介质
CN111899251A (zh) * 2020-08-06 2020-11-06 中国科学院深圳先进技术研究院 一种区分伪造来源和目标区域的copy-move型伪造图像检测方法
CN112150478B (zh) * 2020-08-31 2021-06-22 温州医科大学 一种构建半监督图像分割框架的方法及系统
CN112420205A (zh) * 2020-12-08 2021-02-26 医惠科技有限公司 实体识别模型生成方法、装置及计算机可读存储介质
CN112560925A (zh) * 2020-12-10 2021-03-26 中国科学院深圳先进技术研究院 一种复杂场景目标检测数据集构建方法及系统
CN112508835B (zh) * 2020-12-10 2024-04-26 深圳先进技术研究院 一种基于gan的无造影剂医学图像增强建模方法
CN112507950B (zh) * 2020-12-18 2021-09-03 中国科学院空天信息创新研究院 一种生成对抗式多任务多要素样本自动标注方法及装置
CN112686906B (zh) * 2020-12-25 2022-06-14 山东大学 基于均匀分布迁移引导的图像分割方法及系统
CN112890766A (zh) * 2020-12-31 2021-06-04 山东省千佛山医院 一种乳腺癌辅助治疗设备
CN112837338B (zh) * 2021-01-12 2022-06-21 浙江大学 一种基于生成对抗网络的半监督医学图像分割方法
CN112749791A (zh) * 2021-01-22 2021-05-04 重庆理工大学 一种基于图神经网络和胶囊网络的链路预测方法
CN113160243A (zh) * 2021-03-24 2021-07-23 联想(北京)有限公司 一种图像分割方法及电子设备
CN113284088B (zh) * 2021-04-02 2024-03-29 中国科学院深圳先进技术研究院 一种csm图像分割方法、装置、终端设备及存储介质
CN113223010B (zh) * 2021-04-22 2024-02-27 北京大学口腔医学院 口腔图像多组织全自动分割的方法和系统
CN113052840B (zh) * 2021-04-30 2024-02-02 江苏赛诺格兰医疗科技有限公司 一种基于低信噪比pet图像的处理方法
CN113487617A (zh) * 2021-07-26 2021-10-08 推想医疗科技股份有限公司 数据处理方法、装置、电子设备以及存储介质
CN114021698A (zh) * 2021-10-30 2022-02-08 河南省鼎信信息安全等级测评有限公司 基于胶囊生成对抗网络的恶意域名训练样本扩充方法及装置
CN113850804B (zh) * 2021-11-29 2022-03-18 北京鹰瞳科技发展股份有限公司 基于生成对抗网络的视网膜图像生成系统及方法
CN114581709A (zh) * 2022-03-02 2022-06-03 深圳硅基智能科技有限公司 识别医学图像中的目标的模型训练、方法、设备及介质
CN116168242B (zh) * 2023-02-08 2023-12-01 阿里巴巴（中国）有限公司 像素级标签的生成方法、模型训练方法及设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108062753A (zh) * 2017-12-29 2018-05-22 重庆理工大学 基于深度对抗学习的无监督域自适应脑肿瘤语义分割方法
CN108198179A (zh) * 2018-01-03 2018-06-22 华南理工大学 一种生成对抗网络改进的ct医学图像肺结节检测方法
CN108932484A (zh) * 2018-06-20 2018-12-04 华南理工大学 一种基于Capsule Net的人脸表情识别方法
CN109242849A (zh) * 2018-09-26 2019-01-18 上海联影智能医疗科技有限公司 医学图像处理方法、装置、系统和存储介质
US20190147582A1 (en) * 2017-11-15 2019-05-16 Toyota Research Institute, Inc. Adversarial learning of photorealistic post-processing of simulation with privileged information
CN110503654A (zh) * 2019-08-01 2019-11-26 中国科学院深圳先进技术研究院 一种基于生成对抗网络的医学图像分割方法、系统及电子设备

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10726304B2 (en) * 2017-09-08 2020-07-28 Ford Global Technologies, Llc Refining synthetic data with a generative adversarial network using auxiliary inputs
WO2019118613A1 (en) * 2017-12-12 2019-06-20 Oncoustics Inc. Machine learning to extract quantitative biomarkers from ultrasound rf spectrums
CN108961217B (zh) * 2018-06-08 2022-09-16 南京大学 一种基于正例训练的表面缺陷检测方法
CN109063724B (zh) * 2018-06-12 2022-02-22 中国科学院深圳先进技术研究院 一种增强型生成式对抗网络以及目标样本识别方法
CN109344833B (zh) * 2018-09-04 2020-12-18 中国科学院深圳先进技术研究院 医学图像分割方法、分割系统及计算机可读存储介质
CN109584337B (zh) * 2018-11-09 2022-03-29 暨南大学 一种基于条件胶囊生成对抗网络的图像生成方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190147582A1 (en) * 2017-11-15 2019-05-16 Toyota Research Institute, Inc. Adversarial learning of photorealistic post-processing of simulation with privileged information
CN108062753A (zh) * 2017-12-29 2018-05-22 重庆理工大学 基于深度对抗学习的无监督域自适应脑肿瘤语义分割方法
CN108198179A (zh) * 2018-01-03 2018-06-22 华南理工大学 一种生成对抗网络改进的ct医学图像肺结节检测方法
CN108932484A (zh) * 2018-06-20 2018-12-04 华南理工大学 一种基于Capsule Net的人脸表情识别方法
CN109242849A (zh) * 2018-09-26 2019-01-18 上海联影智能医疗科技有限公司 医学图像处理方法、装置、系统和存储介质
CN110503654A (zh) * 2019-08-01 2019-11-26 中国科学院深圳先进技术研究院 一种基于生成对抗网络的医学图像分割方法、系统及电子设备

Cited By (81)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112950569B (zh) * 2021-02-25 2023-07-25 平安科技（深圳）有限公司 黑色素瘤图像识别方法、装置、计算机设备及存储介质
CN112950569A (zh) * 2021-02-25 2021-06-11 平安科技（深圳）有限公司 黑色素瘤图像识别方法、装置、计算机设备及存储介质
CN113066094B (zh) * 2021-03-09 2024-01-30 中国地质大学（武汉） 一种基于生成对抗网络的地理栅格智能化局部脱敏方法
CN113066094A (zh) * 2021-03-09 2021-07-02 中国地质大学（武汉） 一种基于生成对抗网络的地理栅格智能化局部脱敏方法
CN113052369B (zh) * 2021-03-15 2024-05-10 北京农业智能装备技术研究中心 智能农机作业管理方法及系统
CN113052369A (zh) * 2021-03-15 2021-06-29 北京农业智能装备技术研究中心 智能农机作业管理方法及系统
CN113112454B (zh) * 2021-03-22 2024-03-19 西北工业大学 一种基于任务动态学习部分标记的医学图像分割方法
CN113112454A (zh) * 2021-03-22 2021-07-13 西北工业大学 一种基于任务动态学习部分标记的医学图像分割方法
CN112991304A (zh) * 2021-03-23 2021-06-18 武汉大学 一种基于激光定向能量沉积监测系统的熔池溅射检测方法
CN113171118A (zh) * 2021-04-06 2021-07-27 上海深至信息科技有限公司 一种基于生成式对抗网络的超声检查操作引导方法
CN113130050A (zh) * 2021-04-20 2021-07-16 皖南医学院第一附属医院（皖南医学院弋矶山医院） 一种医学信息显示方法及显示系统
CN113130050B (zh) * 2021-04-20 2023-11-24 皖南医学院第一附属医院（皖南医学院弋矶山医院） 一种医学信息显示方法及显示系统
CN113239978A (zh) * 2021-04-22 2021-08-10 科大讯飞股份有限公司 医学图像预处理模型与分析模型的相关方法和装置
CN113239978B (zh) * 2021-04-22 2024-06-04 科大讯飞股份有限公司 医学图像预处理模型与分析模型的相关方法和装置
CN113628159A (zh) * 2021-06-16 2021-11-09 维库（厦门）信息技术有限公司 一种基于深度学习网络全自动训练方法、装置及存储介质
CN113470046B (zh) * 2021-06-16 2024-04-16 浙江工业大学 一种面向医学图像超像素灰度纹理采样特征的图注意力网络分割方法
CN113470046A (zh) * 2021-06-16 2021-10-01 浙江工业大学 一种面向医学图像超像素灰度纹理采样特征的图注意力网络分割方法
CN113378472B (zh) * 2021-06-23 2022-09-13 合肥工业大学 一种基于生成对抗网络的混合边界电磁逆散射成像方法
CN113378472A (zh) * 2021-06-23 2021-09-10 合肥工业大学 一种基于生成对抗网络的混合边界电磁逆散射成像方法
CN113469084A (zh) * 2021-07-07 2021-10-01 西安电子科技大学 基于对比生成对抗网络的高光谱图像分类方法
CN113469084B (zh) * 2021-07-07 2023-06-30 西安电子科技大学 基于对比生成对抗网络的高光谱图像分类方法
CN113553954A (zh) * 2021-07-23 2021-10-26 上海商汤智能科技有限公司 行为识别模型的训练方法及装置、设备、介质和程序产品
CN113705371B (zh) * 2021-08-10 2023-12-01 武汉理工大学 一种水上视觉场景分割方法及装置
CN113705371A (zh) * 2021-08-10 2021-11-26 武汉理工大学 一种水上视觉场景分割方法及装置
CN113706546B (zh) * 2021-08-23 2024-03-19 浙江工业大学 一种基于轻量级孪生网络的医学图像分割方法及装置
CN113706546A (zh) * 2021-08-23 2021-11-26 浙江工业大学 一种基于轻量级孪生网络的医学图像分割方法及装置
CN113763394B (zh) * 2021-08-24 2024-03-29 同济大学 一种基于医疗风险的医学图像分割控制方法
CN113763394A (zh) * 2021-08-24 2021-12-07 同济大学 一种基于医疗风险的医学图像分割控制方法
CN113902674A (zh) * 2021-09-02 2022-01-07 北京邮电大学 医学影像分割方法和电子设备
CN113902674B (zh) * 2021-09-02 2024-05-24 北京邮电大学 医学影像分割方法和电子设备
CN113936165B (zh) * 2021-09-07 2024-06-07 上海商涌科技有限公司 Ct图像的处理方法、终端及计算机存储介质
CN113936165A (zh) * 2021-09-07 2022-01-14 上海商涌科技有限公司 Ct图像的处理方法、终端及计算机存储介质
CN113935977A (zh) * 2021-10-22 2022-01-14 河北工业大学 一种基于生成对抗网络的太阳能电池板缺陷生成方法
CN113920127A (zh) * 2021-10-27 2022-01-11 华南理工大学 一种训练数据集独立的单样本图像分割方法和系统
CN113920127B (zh) * 2021-10-27 2024-04-23 华南理工大学 一种训练数据集独立的单样本图像分割方法和系统
CN114066964B (zh) * 2021-11-17 2024-04-05 江南大学 一种基于深度学习的水产实时尺寸检测方法
CN114066964A (zh) * 2021-11-17 2022-02-18 江南大学 一种基于深度学习的水产实时尺寸检测方法
CN114240950B (zh) * 2021-11-23 2023-04-07 电子科技大学 一种基于深度神经网络的脑部肿瘤图像生成和分割方法
CN114240950A (zh) * 2021-11-23 2022-03-25 电子科技大学 一种基于深度神经网络的脑部肿瘤图像生成和分割方法
CN114140368A (zh) * 2021-12-03 2022-03-04 天津大学 一种基于生成式对抗网络的多模态医学图像合成方法
WO2023097640A1 (zh) * 2021-12-03 2023-06-08 宁德时代新能源科技股份有限公司 用于生成包含特定特征的图像样本的方法和系统
US11804037B1 (en) 2021-12-03 2023-10-31 Contemporary Amperex Technology Co., Limited Method and system for generating image sample having specific feature
CN114140368B (zh) * 2021-12-03 2024-04-23 天津大学 一种基于生成式对抗网络的多模态医学图像合成方法
CN114331875A (zh) * 2021-12-09 2022-04-12 上海大学 一种基于对抗边缘学习的印刷工艺中图像出血位预测方法
CN114186735A (zh) * 2021-12-10 2022-03-15 沭阳鸿行照明有限公司 基于人工智能的消防应急照明灯布局优化方法
CN114186735B (zh) * 2021-12-10 2023-10-20 沭阳鸿行照明有限公司 基于人工智能的消防应急照明灯布局优化方法
CN114494322A (zh) * 2022-02-11 2022-05-13 合肥工业大学 一种基于图像融合技术的多模态图像分割方法
CN114494322B (zh) * 2022-02-11 2024-03-01 合肥工业大学 一种基于图像融合技术的多模态图像分割方法
CN114549554A (zh) * 2022-02-22 2022-05-27 山东融瓴科技集团有限公司 基于风格不变性的空气污染源分割方法
CN114549554B (zh) * 2022-02-22 2024-05-14 山东融瓴科技集团有限公司 基于风格不变性的空气污染源分割方法
CN114897782B (zh) * 2022-04-13 2024-04-23 华南理工大学 基于生成式对抗网络的胃癌病理切片图像分割预测方法
CN114897782A (zh) * 2022-04-13 2022-08-12 华南理工大学 基于生成式对抗网络的胃癌病理切片图像分割预测方法
CN114821229A (zh) * 2022-04-14 2022-07-29 江苏集萃清联智控科技有限公司 基于条件生成对抗网络的水下声学数据集增广方法及系统
CN114821229B (zh) * 2022-04-14 2023-07-28 江苏集萃清联智控科技有限公司 基于条件生成对抗网络的水下声学数据集增广方法及系统
CN114882047A (zh) * 2022-04-19 2022-08-09 厦门大学 一种基于半监督与Transformers的医学图像分割方法及系统
CN114549842B (zh) * 2022-04-22 2022-08-02 山东建筑大学 基于不确定性知识域自适应的半监督图像分割方法及系统
CN114549842A (zh) * 2022-04-22 2022-05-27 山东建筑大学 基于不确定性知识域自适应的半监督图像分割方法及系统
CN114677515A (zh) * 2022-04-25 2022-06-28 电子科技大学 基于类间相似性的弱监督语义分割方法
CN114677515B (zh) * 2022-04-25 2023-05-26 电子科技大学 基于类间相似性的弱监督语义分割方法
CN114818734B (zh) * 2022-05-25 2023-10-31 清华大学 基于目标-属性-关系的对抗场景语义分析方法以及装置
CN114818734A (zh) * 2022-05-25 2022-07-29 清华大学 基于目标-属性-关系的对抗场景语义分析方法以及装置
CN115187467A (zh) * 2022-05-31 2022-10-14 北京昭衍新药研究中心股份有限公司 一种基于生成对抗网络的增强型虚拟图像数据生成方法
CN115081920A (zh) * 2022-07-08 2022-09-20 华南农业大学 考勤签到调度管理方法、系统、设备及存储介质
CN115439846B (zh) * 2022-08-09 2023-04-25 北京邮电大学 图像的分割方法、装置、电子设备及介质
CN115439846A (zh) * 2022-08-09 2022-12-06 北京邮电大学 图像的分割方法、装置、电子设备及介质
CN115272136A (zh) * 2022-09-27 2022-11-01 广州卓腾科技有限公司 基于大数据的证件照眼镜反光消除方法、装置、介质及设备
CN115546239A (zh) * 2022-11-30 2022-12-30 珠海横琴圣澳云智科技有限公司 基于边界注意力与距离变换的目标分割方法和装置
CN115546239B (zh) * 2022-11-30 2023-04-07 珠海横琴圣澳云智科技有限公司 基于边界注意力与距离变换的目标分割方法和装置
CN115880440A (zh) * 2023-01-31 2023-03-31 中国科学院自动化研究所 一种基于生成对抗网络的磁粒子三维重建成像方法
CN115880440B (zh) * 2023-01-31 2023-04-28 中国科学院自动化研究所 一种基于生成对抗网络的磁粒子三维重建成像方法
CN117094986B (zh) * 2023-10-13 2024-04-05 中山大学深圳研究院 基于小样本的自适应缺陷检测方法及终端设备
CN117094986A (zh) * 2023-10-13 2023-11-21 中山大学深圳研究院 基于小样本的自适应缺陷检测方法及终端设备
CN117093548A (zh) * 2023-10-20 2023-11-21 公诚管理咨询有限公司 一种招投标管理稽核系统
CN117093548B (zh) * 2023-10-20 2024-01-26 公诚管理咨询有限公司 一种招投标管理稽核系统
CN117152138A (zh) * 2023-10-30 2023-12-01 陕西惠宾电子科技有限公司 一种基于无监督学习的医学图像肿瘤目标检测方法
CN117152138B (zh) * 2023-10-30 2024-01-16 陕西惠宾电子科技有限公司 一种基于无监督学习的医学图像肿瘤目标检测方法
CN117523318B (zh) * 2023-12-26 2024-04-16 宁波微科光电股份有限公司 一种抗光干扰的地铁屏蔽门异物检测方法、装置及介质
CN117523318A (zh) * 2023-12-26 2024-02-06 宁波微科光电股份有限公司 一种抗光干扰的地铁屏蔽门异物检测方法、装置及介质
CN117726642A (zh) * 2024-02-07 2024-03-19 中国科学院宁波材料技术与工程研究所 一种用于光学相干断层扫描图像的高反射病灶分割方法和装置
CN117726642B (zh) * 2024-02-07 2024-05-31 中国科学院宁波材料技术与工程研究所 一种用于光学相干断层扫描图像的高反射病灶分割方法和装置
CN118015021A (zh) * 2024-04-07 2024-05-10 安徽农业大学 基于滑动窗口的主动域自适应跨模态医学图像分割方法

Also Published As

Publication number Publication date
CN110503654A (zh) 2019-11-26
CN110503654B (zh) 2022-04-26

Similar Documents

Publication Publication Date Title
WO2021017372A1 (zh) 一种基于生成对抗网络的医学图像分割方法、系统及电子设备
Yamashita et al. Convolutional neural networks: an overview and application in radiology
WO2020238734A1 (zh) 图像分割模型的训练方法、装置、计算机设备和存储介质
Ma et al. Automatic segmentation of the prostate on CT images using deep learning and multi-atlas fusion
Roth et al. A new 2.5 D representation for lymph node detection using random sets of deep convolutional neural network observations
US20220036564A1 (en) Method of classifying lesion of chest x-ray radiograph based on data normalization and local patch and apparatus thereof
Niyaz et al. Advances in deep learning techniques for medical image analysis
Aslam et al. Neurological Disorder Detection Using OCT Scan Image of Eye
Ye et al. Medical image diagnosis of prostate tumor based on PSP-Net+ VGG16 deep learning network
Raut et al. Gastrointestinal tract disease segmentation and classification in wireless capsule endoscopy using intelligent deep learning model
Zhai et al. An improved full convolutional network combined with conditional random fields for brain MR image segmentation algorithm and its 3D visualization analysis
Ramkumar et al. Multi res U-Net based image segmentation of pulmonary tuberculosis using CT images
Xiao et al. PET and CT image fusion of lung cancer with siamese pyramid fusion network
Chu et al. Epitomized summarization of wireless capsule endoscopic videos for efficient visualization
Mohan et al. Comparison of Convolutional Neural Network for Classifying Lung Diseases from Chest CT Images
CN113222989A (zh) 一种图像分级方法、装置、存储介质及电子设备
Tawfeeq et al. Predication of Most Significant Features in Medical Image by Utilized CNN and Heatmap.
Xu et al. A Tuberculosis Detection Method Using Attention and Sparse R-CNN.
Baskaran et al. MSRFNet for skin lesion segmentation and deep learning with hybrid optimization for skin cancer detection
Sreelekshmi et al. A Review on Multimodal Medical Image Fusion
US20220254012A1 (en) Methods, devices, and systems for determining presence of appendicitis
Nasim et al. Review on multimodality of different medical image fusion techniques
Li et al. Labyrinth morphological modeling and its application on unreferenced segmentation assessment
Huang et al. FR-nnUNet: A Novel Medical Segmentation Network Based on the Fuzzy Regions Recognition Scheme and Improved nnU-Net
Xu et al. The Application of Medical Imaging on Disabled Athletes in Winter Paralympic Games: A Systematic Review

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19939834

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19939834

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 19939834

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 16.02.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 19939834

Country of ref document: EP

Kind code of ref document: A1