WO2022178949A1 - Semantic segmentation method and apparatus for electron microtomography data, device, and medium - Google Patents

Abstract

A semantic segmentation method and apparatus for electron microtomography data, a device, and a medium, relating to the technical field of digital medical treatment. The method comprises: using a cell protein semantic segmentation model to separately perform protein semantic segmentation on each piece of electron microtomography data to be analyzed in a plurality of pieces of electron microtomography data to be analyzed obtained by segmenting the same piece of cell electron microtomography data to be segmented, to obtain a protein semantic segmentation result set, alternative training of adversarial and semi-supervised learning being performed on the basis of a generative network and a discriminant network, and the generative network obtained by means of the alternative training of adversarial and semi-supervised learning being taken as the cell protein semantic segmentation model; and performing data splicing according to the protein semantic segmentation result set to obtain a target protein semantic segmentation result. The data volume for performing protein semantic segmentation using the model each time is decreased, requirements for GPU hardware conditions of model training are reduced, and performance of training having less data volume is effectively improved.

Description

Semantic segmentation method, device, equipment and medium for electron microtomography data

This application claims the priority of the Chinese patent application filed on February 26, 2021 with the application number 2021102196153 and the invention titled "Method, Apparatus, Equipment and Medium for Semantic Segmentation of Electron Microtomography Data", all of which The contents are incorporated herein by reference.

technical field

The present application relates to the field of digital medical technology, and in particular, to a method, device, device and medium for semantic segmentation of electron microscopic tomography data.

Background technique

Electron microtomography data is an important type of 3D (three-dimensional) data in the field of computational biology. Electron microtomography is applicable to a wide range of scales, including proteins at the molecular level, organelles at the subcellular level, and tissue structures at the cellular level. It can be used to obtain the three-dimensional spatial distribution of important molecular machines in the cellular environment and assembly, thereby providing important and beneficial information for a deep understanding of the interaction mechanism of these molecular machines. The semantic segmentation task of cell electron microtomography data is of great significance for studying the spatial distribution and 3D morphology of macromolecular structures in cells.

The inventor realizes that the related data sets of cell electron microtomography in the prior art contain a small amount of data and a single 3D data volume is large, resulting in less related research on the semantic segmentation task of cell electron microtomography data and difficulty in model training. , GPU (graphics processing unit) hardware conditions are difficult to support.

technical problem

In the prior art, cell electron microtomography-related datasets contain a small amount of data and a single 3D data volume is relatively large, resulting in less research on semantic segmentation tasks of cell electron microtomography data, difficulty in model training, and GPU hardware conditions. Difficult to support technical issues.

technical solutions

The main purpose of this application is to provide a semantic segmentation method, device, equipment and medium for electron microtomography data, which aims to solve the problem that the related data sets of cell electron microtomography in the prior art contain a small amount of data and a single 3D data volume is relatively small. This leads to the technical problems of less research on the semantic segmentation task of cell electron microtomography data, difficult model training, and unsupported GPU hardware conditions.

In order to achieve the above purpose of the invention, the present application proposes a method for semantic segmentation of electron microscopic tomography data, the method comprising:

Acquiring a plurality of electron microtomography data to be analyzed obtained by segmenting the same cell electron microtomography data to be segmented;

The cell protein semantic segmentation model is used to perform protein semantic segmentation on each of the electron microtomography data to be analyzed in the plurality of electron microtomography data to be analyzed, so as to obtain the corresponding The set of protein semantic segmentation results, wherein, based on the generation network and the discriminant network, the alternating training of confrontation and semi-supervised learning is performed, and the generation network obtained by the alternating training of confrontation and semi-supervised learning is used as the cell protein semantic segmentation model;

Perform data splicing according to the set of protein semantic segmentation results corresponding to the plurality of electron microscopic tomographic data to be analyzed, to obtain target protein semantic segmentation results corresponding to the plurality of to-be-analyzed electron microscopic tomographic data.

The present application also proposes a semantic segmentation device for electron microtomography data, the device comprising:

The data acquisition module is used to acquire a plurality of electron microtomography data to be analyzed obtained by segmenting the same cell electron microtomography data to be segmented;

A protein semantic segmentation module is used to perform protein semantic segmentation on each of the plurality of electron microscopic tomography data to be analyzed by using a cellular protein semantic segmentation model, and obtain the plurality of to-be-analyzed electron microtomography data. The set of protein semantic segmentation results corresponding to the electron microscopic tomography data of the Cellular protein semantic segmentation model;

A data splicing module, configured to perform data splicing according to the protein semantic segmentation result set corresponding to the plurality of electron microscopic tomographic data to be analyzed, to obtain the target protein semantics corresponding to the plurality of to-be-analyzed electron microscopic tomographic data Split result.

The present application also proposes a computer device, including a memory and a processor, the memory stores a computer program, and the processor implements the following method steps when executing the computer program:

Acquiring a plurality of electron microtomography data to be analyzed obtained by segmenting the same cell electron microtomography data to be segmented;

The cell protein semantic segmentation model is used to perform protein semantic segmentation on each of the electron microtomography data to be analyzed in the plurality of electron microtomography data to be analyzed, so as to obtain the corresponding The set of protein semantic segmentation results, wherein, based on the generation network and the discriminant network, the alternating training of confrontation and semi-supervised learning is performed, and the generation network obtained by the alternating training of confrontation and semi-supervised learning is used as the cell protein semantic segmentation model;

Perform data splicing according to the set of protein semantic segmentation results corresponding to the plurality of electron microscopic tomographic data to be analyzed, to obtain target protein semantic segmentation results corresponding to the plurality of to-be-analyzed electron microscopic tomographic data.

The present application also proposes a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the following method steps are implemented:

Acquiring a plurality of electron microtomography data to be analyzed obtained by segmenting the same cell electron microtomography data to be segmented;

The cell protein semantic segmentation model is used to perform protein semantic segmentation on each of the electron microtomography data to be analyzed in the plurality of electron microtomography data to be analyzed, so as to obtain the corresponding The set of protein semantic segmentation results, wherein, based on the generation network and the discriminant network, the alternating training of confrontation and semi-supervised learning is performed, and the generation network obtained by the alternating training of confrontation and semi-supervised learning is used as the cell protein semantic segmentation model;

Perform data splicing according to the set of protein semantic segmentation results corresponding to the plurality of electron microscopic tomographic data to be analyzed, to obtain target protein semantic segmentation results corresponding to the plurality of to-be-analyzed electron microscopic tomographic data.

beneficial effect

The semantic segmentation method, device, equipment and medium of electron microtomography data of the present application are obtained by segmenting a plurality of electron microtomography data to be analyzed according to the same cell electron microtomography data to be segmented, using The cellular protein semantic segmentation model separately performs protein semantic segmentation on each of the electron micro tomographic data to be analyzed, and obtains the protein semantic segmentation results corresponding to the plurality of electron micro tomographic data to be analyzed. Set, perform data splicing according to the set of protein semantic segmentation results corresponding to multiple electron microtomography data to be analyzed, and obtain the target protein semantic segmentation results corresponding to multiple electron microtomography data to be analyzed. The amount of data for protein semantic segmentation by the semantic segmentation model reduces the requirements for GPU hardware conditions for model training; through the alternate training of adversarial and semi-supervised learning based on the generative network and discriminant network, the alternate training of adversarial and semi-supervised learning is obtained. The generative network is used as a semantic segmentation model of cellular proteins. Based on adversarial training, the performance of training with a small amount of data is effectively improved, and the generalization effect of the model is enhanced. Semi-supervised learning training is used to enhance the performance of the model with unlabeled data.

Description of drawings

1 is a schematic flowchart of a method for semantic segmentation of electron microtomography data according to an embodiment of the present application;

FIG. 2 is a schematic structural block diagram of a device for semantic segmentation of electron microtomography data according to an embodiment of the application;

FIG. 3 is a schematic structural block diagram of a computer device according to an embodiment of the present application.

The realization, functional features and advantages of the present application will be further described with reference to the accompanying drawings in conjunction with the embodiments.

Embodiments of the present invention

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

In order to solve the problem that the related data sets of cell electron microtomography in the prior art contain a small amount of data and a single 3D data volume is large, resulting in less related research on the semantic segmentation task of cell electron microtomography data, difficulty in model training, GPU The technical problem that hardware conditions are difficult to support, the present application proposes a semantic segmentation method of electron microscopic tomography data, the method is applied in the field of artificial intelligence technology, and the method can also be applied in the field of digital medical technology. The method for semantic segmentation of electron microtomography data is segmented according to the same cell electron microtomography data to be segmented, and then the protein semantic segmentation is performed, which reduces the amount of data for performing protein semantic segmentation by using a model each time, and reduces the cost of protein semantic segmentation. For the requirements of GPU hardware conditions for model training, the model is obtained by alternating training of adversarial and semi-supervised learning based on the generative network and discriminant network. Semi-supervised learning training enhances model performance with unlabeled data.

Referring to FIG. 1 , an embodiment of the present application provides a method for semantic segmentation of electron microscopic tomography data, and the method includes:

S1: Acquire a plurality of electron microtomography data to be analyzed obtained by segmenting the same cell electron microtomography data to be segmented;

S2: Using a cellular protein semantic segmentation model to perform protein semantic segmentation on each of the plurality of electron microtomography data to be analyzed, respectively, to obtain the plurality of electron microtomography tomography data to be analyzed The set of protein semantic segmentation results corresponding to the data, wherein the alternate training of confrontation and semi-supervised learning is performed based on the generative network and the discriminant network, and the generative network obtained by the alternate training of confrontation and semi-supervised learning is used as the cell protein semantic segmentation model ;

S3: Perform data splicing according to the set of protein semantic segmentation results corresponding to the plurality of electron microscopic tomographic data to be analyzed, to obtain target protein semantic segmentation results corresponding to the plurality of to-be-analyzed electron microscopic tomographic data.

In this embodiment, a plurality of electron microtomography data to be analyzed obtained by segmenting the same cell electron microtomography data to be segmented, the cell protein semantic segmentation model is used to separate the multiple electron microtomography tomography data to be analyzed respectively. Perform protein semantic segmentation for each electron microtomography data to be analyzed in the data, and obtain a set of protein semantic segmentation results corresponding to multiple electron microtomography data to be analyzed. The semantic segmentation result set is used for data splicing, and the target protein semantic segmentation results corresponding to the multiple electron microtomography data to be analyzed are obtained, which reduces the amount of data for protein semantic segmentation using the cell protein semantic segmentation model each time, and reduces the training of the model. The requirements of GPU hardware conditions; through the alternating training of adversarial and semi-supervised learning based on the generation network and the discriminant network, the generation network obtained by the alternating training of the adversarial and semi-supervised learning is used as a cell protein semantic segmentation model, and the data is effectively improved based on the adversarial training. The performance of the training with less amount of training enhances the generalization effect of the model, and training with semi-supervised learning enhances the performance of the model with unlabeled data.

For S1, a plurality of electron microtomography data to be analyzed obtained by dividing the same cell electron microtomography data to be segmented can be obtained, and multiple electron microtomography data to be analyzed can also be obtained from The database obtains multiple electron microtomography data to be analyzed obtained by dividing the same cell electron microtomography data to be segmented, or it can be sent by a third-party application system according to the same cell electron microtomography data to be segmented. A plurality of electron microtomography data to be analyzed are obtained by slicing the microtomography data.

The same piece of cell electron microtomography data to be segmented is the electron microtomography data extracted from the tissue structure of the cells. The spatial distribution of 12 proteins was included in the cell electron microtomography data.

Wherein, the method of sliding window is used to segment the same cell electron microtomography data to be segmented, and the data obtained from one segment is used as one electron microtomography data to be analyzed. For example, the cell electron microtomography data is 200*512*512, and the sliding window method is used to divide the data into small volume data of 50*64*64, that is to say, the size of the electron microtomography data to be analyzed is 50*64 *64, there is no specific limitation in this example.

For S2, each of the plurality of electron microtomography data to be analyzed is input into the cellular protein semantic segmentation model to perform protein semantic segmentation, and the cellular protein semantic segmentation model is for each to be analyzed. The electron micro tomography data outputs a protein semantic segmentation result, and all protein semantic segmentation results are used as a set of protein semantic segmentation results corresponding to the plurality of electron micro tomographic data to be analyzed.

Wherein, the alternating training of confrontation and semi-supervised learning based on the generation network and the discriminant network, that is, the generation network and the discriminant network are cycled through one confrontation training and one semi-supervised learning until the convergence conditions are met, so that the confrontation training is based on Effectively improve the performance of training with a small amount of data, enhance the generalization effect of the model, use semi-supervised learning training, and use unlabeled data to enhance the performance of the model.

The generating network may select a network that can perform semantic segmentation from the prior art.

The discriminant network can be selected from the prior art networks that can be used for adversarial training.

It can be understood that the protein semantic segmentation result is the protein classification result of each voxel point in the electron microtomography data to be analyzed. For example, the protein classification result is any one of 12 kinds of proteins, which is not specifically limited in this example.

For S3, use the position data carried by the plurality of electron microscopic tomographic data to be analyzed to perform data analysis on all the protein semantic segmentation results in the protein semantic segmentation result set corresponding to the plurality of to-be-analyzed electron microscopic tomographic data splicing to obtain the target protein semantic segmentation result corresponding to the plurality of electron microscopic tomographic data to be analyzed. That is to say, the target protein semantic segmentation result is the protein semantic segmentation result of the cell electron microtomography data to be segmented.

In one embodiment, the cellular protein semantic segmentation model is used to perform protein semantic segmentation on each of the plurality of electron microscopic tomographic data to be analyzed, respectively, to obtain the plurality of to-be-analyzed electron microscopic tomographic data. The steps before the collection of the corresponding protein semantic segmentation results of the electron microtomography data include:

S021: Obtain a labeled training sample set and an unlabeled training sample set obtained by obtaining the same training sample of the cell electron microtomography data to be divided;

S022: obtain a marked training sample as a target marked training sample from the set of marked training samples, and obtain an unmarked training sample as a target unmarked training sample from the set of unmarked training samples;

S023: Perform adversarial training on the generation network and the discrimination network according to the target marked training samples, wherein the generation network adopts a segmentation network U-net++, and the discrimination network adopts a full convolution discriminator;

S024: Perform semi-supervised training on the generation network after adversarial training according to the target unlabeled training samples and the discriminant network after adversarial training;

S025: Repeat the process of obtaining a labeled training sample from the labeled training sample set as a target labeled training sample, and obtaining an unlabeled training sample from the unlabeled training sample set as a target unlabeled training sample step, until the alternating training of adversarial and semi-supervised learning reaches the convergence condition, and the generation network whose alternating training of confrontation and semi-supervised learning both reaches the convergence condition is determined as the cell protein semantic segmentation model.

This embodiment realizes the alternating training of confrontation and semi-supervised learning based on the generation network and the discriminant network. The generation network obtained by the alternating training of confrontation and semi-supervised learning is used as the cell protein semantic segmentation model, which effectively improves the performance based on the confrontation training. The performance of training with a small amount of data enhances the generalization effect of the model. Semi-supervised learning training is used to enhance the performance of the model by using unlabeled data. Through the alternating training of confrontation and semi-supervised learning, the performance of each confrontation training is realized. The results are augmented once to further improve the accuracy of the model.

For S021, the labeled training sample set and the unlabeled training sample set obtained from the same training sample of the cell electron microtomography data to be segmented input by the user can be obtained, or the same copy of the cell electron micrograph to be segmented can be obtained from the database The set of labeled training samples and the set of unlabeled training samples obtained from the training samples of microscopic tomography data can also be the set of labeled training samples obtained from the same training sample of cell electron microtomography data to be segmented sent by a third-party application system and a set of unlabeled training samples.

Optionally, the number of labeled training samples in the labeled training sample set is greater than the number of unlabeled training samples in the unlabeled training sample set.

The labeled training samples include: electron microscopy tomography sample data and protein calibration data, where the protein calibration data is the protein classification result of each voxel point in the electron microscopy tomography sample data. Each of the labeled training samples includes one electron microtomography sample data and one protein calibration data.

The unlabeled training samples include: electron microtomography sample data.

For S022, sequentially obtain a labeled training sample from the labeled training sample set as a target labeled training sample, and sequentially obtain an unlabeled training sample from the unlabeled training sample set as a target unlabeled training sample.

For S023, the method for adversarial training of the generation network and the discriminant network according to the electron microtomography sample data and protein calibration data of the target marked training sample can be selected from the prior art, thereby effectively increasing the amount of data Less training performance, enhanced model generalization.

Optionally, the generation network adopts a segmentation network U-net++, and the segmentation network U-net++ sequentially includes: 4 convolution layers, 4 deconvolution layers, and 12 1*1 convolution kernels. 4 convolutional layers are used for feature extraction, 4 deconvolutional layers are used for deconvolution reduction, and 12 1*1 convolution kernels are used to obtain the classification probabilities of 12 categories (that is, the 12 proteins). classification probability).

For S024, the method for semi-supervised training of the generation network after adversarial training according to the target unlabeled training samples and the discriminant network after adversarial training can be selected from the prior art, so as to use semi-supervised learning and training, Leveraging unlabeled data enhances the performance of the model.

For S025, steps S022 to S025 are repeatedly executed until the alternate training of adversarial and semi-supervised learning reaches the convergence condition.

Wherein, the condition that the alternating training of adversarial and semi-supervised learning both reach the convergence condition includes: the loss value of adversarial training and the loss value of semi-supervised training both reach the first convergence condition, or, the number of training times of alternating training of adversarial and semi-supervised learning The second convergence condition is reached.

Wherein, the loss value of the adversarial training and the loss value of the semi-supervised training both reach the first convergence condition, which refers to the loss value corresponding to the generating network in the adversarial training and the loss corresponding to the discriminating network in the adversarial training. value and the loss value of semi-supervised training all reach the first convergence condition.

The first convergence condition means that the size of the loss value calculated twice adjacent to the same network satisfies the Lipschitz condition (the Lipschitz continuity condition).

The number of training times of the alternating training of adversarial and semi-supervised learning reaches the second convergence condition, which refers to the number of times that the generative network and the discriminative network are used for the alternating training of adversarial and semi-supervised learning, that is, the alternating training of adversarial and semi-supervised learning. Once, the number of training sessions for alternating training with adversarial and semi-supervised learning is increased by 1.

In one embodiment, the above-mentioned steps of obtaining the labeled training sample set and the unlabeled training sample set obtained from the same training sample of the cell electron microtomography data to be segmented include:

S0211: Obtain the training sample of the cell electron microtomography data;

S0212: Use a sliding window method to segment the cell electron microtomography data training sample to obtain a plurality of electron microtomography sample data;

S0213: Use a preset ratio to divide the plurality of electron microscopy tomography sample data to obtain a to-be-labeled training sample set and an unlabeled training sample set, wherein the electron microscopy tomography sample data in the to-be-labeled training sample set The quantity is greater than the quantity of the electron microscopy tomography sample data in the unlabeled training sample set;

S0214: Perform protein semantic segmentation and calibration on each of the electron microscopy tomography sample data in the to-be-labeled training sample set, respectively, to obtain the labeled training sample set.

This embodiment realizes that a labeled training sample set and an unlabeled training sample set are determined from one of the cell electron microscopy tomography data training samples, so that model training can be performed even with a small amount of data.

For S0211, the training sample of the cell electron microscope tomography data input by the user can be obtained, the training sample of the cell electron microscope tomography data can also be obtained from the database, or the cell electron microscope tomography data sent by a third-party application system can be obtained. Microtomography data training samples.

The training sample of cell electron micro tomography data is the electron micro tomography data extracted from the tissue structure of cells. The spatial distribution of 12 proteins was included in the training sample of TEM data.

For S0212, a sliding window method is used to segment a training sample of the cell electron micro tomography data, and the data obtained from one segment is used as an electron micro tomography sample data.

For S0213, use a preset ratio to divide the plurality of electron micro tomography sample data, that is, use a part of the electron micro tomography sample data in the plurality of electron micro tomography sample data as the training sample to be marked Set, and use another part of the electron microtomography sample data in the plurality of electron microtomography sample data as an unlabeled training sample set. It can be understood that the same electron microtomography sample data can only be divided into one set (that is, one set of the labeled training sample set and the unlabeled training sample set).

Optionally, the preset ratio is set to 85:15, that is, 85% of the electron microtomography sample data in the multiple electron microtomography sample data is used as the set of training samples to be marked, and the multiple electron microtomography tomography sample data The remaining 15% of the electron microtomography sample data in the sample data is used as a set of unlabeled training samples.

For S0214, perform protein semantic segmentation and calibration on each of the electron microtomography sample data in the to-be-labeled training sample set respectively, and use the result of protein semantic segmentation and calibration on one of the electron microtomography sample data as a protein Calibration data; take all the electron microscopy tomography sample data and the respective corresponding protein calibration data in the to-be-labeled training sample set as the labeled training sample set.

In one embodiment, the above-mentioned step of adversarial training of the generation network and the discriminant network according to the target marked training samples includes:

S0231: Input the electron microtomography sample data of the target labeled training sample into the generation network to perform protein semantic segmentation, and obtain a first training result;

S0232: Input the protein calibration data of the target marked training sample and the first training result into the discrimination network for discrimination, and obtain a first confidence result;

S0233: Use the protein calibration data of the target labeled training sample, the first training result, and the first confidence result to perform adversarial training on the generation network and the discriminant network.

This embodiment effectively improves the performance of training with a small amount of data based on adversarial training, and enhances the effect of model generalization.

For S0231, input the electron microscopic tomography sample data of the target labeled training sample into the generation network to perform protein semantic segmentation, and use the result of the protein semantic segmentation as the first training result. That is, the first training result is the protein classification result of each voxel point in the electron microtomography sample data of the target labeled training sample.

For S0232, input the protein calibration data of the target labeled training sample and the first training result into the discrimination network for discrimination, and obtain a first confidence result corresponding to the first training result. That is, each confidence result in the first confidence result is a confidence level for each protein classification result in the first training result.

For S0233, use the protein calibration data of the target marked training sample and the first training result to train the generation network, and update the parameters of the generation network once during training; use the first confidence result The discriminant network is trained, and the parameters of the discriminant network are updated once during training.

In one embodiment, the above-mentioned step of adversarial training of the generation network and the discriminant network using the protein calibration data of the target labeled training samples, the first training result and the first confidence result ,include:

S02331: Input the protein calibration data of the target labeled training sample and the first training result into a first loss function for calculation, obtain a first loss value of the generation network, and update according to the first loss value the parameters of the generating network;

S02332: Input the first confidence result into a second loss function for calculation, obtain a second loss value of the discriminant network, and update the parameters of the discriminant network according to the second loss value;

Wherein, the calculation formula L _ce of the first loss function is:

The calculation formula L _adv of the second loss function is:

_Xn is the electron microtomography sample data of the target labeled training sample, h is the width of the size of the electron microtomography sample data of the target labeled training sample, w is the target height of the dimension of the electron microtomography sample data of the labeled training sample, c is the channel number of the dimension of the electron microtomography sample data of the target labeled training sample, S(X _n ) ^{(h,w , c)} is the first training result, log() is a logarithmic function, Y _n ^(h,w,c) is the protein calibration data of the target labeled training sample, C is the number of cell protein species ; D(S(X _n )) ^(h,w) is the first confidence result.

This example uses semi-supervised learning training to enhance the performance of the model with unlabeled data.

For S02331, the method for updating the parameters of the generation network according to the first loss value can be selected from the prior art, and details are not described here.

For S02332, the method for updating the parameters of the discriminating network according to the second loss value can be selected from the prior art, and details are not described here.

h×w×c is the size of the electron microtomography sample data of the target marked training sample, that is, the length h, the width w, and the number of channels c of the electron microtomography sample data.

In one embodiment, the above-mentioned step of performing semi-supervised training on the generation network after adversarial training according to the target unlabeled training samples and the discriminant network after adversarial training includes:

S0241: Input the electron microtomography sample data of the target unlabeled training sample into the generation network to perform protein semantic segmentation to obtain a second training result;

S0242: Input the second training result into the discrimination network for discrimination, and obtain a second confidence result;

S0243: Determine a reliable result corresponding to the target unlabeled training sample according to the second confidence result;

S0244: Perform semi-supervised training on the generation network by using the reliable results and the second training results corresponding to the target unlabeled training samples.

This embodiment thus enhances the performance of the model with unlabeled data using semi-supervised learning training.

For S0241, the electron microtomography sample data of the target unlabeled training sample is input into the generation network to perform protein semantic segmentation, and the result of the protein semantic segmentation is used as the second training result. That is, the second training result is the protein classification result of each voxel point in the electron microtomography sample data of the target unlabeled training sample.

For S0242, the second training result is input into the discrimination network for discrimination, and a second confidence result is obtained. That is, each confidence result in the second confidence result is a confidence level for each protein classification result in the second training result.

For S0243, binarize the second confidence result, and use the data satisfying the confidence threshold in the binarized second confidence result as the reliable result corresponding to the target unlabeled training sample.

For S0244, semi-supervised training is performed on the generation network by using the reliable result corresponding to the target unlabeled training sample and the second training result, and the parameters of the generation network are updated once during training.

In one embodiment, the above-mentioned step of semi-supervised training of the generating network by using the reliable result corresponding to the target unlabeled training sample and the second training result includes:

S02441: Input the reliable result and the second training result corresponding to the target unlabeled training sample into a third loss function for calculation, to obtain a third loss value of the generation network, according to the third loss value updating the parameters of the generating network;

Wherein, the calculation formula L _semi of the third loss function is:

X _n is the SEM sample data of the target unlabeled training sample, h×w×c is the size of the SEM sample data of the target unlabeled training sample, S(X _n ) ^(h,w,c) is the second training result, D(S(X _n )) ^(h,w) is the reliable result corresponding to the target unlabeled training sample, log() is the logarithm function, T _semi is the threshold that controls the sensitivity of the self-learning process,

is the target value of self-learning, I() is the indicator function, the target value of self-learning

And the indicator function I() is constant.

This embodiment thus enhances the performance of the model with unlabeled data using semi-supervised learning training.

For S02441, the method for updating the parameters of the generating network according to the third loss value can be selected from the prior art, and details are not described here.

The threshold for controlling the sensitivity of the self-learning process can be obtained from a database, a third-party application system, or written into a program file implementing the present application.

Referring to FIG. 2 , the present application also proposes a semantic segmentation device for electron microtomography data, the device includes:

The data acquisition module 100 is used for acquiring a plurality of electron microscopy tomography data to be analyzed obtained by dividing according to the same cell electron microscopy tomography data to be segmented;

The protein semantic segmentation module 200 is configured to perform protein semantic segmentation on each of the plurality of electron microscopic tomographic data to be analyzed in the plurality of electron microscopic tomographic data to be analyzed by using a cellular protein semantic segmentation model, and obtain the The set of protein semantic segmentation results corresponding to the analyzed electron microscopic tomography data, wherein the alternate training of confrontation and semi-supervised learning is performed based on the generation network and the discriminant network, and the generation network obtained by the alternate training of confrontation and semi-supervised learning is used as the described cellular protein semantic segmentation model;

The data splicing module 300 is configured to perform data splicing according to the protein semantic segmentation result set corresponding to the plurality of electron microscopic tomographic data to be analyzed, so as to obtain the target protein corresponding to the plurality of to-be-analyzed electron microscopic tomographic data Semantic segmentation results.

The present application also proposes a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when the processor executes the computer program.

The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps of any one of the methods described above.

In this embodiment, a plurality of electron microtomography data to be analyzed obtained by segmenting the same cell electron microtomography data to be segmented, the cell protein semantic segmentation model is used to separate the multiple electron microtomography tomography data to be analyzed respectively. Perform protein semantic segmentation for each electron microtomography data to be analyzed in the data, and obtain a set of protein semantic segmentation results corresponding to multiple electron microtomography data to be analyzed. The semantic segmentation result set is used for data splicing, and the target protein semantic segmentation results corresponding to the multiple electron microtomography data to be analyzed are obtained, which reduces the amount of data for protein semantic segmentation using the cell protein semantic segmentation model each time, and reduces the training of the model. The requirements of GPU hardware conditions; through the alternating training of adversarial and semi-supervised learning based on the generation network and the discriminant network, the generation network obtained by the alternating training of the adversarial and semi-supervised learning is used as a cell protein semantic segmentation model, and the data is effectively improved based on the adversarial training. The performance of the training with less amount of training enhances the generalization effect of the model, and training with semi-supervised learning enhances the performance of the model with unlabeled data.

Referring to Fig. 3, an embodiment of the present application further provides a computer device, the computer device may be a server, and its internal structure may be as shown in Fig. 3 . The computer device includes a processor, memory, a network interface, and a database connected by a system bus. Among them, the processor of the computer design is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The nonvolatile storage medium stores an operating system, a computer program, and a database. The memory provides an environment for the execution of the operating system and computer programs in the non-volatile storage medium. The database of the computer equipment is used for storing data such as semantic segmentation methods of electron microscopic tomography data. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program, when executed by the processor, implements a method for semantic segmentation of electron microtomography data. The method for semantic segmentation of electron microscopic tomography data includes: acquiring a plurality of electron microscopic tomographic data to be analyzed obtained by segmenting the same piece of cell electron microscopic tomographic data to be segmented; adopting a cell protein semantic segmentation model Perform protein semantic segmentation on each of the plurality of electron microscopic tomographic data to be analyzed, respectively, to obtain a set of protein semantic segmentation results corresponding to the plurality of to-be-analyzed electron microscopic tomographic data , wherein the alternate training of confrontation and semi-supervised learning is performed based on the generation network and the discriminant network, and the generation network obtained by the alternate training of confrontation and semi-supervised learning is used as the cell protein semantic segmentation model; Perform data splicing on the set of protein semantic segmentation results corresponding to the electron microscopic tomography data to obtain the target protein semantic segmentation results corresponding to the plurality of electron microscopic tomographic data to be analyzed.

In this embodiment, a plurality of electron microtomography data to be analyzed obtained by segmenting the same cell electron microtomography data to be segmented, the cell protein semantic segmentation model is used to separate the multiple electron microtomography tomography data to be analyzed respectively. Perform protein semantic segmentation for each electron microtomography data to be analyzed in the data, and obtain a set of protein semantic segmentation results corresponding to multiple electron microtomography data to be analyzed. The semantic segmentation result set is used for data splicing, and the target protein semantic segmentation results corresponding to the multiple electron microtomography data to be analyzed are obtained, which reduces the amount of data for protein semantic segmentation using the cell protein semantic segmentation model each time, and reduces the training of the model. The requirements of GPU hardware conditions; through the alternating training of adversarial and semi-supervised learning based on the generation network and the discriminant network, the generation network obtained by the alternating training of the adversarial and semi-supervised learning is used as a cell protein semantic segmentation model, and the data is effectively improved based on the adversarial training. The performance of the training with less amount of training enhances the generalization effect of the model, and training with semi-supervised learning enhances the performance of the model with unlabeled data.

An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements a method for semantic segmentation of electron microscopic tomography data, including the steps of: obtaining data according to the same A plurality of electron microscopy tomography data to be analyzed obtained by segmenting the segmented cell electron microscopy tomography data; using a cell protein semantic segmentation model to separately analyze each of the plurality of electron microscopy tomography data to be analyzed Perform protein semantic segmentation on the electron micro tomography data obtained by obtaining a set of protein semantic segmentation results corresponding to the plurality of electron micro tomography data to be analyzed, wherein the alternating training of confrontation and semi-supervised learning is performed based on the generation network and the discriminant network, The generation network obtained by the alternate training of confrontation and semi-supervised learning is used as the cellular protein semantic segmentation model; data splicing is performed according to the protein semantic segmentation result set corresponding to the plurality of electron microscopic tomography data to be analyzed, Obtain the target protein semantic segmentation result corresponding to the plurality of electron microtomography data to be analyzed.

The semantic segmentation method for electron microtomography data performed above is obtained by segmenting a plurality of electron microtomography data to be analyzed according to the same piece of cell electron microtomography data to be segmented, using a cell protein semantic segmentation model to separate the data. Perform protein semantic segmentation on each electron micro tomography data to be analyzed in the plurality of electron micro tomography data to be analyzed, and obtain a set of protein semantic segmentation results corresponding to the plurality of electron micro tomography data to be analyzed. Data splicing is performed on the set of protein semantic segmentation results corresponding to the analyzed electron micro tomography data, and the target protein semantic segmentation results corresponding to multiple electron micro tomography data to be analyzed are obtained, which reduces the use of the cell protein semantic segmentation model each time for protein semantic segmentation. The amount of divided data reduces the requirements for GPU hardware conditions for model training; through alternate training of adversarial and semi-supervised learning based on the generative network and discriminant network, the generative network obtained by alternate training of adversarial and semi-supervised learning is used as a cell protein Semantic segmentation model, based on adversarial training, effectively improves the performance of training with less data, enhances the generalization effect of the model, uses semi-supervised learning training, and uses unlabeled data to enhance the performance of the model.

The computer storage medium can be non-volatile or volatile.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage In the medium, when the computer program is executed, it may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database or other medium provided in this application and used in the embodiments may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, device, article or method comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, apparatus, article or method. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, apparatus, article, or method that includes the element.

The above are only the preferred embodiments of the present application, and are not intended to limit the scope of the patent of the present application. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present application, or directly or indirectly applied to other related The technical field is similarly included in the scope of patent protection of this application.

Claims (20)

1. A method for semantic segmentation of electron microscopic tomography data, wherein the method comprises:

   Acquiring a plurality of electron microtomography data to be analyzed obtained by segmenting the same cell electron microtomography data to be segmented;

   The cell protein semantic segmentation model is used to perform protein semantic segmentation on each of the electron microtomography data to be analyzed in the plurality of electron microtomography data to be analyzed, so as to obtain the corresponding The set of protein semantic segmentation results, wherein, based on the generation network and the discriminant network, the alternating training of confrontation and semi-supervised learning is performed, and the generation network obtained by the alternating training of confrontation and semi-supervised learning is used as the cell protein semantic segmentation model;

   Perform data splicing according to the set of protein semantic segmentation results corresponding to the plurality of electron microscopic tomographic data to be analyzed, to obtain target protein semantic segmentation results corresponding to the plurality of to-be-analyzed electron microscopic tomographic data.
2. The method for semantic segmentation of electron microscopic tomography data according to claim 1, wherein the use of a cell protein semantic segmentation model is used to separately analyze each electron micrograph of the plurality of electron microscopic tomographic data to be analyzed. Before the steps of performing protein semantic segmentation on the tomographic data, and obtaining a set of protein semantic segmentation results corresponding to the plurality of electron microscopic tomographic data to be analyzed, the steps include:

   Obtaining a set of labeled training samples and a set of unlabeled training samples obtained from the same training sample of cell electron microtomography data to be segmented;

   Obtain a labeled training sample from the labeled training sample set as a target labeled training sample, and obtain an unlabeled training sample from the unlabeled training sample set as a target unlabeled training sample;

   Adversarial training is performed on the generation network and the discriminant network according to the target labeled training samples, wherein the generation network adopts a segmentation network U-net++, and the discriminant network adopts a fully convolutional discriminator;

   Perform semi-supervised training on the generation network after adversarial training according to the target unlabeled training samples and the discriminant network after adversarial training;

   Repeating the steps of obtaining a labeled training sample from the labeled training sample set as a target labeled training sample, and obtaining an unlabeled training sample from the unlabeled training sample set as a target unlabeled training sample, Until the alternate training of adversarial and semi-supervised learning reaches the convergence condition, the generation network whose alternating training of confrontation and semi-supervised learning both reaches the convergence condition is determined as the cell protein semantic segmentation model.
3. The method for semantic segmentation of electron microscopy tomography data according to claim 2, wherein the set of labeled training samples and the set of unlabeled training samples obtained by acquiring the same training sample of cell electron microscopy tomography data to be segmented steps, including:

   obtaining the training sample of the cell electron microtomography data;

   The cell electron micro tomography data training sample is segmented by the sliding window method to obtain a plurality of electron micro tomography sample data;

   The plurality of electron microscopy tomography sample data are divided by a preset ratio to obtain a to-be-labeled training sample set and an unlabeled training sample set, wherein the number of electron microscopy tomography sample data in the to-be-labeled training sample set greater than the quantity of the electron microtomography sample data in the unlabeled training sample set;

   Perform protein semantic segmentation and calibration on each of the electron microscopy tomography sample data in the to-be-labeled training sample set, respectively, to obtain the labeled training sample set.
4. The method for semantic segmentation of electron microscopic tomography data according to claim 2, wherein the step of adversarial training of the generation network and the discriminant network according to the target marked training samples comprises:

   Inputting the electron microscopy tomography sample data of the target marked training sample into the generating network to perform protein semantic segmentation to obtain a first training result;

   Inputting the protein calibration data of the target labeled training sample and the first training result into the discriminant network for discrimination, and obtaining a first confidence result;

   The generator network and the discriminant network are adversarially trained using the protein calibration data of the target labeled training samples, the first training result, and the first confidence result.
5. The method for semantic segmentation of electron microscopy tomography data according to claim 4, wherein said protein calibration data using said target labeled training sample, said first training result and said first confidence result are paired The steps of adversarial training of the generating network and the discriminating network include:

   Inputting the protein calibration data and the first training result of the target labeled training sample into a first loss function for calculation to obtain a first loss value of the generation network, and updating the first loss value according to the first loss value. Generate the parameters of the network;

   Inputting the first confidence result into a second loss function for calculation to obtain a second loss value of the discriminant network, and updating the parameters of the discriminant network according to the second loss value;

   Wherein, the calculation formula L _ce of the first loss function is:

   

   The calculation formula L _adv of the second loss function is:

   

   _Xn is the electron microtomography sample data of the target labeled training sample, h is the width of the size of the electron microtomography sample data of the target labeled training sample, w is the target labeled training sample The height of the size of the SEM sample data of the training sample, c is the channel number of the size of the SEM sample data of the target labeled training sample, S(X _n ) ^{(h,w, c)} is the first training result, log() is a logarithmic function, Y _n ^{(h, w, c)} is the protein calibration data of the target labeled training sample, and C is the number of types of cellular proteins; D(S(X _n )) ^(h,w) is the first confidence result.
6. The method for semantic segmentation of electron microtomography data according to claim 2, wherein the generating network after adversarial training is semi-supervised according to the target unlabeled training samples and the discriminant network after adversarial training The training steps include:

   Inputting the electron microtomography sample data of the target unlabeled training sample into the generating network to perform protein semantic segmentation to obtain a second training result;

   Inputting the second training result into the discriminating network for discrimination to obtain a second confidence result;

   According to the second confidence result, determine the reliable result corresponding to the target unlabeled training sample;

   Semi-supervised training is performed on the generating network using the trusted results and the second training results corresponding to the target unlabeled training samples.
7. The method for semantic segmentation of electron microscopic tomography data according to claim 6, wherein the generating network is semi-processed by using the reliable results corresponding to the target unlabeled training samples and the second training results. The steps of supervised training include:

   Input the reliable result and the second training result corresponding to the target unlabeled training sample into a third loss function for calculation, to obtain the third loss value of the generation network, and update the third loss value according to the third loss value. Describe the parameters of the generated network;

   Wherein, the calculation formula L _semi of the third loss function is:

   

   X _n is the SEM sample data of the target unlabeled training sample, h×w×c is the size of the SEM sample data of the target unlabeled training sample, S(X _n ) ^(h,w,c) is the second training result, D(S(X _n )) ^(h,w) is the reliable result corresponding to the target unlabeled training sample, log() is the logarithm function, T _semi is the threshold that controls the sensitivity of the self-learning process,

   

   is the target value of self-learning, I() is the indicator function, the target value of self-learning

   

   And the indicator function I() is constant.
8. A device for semantic segmentation of electron microtomography data, wherein the device comprises:

   The data acquisition module is used to acquire a plurality of electron microtomography data to be analyzed obtained by segmenting the same cell electron microtomography data to be segmented;

   A protein semantic segmentation module is used to perform protein semantic segmentation on each of the plurality of electron microscopic tomography data to be analyzed by using a cellular protein semantic segmentation model, and obtain the plurality of to-be-analyzed electron microtomography data. The set of protein semantic segmentation results corresponding to the electron microscopic tomography data of the Cellular protein semantic segmentation model;

   A data splicing module, configured to perform data splicing according to the protein semantic segmentation result set corresponding to the plurality of electron microscopic tomographic data to be analyzed, to obtain the target protein semantics corresponding to the plurality of to-be-analyzed electron microscopic tomographic data Split result.
9. A computer device includes a memory and a processor, wherein the memory stores a computer program, wherein the processor implements the following method steps when executing the computer program:

   Acquiring a plurality of electron microtomography data to be analyzed obtained by segmenting the same cell electron microtomography data to be segmented;

   The cell protein semantic segmentation model is used to perform protein semantic segmentation on each of the electron microtomography data to be analyzed in the plurality of electron microtomography data to be analyzed, so as to obtain the corresponding The set of protein semantic segmentation results, wherein, based on the generation network and the discriminant network, the alternating training of confrontation and semi-supervised learning is performed, and the generation network obtained by the alternating training of confrontation and semi-supervised learning is used as the cell protein semantic segmentation model;

   Perform data splicing according to the set of protein semantic segmentation results corresponding to the plurality of electron microscopic tomographic data to be analyzed, to obtain target protein semantic segmentation results corresponding to the plurality of to-be-analyzed electron microscopic tomographic data.
10. The computer device according to claim 9, wherein the protein semantic segmentation is performed on each of the plurality of electron microscopic tomographic data to be analyzed by using a cellular protein semantic segmentation model, respectively, Before the step of obtaining the protein semantic segmentation result set corresponding to the plurality of electron microtomography data to be analyzed, the steps include:

    Obtaining a set of labeled training samples and a set of unlabeled training samples obtained from the same training sample of cell electron microtomography data to be segmented;

    Obtain a labeled training sample from the labeled training sample set as a target labeled training sample, and obtain an unlabeled training sample from the unlabeled training sample set as a target unlabeled training sample;

    Adversarial training is performed on the generation network and the discrimination network according to the target marked training samples, wherein the generation network adopts a segmentation network U-net++, and the discrimination network adopts a fully convolutional discriminator;

    Perform semi-supervised training on the generation network after adversarial training according to the target unlabeled training samples and the discriminant network after adversarial training;

    Repeating the steps of obtaining a labeled training sample from the labeled training sample set as a target labeled training sample, and obtaining an unlabeled training sample from the unlabeled training sample set as a target unlabeled training sample, Until the alternate training of adversarial and semi-supervised learning reaches the convergence condition, the generation network whose alternating training of confrontation and semi-supervised learning both reaches the convergence condition is determined as the cell protein semantic segmentation model.
11. The computer device according to claim 10, wherein the step of obtaining the labeled training sample set and the unlabeled training sample set obtained from the same training sample of the cell electron microtomography data to be divided comprises:

    obtaining the training sample of the cell electron microtomography data;

    The cell electron micro tomography data training sample is segmented by the sliding window method to obtain a plurality of electron micro tomography sample data;

    The plurality of electron microscopy tomography sample data are divided by a preset ratio to obtain a to-be-labeled training sample set and an unlabeled training sample set, wherein the number of electron microscopy tomography sample data in the to-be-labeled training sample set greater than the quantity of the electron microtomography sample data in the unlabeled training sample set;

    Perform protein semantic segmentation and calibration on each of the electron microscopy tomography sample data in the to-be-labeled training sample set, respectively, to obtain the labeled training sample set.
12. The computer device according to claim 10, wherein the step of performing adversarial training on the generation network and the discriminant network according to the target labeled training samples comprises:

    Inputting the electron microscopy tomography sample data of the target marked training sample into the generating network to perform protein semantic segmentation to obtain a first training result;

    Inputting the protein calibration data of the target labeled training sample and the first training result into the discriminant network for discrimination, and obtaining a first confidence result;

    The generator network and the discriminant network are adversarially trained using the protein calibration data of the target labeled training samples, the first training result, and the first confidence result.
13. 13. The computer device of claim 12, wherein said generating network and said generating network using said protein calibration data of said target labeled training sample, said first training result and said first confidence result The steps of adversarial training of the discriminative network include:

    Inputting the protein calibration data and the first training result of the target labeled training sample into a first loss function for calculation to obtain a first loss value of the generation network, and updating the first loss value according to the first loss value. Generate the parameters of the network;

    Inputting the first confidence result into a second loss function for calculation to obtain a second loss value of the discriminant network, and updating the parameters of the discriminant network according to the second loss value;

    Wherein, the calculation formula L _ce of the first loss function is:

    

    The calculation formula L _adv of the second loss function is:

    

    _Xn is the electron microtomography sample data of the target labeled training sample, h is the width of the size of the electron microtomography sample data of the target labeled training sample, w is the target labeled training sample The height of the size of the SEM sample data of the training sample, c is the channel number of the size of the SEM sample data of the target labeled training sample, S(X _n ) ^{(h,w, c)} is the first training result, log() is a logarithmic function, Y _n ^{(h, w, c)} is the protein calibration data of the target labeled training sample, and C is the number of types of cellular proteins; D(S(X _n )) ^(h,w) is the first confidence result.
14. The computer device according to claim 10, wherein the step of performing semi-supervised training on the generation network after adversarial training according to the target unlabeled training samples and the discriminant network after adversarial training comprises:

    Inputting the electron microtomography sample data of the target unlabeled training sample into the generating network to perform protein semantic segmentation to obtain a second training result;

    Inputting the second training result into the discriminating network for discrimination to obtain a second confidence result;

    According to the second confidence result, determine the reliable result corresponding to the target unlabeled training sample;

    Semi-supervised training is performed on the generating network using the trusted results and the second training results corresponding to the target unlabeled training samples.
15. A computer-readable storage medium on which a computer program is stored, wherein when the computer program is executed by a processor, the following method steps are implemented:

    Acquiring a plurality of electron microtomography data to be analyzed obtained by segmenting the same cell electron microtomography data to be segmented;

    The cell protein semantic segmentation model is used to perform protein semantic segmentation on each of the electron microtomography data to be analyzed in the plurality of electron microtomography data to be analyzed, so as to obtain the corresponding The set of protein semantic segmentation results, wherein, based on the generation network and the discriminant network, the alternating training of confrontation and semi-supervised learning is performed, and the generation network obtained by the alternating training of confrontation and semi-supervised learning is used as the cell protein semantic segmentation model;

    Perform data splicing according to the set of protein semantic segmentation results corresponding to the plurality of electron microscopic tomographic data to be analyzed, to obtain target protein semantic segmentation results corresponding to the plurality of to-be-analyzed electron microscopic tomographic data.
16. The computer-readable storage medium according to claim 15, wherein the protein analysis is performed on each of the plurality of electron microtomography data to be analyzed by using a cellular protein semantic segmentation model. Semantic segmentation, before the step of obtaining a set of protein semantic segmentation results corresponding to the plurality of electron microscopic tomographic data to be analyzed, includes:

    Obtaining a set of labeled training samples and a set of unlabeled training samples obtained from the same training sample of cell electron microtomography data to be segmented;

    Obtain a labeled training sample from the labeled training sample set as a target labeled training sample, and obtain an unlabeled training sample from the unlabeled training sample set as a target unlabeled training sample;

    Adversarial training is performed on the generation network and the discriminant network according to the target labeled training samples, wherein the generation network adopts a segmentation network U-net++, and the discriminant network adopts a fully convolutional discriminator;

    Perform semi-supervised training on the generation network after adversarial training according to the target unlabeled training samples and the discriminant network after adversarial training;

    Repeating the steps of obtaining a labeled training sample from the labeled training sample set as a target labeled training sample, and obtaining an unlabeled training sample from the unlabeled training sample set as a target unlabeled training sample, Until the alternate training of adversarial and semi-supervised learning reaches the convergence condition, the generation network whose alternating training of confrontation and semi-supervised learning both reaches the convergence condition is determined as the cell protein semantic segmentation model.
17. The computer-readable storage medium according to claim 16, wherein the step of obtaining the labeled training sample set and the unlabeled training sample set obtained from the same training sample of the cell electron microtomography data to be segmented comprises:

    obtaining the training sample of the cell electron microtomography data;

    The cell electron micro tomography data training sample is segmented by the sliding window method to obtain a plurality of electron micro tomography sample data;

    The plurality of electron microscopy tomography sample data are divided by a preset ratio to obtain a to-be-labeled training sample set and an unlabeled training sample set, wherein the number of electron microscopy tomography sample data in the to-be-labeled training sample set greater than the quantity of the electron microtomography sample data in the unlabeled training sample set;

    Perform protein semantic segmentation and calibration on each of the electron microscopy tomography sample data in the to-be-labeled training sample set, respectively, to obtain the labeled training sample set.
18. The computer-readable storage medium of claim 16, wherein the step of adversarially training the generator network and the discriminant network according to the target labeled training samples comprises:

    Inputting the electron microscopy tomography sample data of the target marked training sample into the generating network to perform protein semantic segmentation to obtain a first training result;

    Inputting the protein calibration data of the target labeled training sample and the first training result into the discriminant network for discrimination, and obtaining a first confidence result;

    The generator network and the discriminant network are adversarially trained using the protein calibration data of the target labeled training samples, the first training result, and the first confidence result.
19. 19. The computer-readable storage medium of claim 18, wherein the generative network is generated using the protein calibration data of the target labeled training samples, the first training result, and the first confidence result. The steps of adversarial training with the discriminant network include:

    Inputting the protein calibration data and the first training result of the target labeled training sample into a first loss function for calculation to obtain a first loss value of the generation network, and updating the first loss value according to the first loss value. Generate the parameters of the network;

    Inputting the first confidence result into a second loss function for calculation to obtain a second loss value of the discriminant network, and updating the parameters of the discriminant network according to the second loss value;

    Wherein, the calculation formula L _ce of the first loss function is:

    

    The calculation formula L _adv of the second loss function is:

    

    _Xn is the electron microtomography sample data of the target labeled training sample, h is the width of the size of the electron microtomography sample data of the target labeled training sample, w is the target labeled training sample The height of the size of the SEM sample data of the training sample, c is the channel number of the size of the SEM sample data of the target labeled training sample, S(X _n ) ^{(h,w, c)} is the first training result, log() is a logarithmic function, Y _n ^{(h, w, c)} is the protein calibration data of the target labeled training sample, and C is the number of types of cellular proteins; D(S(X _n )) ^(h,w) is the first confidence result.
20. The computer-readable storage medium according to claim 16, wherein the step of performing semi-supervised training on the generation network after adversarial training according to the target unlabeled training samples and the discriminant network after adversarial training, include:

    Inputting the electron microtomography sample data of the target unlabeled training sample into the generating network to perform protein semantic segmentation to obtain a second training result;

    Inputting the second training result into the discriminating network for discrimination to obtain a second confidence result;

    According to the second confidence result, determine the reliable result corresponding to the target unlabeled training sample;

    Semi-supervised training is performed on the generating network using the trusted results and the second training results corresponding to the target unlabeled training samples.

Images