CN111860618B - Bidirectional GAN model for pathological data conversion and construction and application methods thereof - Google Patents
Bidirectional GAN model for pathological data conversion and construction and application methods thereof Download PDFInfo
- Publication number
- CN111860618B CN111860618B CN202010627094.0A CN202010627094A CN111860618B CN 111860618 B CN111860618 B CN 111860618B CN 202010627094 A CN202010627094 A CN 202010627094A CN 111860618 B CN111860618 B CN 111860618B
- Authority
- CN
- China
- Prior art keywords
- data
- generator
- reverse
- discriminator
- pathological
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013256 Gubra-Amylin NASH model Methods 0.000 title claims abstract description 76
- 230000001575 pathological effect Effects 0.000 title claims abstract description 51
- 230000002457 bidirectional effect Effects 0.000 title claims abstract description 49
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 40
- 238000000034 method Methods 0.000 title claims description 50
- 238000010276 construction Methods 0.000 title description 6
- 238000013145 classification model Methods 0.000 claims abstract description 12
- 231100000915 pathological change Toxicity 0.000 claims abstract description 7
- 230000036285 pathological change Effects 0.000 claims abstract description 7
- 238000013501 data transformation Methods 0.000 claims abstract description 5
- 238000012549 training Methods 0.000 claims description 58
- 210000004072 lung Anatomy 0.000 claims description 28
- 206010035664 Pneumonia Diseases 0.000 claims description 20
- 238000012216 screening Methods 0.000 claims description 9
- 230000007170 pathology Effects 0.000 claims description 6
- 230000011218 segmentation Effects 0.000 claims description 5
- 208000019693 Lung disease Diseases 0.000 claims description 4
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims description 4
- 238000007781 pre-processing Methods 0.000 claims description 4
- 230000000877 morphologic effect Effects 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 2
- 230000009466 transformation Effects 0.000 claims 1
- 230000003902 lesion Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 230000015572 biosynthetic process Effects 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000003786 synthesis reaction Methods 0.000 description 3
- 201000010099 disease Diseases 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000012938 design process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000035876 healing Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
- G06V2201/031—Recognition of patterns in medical or anatomical images of internal organs
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The present invention provides a bi-directional GAN model for pathological data transformation, comprising: a forward generator and a reverse generator having the same structure but different parameter sets; the system comprises a reverse semantic discriminator and a forward semantic discriminator which have the same structure, wherein a pixel weighting module is added on the basis of a classification model, the input end of the reverse semantic discriminator is connected with the output end of a reverse generator, and the input end of the forward semantic discriminator is connected with the output end of the forward generator; the device comprises a reverse morphology discriminator and a forward morphology discriminator which have the same structure, wherein a channel weighting module is added on the basis of a classification model, the input end of the reverse morphology discriminator is connected with the output end of a reverse generator, the input end of the forward morphology discriminator is connected with the output end of the forward generator, and the bidirectional GAN model can realize bidirectional conversion of pathological data, so that medical staff is assisted in predicting and diagnosing pathological changes of doctors and patients.
Description
Technical Field
The invention relates to the technical field of medical image synthesis, in particular to a bidirectional GAN model for pathological data conversion and a construction and application method thereof.
Background
The generation type countermeasure network (GAN), proposed by Goodfellow in 2014, aims to solve how to learn new samples from training samples, so that deep learning has a qualitative breakthrough in the application of data generation field. Taking a training sample as an image as an example, a generated model in the GAN model generates a simulation image based on the input image, a judging model judges the difference between the simulation image and a real image, and the two models are used for countertraining together to finally reach the GAN model meeting the training target requirement.
The GAN model is widely applied in the field of medical image synthesis, for example, CN109493308a provides a medical image synthesis and classification method for generating an countermeasure network based on conditional multi-discrimination, the method uses two networks to train in the countermeasure process, and simultaneously uses multiple discrimination networks with added conditions to comprehensively judge the generated image, so as to generate a synthesized image with more additional information; for example, CN110458786a provides a method for generating a priori GAN model medical image, which searches for similar areas on unlabeled images and labeled images, labels and corrects the unlabeled images to reduce or cancel the participation of later experts in data labeling, and then the above-mentioned GAN model can only perform a task of directly generating data, i.e. the GAN model can only complete unidirectional conversion of data, and has requirements on training data.
In actual clinic, the method is not only a GAN model needing unidirectional data generation, but also a GAN model needing bidirectional data conversion. That is, it is desirable for medical staff to acquire pathological images after lesions as well as to acquire pathological images after healing, which is advantageous for assisting medical staff in predicting and diagnosing diseases. However, the clinical pathological image sample size is limited, and the pathological images of the same patient before and after the pathological changes are difficult to acquire, so that the training difficulty of the current GAN model is increased. For example, taking a sudden burst of a new type of pneumonia as an example, the clinical lack of pathological images before and after lesions of the pneumonia data makes the training result of the current GAN model poor, and it is difficult to meet the requirements of assisting medical staff in diagnosing and predicting the pneumonia condition.
Disclosure of Invention
The invention aims to provide a bidirectional GAN model for pathological data conversion and a construction and application method thereof, wherein the bidirectional GAN model can realize bidirectional conversion of pathological data, namely conversion between negative data and positive data, so as to assist medical staff in predicting and diagnosing pathological conditions of doctors and patients, and particularly, the bidirectional GAN model can be used for bidirectional conversion of pneumonia data.
The invention aims to provide a bidirectional GAN model for pathological data conversion and a construction and application method thereof, wherein the bidirectional GAN model introduces a double-discriminant mechanism to respectively discriminate from the form and the semanteme of data, so that the discrimination capability of a discriminant module on generated data is improved, and the supervision effect on a generator is enhanced in the training process.
The invention aims to provide a bidirectional GAN model for pathological data conversion and a construction and application method thereof, wherein the GAN model can utilize unpaired data to complete training of the model, solves the problems that paired pathological data are difficult to collect and the pathological data sample size is small, and can screen the pathological data during training so as to reduce the calculated amount and the calculated cost.
In order to achieve the above object, the present technical solution provides a bidirectional GAN model for pathological data conversion, including:
a forward generator and a reverse generator having the same structure but different parameter sets;
the system comprises a reverse semantic discriminator and a forward semantic discriminator which have the same structure, wherein a pixel weighting module is added on the basis of a classification model, the input end of the reverse semantic discriminator is connected with the output end of a reverse generator, and the input end of the forward semantic discriminator is connected with the output end of the forward generator;
The device comprises a reverse shape discriminator and a forward shape discriminator which have the same structure, wherein a channel weighting module is added on the basis of a classification model, the input end of the reverse shape discriminator is connected with the output end of a reverse generator, and the input end of the forward shape discriminator is connected with the output end of the forward generator.
The present solution provides a method for applying a bidirectional GAN model for pathological data transformation, wherein the bidirectional GAN model is as described above, and comprises the following steps: and inputting CT image slices corresponding to the preprocessed CT image data into the trained bidirectional GAN model to obtain converted CT image slices, wherein the CT image slices under the diseased condition can be converted if the input CT image slices are not diseased, and the CT image slices under the non-diseased condition can be converted if the input CT image slices are diseased.
Compared with the prior art, the scheme has the following characteristics and beneficial effects: the GAN model of the scheme introduces a double-discriminant mechanism, namely a morphology discriminant and a semantic discriminant, and the double-discriminant mechanism judges a pathological region, so that the training of converting negative data and positive data can be realized without the need of the pathological data from the same patient; the dual-discriminant mechanism can improve the discriminant capability of the discriminant, strengthen the supervision function of the generator in the training process (supervise the generation model of the generator from the morphology and semanteme through the morphology discriminant and the semanteme discriminant respectively), and promote the training of the generator; in addition, the generator of the scheme is configured with two sets of parameter data, so that the generator can provide different conversion capacities by matching with different parameter data, and bidirectional conversion of pathological data is realized.
Drawings
Fig. 1 is a schematic diagram of an embodiment of a bi-directional GAN model for pneumonia data bi-directional conversion according to the invention.
Fig. 2 is a schematic diagram of a framework of a bi-directional GAN model according to the invention.
Fig. 3 is a schematic diagram of the structure of a generator of the bi-directional GAN model according to the invention.
Fig. 4 is a schematic structural diagram of a morphology discriminator of the bidirectional GAN model according to the invention.
Fig. 5 is a schematic structural diagram of a semantic arbiter of a bi-directional GAN model according to the present invention.
In the figure: 11-forward generator, 12-backward generator, 21-backward semantic discriminant, 22-backward morphology discriminant, 31-forward semantic discriminant, 32-forward morphology discriminant.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which are derived by a person skilled in the art based on the embodiments of the invention, fall within the scope of protection of the invention.
It should be appreciated that embodiments of the invention may be implemented or realized by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer readable memory. The methods may be implemented in a computer program using standard programming techniques, including a non-transitory computer readable storage medium configured with a computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner, in accordance with the methods and drawings described in the specific embodiments. Each program may be implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Furthermore, the program can be run on a programmed application specific integrated circuit for this purpose.
Furthermore, the operations of the processes described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes (or variations and/or combinations thereof) described herein may be performed under control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications), by hardware, or combinations thereof, collectively executing on one or more processors. The computer program includes a plurality of instructions executable by one or more processors.
The GAN model is used as an unsupervised learning method and consists of two independent neural networks of a generator and a discriminator, the working principle of the method is that the generator generates false samples in the training process, the discriminator is used for judging whether the samples of the generator are true data or false data, and the discrimination result is fed back to the generator and the discriminator so as to finally achieve the balanced and harmonious state of the generator and the discriminator, and finally the generator with higher quality and the discriminator with strong judging capability are obtained. However, the current GAN model can only realize unidirectional conversion of data, and in order to ensure the discrimination accuracy of the discriminator, front and rear training data with high pairing degree are often required.
For pathological data, especially for pneumonia data, it is often difficult to obtain completely paired pneumonia data, but the pneumonia focus area of the pathological data of different patients is not obvious enough, so that a single discriminator is difficult to supervise the converted data, the training process of a generator is affected, in the actual design process, the inventor also finds that the generated result of the GAN model trained by the single discriminator cannot keep the basic form of the input data, and compared with the original data, the form of the data generated by the generator is greatly changed, so that the use requirement cannot be met.
Aiming at the problems existing when the conventional GAN model is applied to pathological data conversion, the technical scheme provides a GAN model based on a double-discriminant mechanism, wherein a morphology discriminant and a semantic discriminant are introduced into the GAN model aiming at the generation result of a generator, the discrimination capability of the discriminant is improved, the generation model of the generator is supervised from morphology and semanteme respectively through the morphology discriminant and the semantic discriminant in the training process, the training of the generator is facilitated, and the GAN model capable of realizing bidirectional data conversion can be obtained by utilizing unpaired data training.
Specifically, as shown in fig. 2, the structure of the bidirectional GAN model at least includes:
The device comprises a forward generator 11 and a reverse generator 12, wherein the structures of the forward generator 11 and the reverse generator 12 are the same, the forward generator 11 is trained to obtain a first parameter set, and the reverse generator 12 is trained to obtain a second parameter set;
The system comprises a reverse semantic discriminator 21 and a forward semantic discriminator 31, wherein the reverse semantic discriminator 21 and the forward semantic discriminator 31 have the same structure, a pixel weighting module is added on the basis of a classification model, the input end of the reverse semantic discriminator 21 is connected with the output end of a reverse generator 12, and the input end of the forward semantic discriminator 31 is connected with the output end of a forward generator 11;
The reverse morphology discriminator 22 and the forward morphology discriminator 32 have the same structure, a channel weighting module is added on the basis of a classification model, the input end of the reverse morphology discriminator 22 is connected with the output end of the reverse generator 12, and the input end of the forward morphology discriminator 32 is connected with the output end of the forward generator 11.
The output data of the reverse semantic discriminator 21 and the reverse morphology discriminator 22 are fused to generate a reverse discrimination result, and the output data of the forward semantic discriminator 31 and the forward morphology discriminator 32 are fused to generate a forward discrimination result.
It is particularly worth mentioning that the forward generator 11 and the reverse generator 12 may be the same generator. When the generator configures the first parameter set, the generator is defined as a forward generator 11; when the generator configures the second set of parameters, the generator is defined as an inverse generator 12. That is, two sets of parameters are configured corresponding to the same generator, and by matching different parameters, the generator can provide bidirectional conversion capability.
In this scheme, the forward generator 11 and the reverse generator 12 are structured as shown in fig. 3, and in this scheme, the forward generator 11 and the reverse generator 12 employ convolutional neural networks having an encode and decode structure, and are hereinafter collectively referred to as the forward generator 11 and the reverse generator 12. The generator provided by the scheme only simulates pathological changes of pathological areas, and because the scheme is provided with the double discriminators, the generator can adopt unpaired data pairs as training data sets during training, wherein pathological data without pathological changes are defined as negative data, pathological data with pathological changes are defined as positive data, the negative data and the positive data can be not from the same patient, an initialization parameter set is loaded in the initial stage of training of the generator, a first parameter set and a second parameter set are obtained after training, and then a forward generator 11 and a reverse generator 12 are obtained respectively.
It should be noted that the initialization parameter set may be selected in a normal distributed manner, so that the parameter set of the generator may be iteratively updated more quickly as the training process proceeds.
The structure of the reverse semantic discriminant 21 and the forward semantic discriminant 31 is shown in fig. 5, hereinafter the semantic discriminant is collectively referred to as the reverse semantic discriminant 21 and the forward semantic discriminant 31, the semantic discriminant is essentially based on a classification model in a convolutional neural network, a feature map weighting module (shown as a middle position structure in fig. 5) is added in the classification model, the feature map weighting module is a weighting module based on pixel information of an image, and the addition of the feature map weighting module can make the semantic discriminant more sensitive to morphological information.
The structure of the reverse morphology discriminator 22 and the forward morphology discriminator 32 is shown in fig. 4, and the morphology discriminators are hereinafter collectively referred to as the reverse morphology discriminator 22 and the forward morphology discriminator 32, the morphology discriminators are essentially based on classification models in convolutional neural networks, and channel weighting modules (shown in the middle position structure in fig. 4) are added in the classification models, and the channel weighting modules are based on weighting modules of the morphology information of the image, and the addition of the channel weighting modules can make the morphology discriminators more sensitive to the morphology information.
Although the scheme of adopting multiple discriminators is also adopted in the prior art, stacking operation is carried out on the discriminators in the prior art, the structure of the discriminators is not changed, and the scheme is optimized aiming at the structure of the discriminators, so that different discriminators divide work to discriminate tasks, the robust value of an output result is increased, a double-discriminator mechanism utilizes various data information, the supervision capability in a GAN model is greatly enhanced, and the training process of a generator is promoted.
In addition, the scheme weights the loss generated during the conversion of each discriminator and the generator, optimizes the discriminators and the generator, and finally obtains the bidirectional GAN model capable of realizing pathological data conversion.
The training process of the bidirectional GAN model is also a great technical key point of the scheme, the bidirectional GAN model is optimized by utilizing the loss of bidirectional conversion, and the capacities of the generator and the discriminator are improved through a plurality of iterative loops, so that an equilibrium state is finally obtained.
That is, the bidirectional GAN model according to the present embodiment is trained using pathology data for the same pathology, wherein pathology data without occurrence of a lesion is negative data, pathology data with occurrence of a lesion is positive data, the negative data and the positive data are collectively referred to as training data, a forward generator (11) after training sets a first parameter set, a reverse generator (12) sets a second parameter set, and the training data is subjected to bidirectional conversion of the forward generator and the reverse generator during training, so as to obtain generation loss, pseudo data generated by the training data is discriminated by a discriminator, so as to obtain discrimination loss, weighted generation loss and discrimination loss are collectively referred to as training data, and the optimization generator and the discriminator are synchronized.
Specifically, the training process of the bidirectional GAN model is as follows:
Preparing a training data set: preparing a plurality of sets of pathological data for the same pathological condition, wherein pathological data without lesions are positioned as a negative data set, the negative data set is composed of a plurality of negative data A, pathological data with lesions are defined as a positive data set, the positive data set is composed of a plurality of positive data B, and the negative data set and the positive data set are not necessarily from the same patient.
Constructing a bidirectional GAN model: wherein the structural framework of the bidirectional GAN model is as described above;
Training a bidirectional GAN model: negative data a and positive data B are input to the bidirectional GAN model, specifically, negative data a is input to at least the input terminal of the forward generator 11 and the reverse arbiter (the reverse semantic arbiter 21 and the reverse morphology arbiter 22), and positive data B is input to at least the input terminal of the reverse generator 12 and the forward arbiter (the forward semantic arbiter 31 and the forward morphology arbiter 32):
The negative data A is generated by a forward generator 11 to obtain false data a (fake), the false data a (fake) is input into a forward discriminator for discrimination, the forward discrimination loss is obtained by the relationship between the positive data B and the positive discrimination data a (fake), then the second false data a (fake ') is obtained by the reverse generator 12, the forward generation loss is obtained by comparing the positive data a (fake') with the negative data A, the forward discrimination loss and the forward generation loss are weighted to obtain a target to be optimized, and the forward generator and the forward discriminator are optimized.
Meanwhile, the positive data B is generated by the reverse generator 12 to obtain false data B (false), the false data B (false) is input into a reverse discriminator for discrimination, the reverse discriminator discriminates the relation between the B (false) and the negative data A to obtain reverse discrimination loss, then the B (false) is generated by the forward generator 11 to obtain second false data B (false '), the B (false') and the positive data B are compared to obtain reverse generation loss, the weighted reverse discrimination loss and the reverse generation loss are obtained to be optimized, and meanwhile, the reverse generator and the reverse discriminator are optimized.
The data generated by the generator and the real data still have differences, and the generated data has 'marks' brought by the operation of the generator, so that the data can be distinguished by a discriminator, and the successful distinction of the real data and the generated data by the discriminator can promote the generation effect of the generator. The two belong to a competing relationship during the training process. It is not necessary that the positive data B and the negative data a come from the same patient, which is mainly to reduce the difficulty of data collection (it is difficult to collect CT images of the same patient before and after the disease).
Acquisition of forward discrimination loss and reverse discrimination loss:
The judgment result (0 or 1) obtained by the judgment device and the correct result (0 or 1) of whether the data is real data or generated data are directly used for calculating the cross entropy as the judgment loss. Wherein the discriminator discriminates that the generated data is the generated data and the real data, which are directly represented by 0 or 1.
In the step of weighting discrimination loss and generation loss to obtain a target to be optimized in the scheme, the conversion loss weight of a selection discriminator and a generator is 1.5:1, wherein the loss weights of the semantic discriminant and the morphological discriminant in the same group are consistent, and the loss proportion during conversion of the generator is consistent.
The process of optimizing the generator and the arbiter is synchronously performed, in the training process, the generating capacity of the generator is increased, so that errors of the discrimination result of the arbiter are increased, namely loss of the arbiter is increased, the optimizer calculates counter-propagation gradients at the moment, and updates and adjusts parameters of the arbiter, so that the arbiter is optimized, and the quality of the generating result of the generator is measured according to the quality of the discrimination result of the arbiter, and after the capacity of the arbiter is improved, the loss of the generator is increased, so that the optimization of the generator is promoted in turn.
Testing a bidirectional GAN model: the trained bidirectional GAN model has a forward generator 11 corresponding to a first parameter set and a reverse generator 12 corresponding to a second parameter set, and is tested using data.
The bidirectional GAN model provided by the present invention can be applied to pathological data conversion, that is, the present invention additionally provides a pathological data conversion method based on the bidirectional GAN model, comprising the steps of:
And inputting CT image slices corresponding to the preprocessed CT image data into the trained bidirectional GAN model to obtain converted CT image slices, wherein the CT image slices under the diseased condition can be converted if the input CT image slices are not diseased, and the CT image slices under the non-diseased condition can be converted if the input CT image slices are diseased. .
The application description is carried out by converting pneumonia data into specific embodiments, and the conditions of the embodiments are as follows:
Selecting pneumonia data corresponding to a pneumonia disease as pathology data, preparing a training data set, wherein a patient lung CT image with the lung disease is selected as positive data B, a patient lung CT image without the lung disease is selected as negative data, and screening the positive data B and the negative data, wherein the screening process is as follows: converting the CT image into a two-dimensional slice, inputting the two-dimensional slice into a trained lung segmentation model for screening, reserving the slice to contain lung region pixel areas larger than a preset prefabricated slice, marking the lung region of the screened data, manually screening and removing the data with overlarge noise, finally obtaining screened positive data B and negative data A, inputting a training data set into a two-way GAN model for training to obtain an optimized two-way GAN model suitable for pneumonia data conversion.
The application process of the bidirectional GAN model suitable for pneumonia data conversion is as follows:
inputting CT image data to be converted, and preprocessing CT influence data, wherein the preprocessing step comprises the steps of adjusting the data by using a wide window level and carrying out normalization operation;
Segmenting the CT image data by using a lung segmentation model to obtain a lung slice containing a lung region;
The slices containing the lung regions are input into a bidirectional GAN model suitable for pneumonia data conversion to obtain a diseased lung slice if a lung slice of an undepathic patient is input, and a healed lung slice if a lung slice of a diseased patient is input.
In the application scheme, the lung segmentation model is used for screening input data, and only the screened lung sections are predicted, so that the calculated amount and the additional interference are reduced; meanwhile, when the training data sets are prepared, the judgment and classification are carried out slice by slice according to the area of the manually marked pneumonia area as an index, so that interference caused by slices without pneumonia features in positive data is avoided, the difference between the two data sets is further enlarged, the judgment difficulty of a discriminator is further reduced, and the training task is facilitated.
The present application is not limited to the above-mentioned preferred embodiments, and any person who can obtain other various products under the teaching of the present application can make any changes in shape or structure, and all the technical solutions that are the same or similar to the present application fall within the scope of the present application.
Claims (9)
1. A training method of a bidirectional GAN model for pathological data conversion is characterized in that,
Preparing a training data set: preparing a plurality of sets of pathological data aiming at the same pathological condition, wherein the pathological data without pathological changes are positioned as a negative data set, the negative data set consists of a plurality of negative data, the pathological data with pathological changes are defined as a positive data set, and the positive data set consists of a plurality of positive data;
Constructing a bidirectional GAN model: the system comprises a forward generator (11) and a reverse generator (12) which are identical in structure and different in parameter set, and a reverse semantic discriminator (21) and a forward semantic discriminator (31) which are identical in structure, wherein a pixel weighting module is added on the basis of a classification model, the input end of the reverse semantic discriminator (21) is connected with the output end of the reverse generator (12), and the input end of the forward semantic discriminator (31) is connected with the output end of the forward generator (11); the system comprises a reverse shape discriminator (22) and a forward shape discriminator (32) with the same structure, wherein a channel weighting module is added on the basis of a classification model, the input end of the reverse shape discriminator (22) is connected with the output end of a reverse generator (12), and the input end of the forward shape discriminator (32) is connected with the output end of a forward generator (11);
Training a bidirectional GAN model: the negative data is at least input into the input end of the forward generator (11) and the reverse discriminator, the positive data B is at least input into the input end of the reverse generator (12) and the forward discriminator, the negative data obtains forward discrimination loss and forward generation loss in the training process, the weighted forward discrimination loss and forward generation loss obtain the target to be optimized, the positive data obtains reverse discrimination loss and reverse generation loss in the training process, the weighted reverse discrimination loss and reverse generation loss obtain the target to be optimized, and the reverse generator and the reverse discriminator are optimized.
2. Training method for a bi-directional GAN model for pathological data transformation according to claim 1, characterized in that the trained forward generator (11) sets a first set of parameters and the reverse generator (12) sets a second set of parameters.
3. The training method of a bidirectional GAN model for pathological data conversion according to claim 1, wherein the training data is subjected to bidirectional conversion by a forward generator and a reverse generator in the training process to obtain a generation loss, the pseudo data generated by the training data is discriminated by a discriminator to obtain a discrimination loss, the weighted generation loss and the discrimination loss are subjected to optimization, and the generator and the discriminator are optimized synchronously.
4. The training method of a bi-directional GAN model for pathological data transformation according to claim 1, wherein when the discrimination loss and the generation loss are weighted, the discrimination and generator transformation loss weight is 1.5:1, a step of; the loss weights of the semantic discriminators and the morphological discriminators in the same group are consistent.
5. The method of training a bi-directional GAN model for pathological data conversion according to claim 1, wherein negative data and positive data are unpaired data.
6. A method of applying a bi-directional GAN model for pathological data conversion, wherein the bi-directional GAN model is trained by the training method for the bi-directional GAN model for pathological data conversion according to any one of claims 1 to 5, comprising the steps of: and inputting CT image slices corresponding to the preprocessed CT image data into the trained bidirectional GAN model to obtain converted CT image slices, wherein the CT image slices under the diseased condition can be converted if the input CT image slices are not diseased, and the CT image slices under the non-diseased condition can be converted if the input CT image slices are diseased.
7. A method for applying a bi-directional GAN model for pathological data conversion, wherein the bi-directional GAN model is trained by the training method for a bi-directional GAN model for pathological data conversion according to any one of claims 1 to 5, wherein the pathological data is pneumonia data, characterized by comprising the steps of:
inputting CT image data to be converted, and preprocessing CT influence data, wherein the preprocessing step comprises the steps of adjusting the data by using a wide window level and carrying out normalization operation;
Segmenting the CT image data by using a lung segmentation model to obtain a lung slice containing a lung region;
The slices containing the lung regions are input into a bidirectional GAN model suitable for pneumonia data conversion to obtain a diseased lung slice if a lung slice of an undepathic patient is input, and a healed lung slice if a lung slice of a diseased patient is input.
8. The method according to claim 7, wherein the pneumonia data corresponding to the pneumonia disease is selected as the pathology data, a training data set is prepared, wherein a patient lung CT image having the lung disease is selected as the positive data, a patient lung CT image not having the lung disease is selected as the negative data, and the positive data and the negative data are subjected to a screening process.
9. The method of applying a bi-directional GAN model for pathological data transformation according to claim 8, the screening process being as follows: converting the CT image into a two-dimensional slice, inputting the two-dimensional slice into a trained lung segmentation model for screening, reserving the slice with the lung region pixel area larger than a set threshold value in the slice, marking the lung region of the screened data, manually screening and removing the data with overlarge noise to obtain screened positive data and negative data, and inputting a training data set into a bidirectional GAN model for training.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010627094.0A CN111860618B (en) | 2020-07-01 | 2020-07-01 | Bidirectional GAN model for pathological data conversion and construction and application methods thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010627094.0A CN111860618B (en) | 2020-07-01 | 2020-07-01 | Bidirectional GAN model for pathological data conversion and construction and application methods thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111860618A CN111860618A (en) | 2020-10-30 |
| CN111860618B true CN111860618B (en) | 2024-05-14 |
Family
ID=72989723
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010627094.0A Active CN111860618B (en) | 2020-07-01 | 2020-07-01 | Bidirectional GAN model for pathological data conversion and construction and application methods thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111860618B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110335193A (en) * | 2019-06-14 | 2019-10-15 | 大连理工大学 | A kind of unsupervised image conversion method based on the aiming field guiding for generating confrontation network |
| CN110444277A (en) * | 2019-07-19 | 2019-11-12 | 重庆邮电大学 | It is a kind of based on generating multipair anti-multi-modal brain MRI image bi-directional conversion method more |
| CN110503626A (en) * | 2019-07-09 | 2019-11-26 | 上海交通大学 | Based on space-semantic significance constraint CT image modalities alignment schemes |
| KR102095684B1 (en) * | 2019-10-15 | 2020-03-31 | 주식회사 루닛 | Method for domain adaptation based on adversarial learning and apparatus thereof |
| CN111028146A (en) * | 2019-11-06 | 2020-04-17 | 武汉理工大学 | Image super-resolution method for generating countermeasure network based on double discriminators |
-
2020
- 2020-07-01 CN CN202010627094.0A patent/CN111860618B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110335193A (en) * | 2019-06-14 | 2019-10-15 | 大连理工大学 | A kind of unsupervised image conversion method based on the aiming field guiding for generating confrontation network |
| CN110503626A (en) * | 2019-07-09 | 2019-11-26 | 上海交通大学 | Based on space-semantic significance constraint CT image modalities alignment schemes |
| CN110444277A (en) * | 2019-07-19 | 2019-11-12 | 重庆邮电大学 | It is a kind of based on generating multipair anti-multi-modal brain MRI image bi-directional conversion method more |
| KR102095684B1 (en) * | 2019-10-15 | 2020-03-31 | 주식회사 루닛 | Method for domain adaptation based on adversarial learning and apparatus thereof |
| CN111028146A (en) * | 2019-11-06 | 2020-04-17 | 武汉理工大学 | Image super-resolution method for generating countermeasure network based on double discriminators |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111860618A (en) | 2020-10-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110689038B (en) | Training method and device for neural network model and medical image processing system | |
| CN109919928B (en) | Medical image detection method and device and storage medium | |
| WO2023077603A1 (en) | Prediction system, method and apparatus for abnormal brain connectivity, and readable storage medium | |
| CN110503630B (en) | Cerebral hemorrhage classifying, positioning and predicting method based on three-dimensional deep learning model | |
| CN107247971B (en) | Intelligent analysis method and system for ultrasonic thyroid nodule risk index | |
| CN112102266B (en) | Attention mechanism-based cerebral infarction medical image classification model training method | |
| CN113314205B (en) | Efficient medical image labeling and learning system | |
| WO2021186592A1 (en) | Diagnosis assistance device and model generation device | |
| CN105640577A (en) | Method and system automatically detecting local lesion in radiographic image | |
| CN109685768A (en) | Lung neoplasm automatic testing method and system based on lung CT sequence | |
| CN109034360A (en) | A kind of ant colony method constructing brain effective connectivity network from fMRI and DTI data | |
| CN110731773A (en) | abnormal electrocardiogram screening method based on fusion of global and local depth features of electrocardiogram | |
| CN117115045B (en) | Method for improving medical image data quality based on Internet generation type artificial intelligence | |
| CN112308822A (en) | Intervertebral disc CT image detection method based on deep convolutional neural network | |
| CN110930378A (en) | Emphysema image processing method and system based on low data demand | |
| WO2021184195A1 (en) | Medical image reconstruction method, and medical image reconstruction network training method and apparatus | |
| CN111860618B (en) | Bidirectional GAN model for pathological data conversion and construction and application methods thereof | |
| CN116543154A (en) | Medical image segmentation method based on multi-level semantic features | |
| CN116703850A (en) | Medical image segmentation method based on field self-adaption | |
| CN116433970A (en) | Thyroid nodule classification method, thyroid nodule classification system, intelligent terminal and storage medium | |
| CN111881979B (en) | Multi-modal data annotation device and computer-readable storage medium containing program | |
| CN114331996A (en) | Medical image classification method and system based on self-coding decoder | |
| CN114581459A (en) | Improved 3D U-Net model-based segmentation method for image region of interest of preschool child lung | |
| CN114067156A (en) | High-resolution medical image-based stroke classification system and classification method | |
| CN112396597A (en) | Method and device for rapidly screening unknown cause pneumonia images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |