CN112785498A - Pathological image hyper-resolution modeling method based on deep learning - Google Patents
Pathological image hyper-resolution modeling method based on deep learning Download PDFInfo
- Publication number
- CN112785498A CN112785498A CN202011636550.4A CN202011636550A CN112785498A CN 112785498 A CN112785498 A CN 112785498A CN 202011636550 A CN202011636550 A CN 202011636550A CN 112785498 A CN112785498 A CN 112785498A
- Authority
- CN
- China
- Prior art keywords
- resolution
- resolution image
- image
- loss
- generator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000001575 pathological effect Effects 0.000 title claims abstract description 44
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000013135 deep learning Methods 0.000 title claims abstract description 17
- 238000012549 training Methods 0.000 claims abstract description 25
- 230000008447 perception Effects 0.000 claims abstract description 11
- 238000012545 processing Methods 0.000 claims description 17
- 238000009826 distribution Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 12
- 230000003042 antagnostic effect Effects 0.000 claims description 11
- 238000004422 calculation algorithm Methods 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 8
- 238000012935 Averaging Methods 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 7
- 238000005457 optimization Methods 0.000 claims description 5
- 238000013528 artificial neural network Methods 0.000 claims description 4
- 238000005520 cutting process Methods 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 4
- 238000013527 convolutional neural network Methods 0.000 claims description 3
- 210000001525 retina Anatomy 0.000 claims description 2
- 238000000926 separation method Methods 0.000 claims description 2
- 230000007170 pathology Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000003860 storage Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000000137 annealing Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000002207 retinal effect Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a pathological image hyper-resolution modeling method based on deep learning, which comprises the steps of obtaining a low-resolution image from a pathological real high-resolution image, and generating a high-resolution image-low-resolution image pair; establishing a generator and discriminator framework; based on a low-resolution image pre-training generator, enabling an absolute error value of a virtual high-resolution image and a real high-resolution image generated by the generator to reach a minimum error set value; based on the confrontation loss of the virtual high-resolution image and the real high-resolution image, optimizing the discriminator, and based on the perception loss of the virtual high-resolution image and the real high-resolution image, optimizing the generator until the model training is finished to obtain the hyper-resolution model, thereby realizing the purpose of reconstructing the high-resolution image based on the low-resolution image.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a pathological image hyper-resolution modeling method based on deep learning.
Background
At present, the higher the resolution of a pathological image is, the more accurate the analysis of the disease condition is, but in most cases, the resolution of the obtained pathological image is not high, and in such cases, the image needs to be processed, and the resolution of the image is improved to facilitate the subsequent analysis.
In the prior art, the super-resolution image reconstruction has a wide application range including military affairs, medicine, public security, computer vision and the like, and the super-resolution image reconstruction refers to a technology of converting an existing low-resolution image into a high-resolution image by using a signal processing and image processing method and a software algorithm mode. Methods for image super-resolution reconstruction are numerous and basically divided into super-resolution reconstruction based on interpolation, super-resolution reconstruction based on reconstruction and super-resolution reconstruction based on learning.
The super-resolution reconstruction based on learning is to use a large amount of training data to learn a certain corresponding relation between a low-resolution image and a high-resolution image, and then predict the high-resolution image corresponding to the low-resolution image according to the learned mapping relation, thereby realizing the super-resolution reconstruction process of the image.
Therefore, super-resolution reconstruction of images, especially pathological images, is a problem to be solved at present.
Disclosure of Invention
The invention aims to provide a pathological image hyper-resolution modeling method based on deep learning, which comprises the steps of converting a pathological high-resolution image into a low-resolution image, forming a high-resolution image-low-resolution image pair, setting a generator and discriminator framework, pre-training a generator based on the low-resolution image, optimizing the discriminator based on the confrontation loss of a virtual high-resolution image and a real high-resolution image, optimizing the generator based on the perception loss of the virtual high-resolution image and the real high-resolution image, and establishing a hyper-resolution model according to the optimized generator and discriminator to achieve the purpose of reconstructing the high-resolution image based on the low-resolution image.
In a first aspect, the above object of the present invention is achieved by the following technical solutions:
a pathological image hyper-resolution modeling method based on deep learning comprises selecting pathological high-resolution images, converting the images into low-resolution images through an algorithm, forming a high-resolution image-low-resolution image pair, setting a generator framework and a discriminator framework, pre-training a generator based on absolute value loss in a first stage to enable the absolute error value between a virtual high-resolution image generated by the generator based on the low-resolution images and a real high-resolution image to reach a minimum error set value, generating a virtual high-resolution image based on the pre-trained generator in a second stage, calculating the confrontation loss between the virtual high-resolution image and the real high-resolution image by adopting a GAN alternative training method, optimizing the discriminator based on the confrontation loss, calculating the perception loss between the virtual high-resolution image and the real high-resolution image, optimizing the generator based on, and adjusting the weight of each layer of the neural network through a back propagation algorithm to obtain a hyper-resolution model when the discriminator reaches Nash balance.
The invention is further configured to: the method comprises the steps of cutting a certain number of pathological high-resolution images to obtain high-resolution sub-images with set sizes, outputting a high-resolution sub-image noise distribution file by adopting a KernelGAN algorithm to the high-resolution sub-images, carrying out double-triple down-sampling on the high-resolution sub-images to obtain sub-images with sub-low resolution, disturbing noise in the noise distribution of the high-resolution sub-images, randomly distributing the noise to the sub-images with sub-low resolution to obtain sub-images with low resolution, and splitting the high-resolution sub-images and the sub-images with low resolution to obtain real high-resolution images and.
The invention is further configured to: based on a convolutional neural network, four main networks are adopted to separate a convolution block TrunkA, a symmetric convolution block TrunkB, a separation convolution block TrunkC and a large-field convolution block TrunkD, and a generator framework is set, wherein the TrunkA and the TrunkC comprise grouping convolution and point convolution and are used for reducing the parameter quantity and increasing the network depth; TrunkB is a symmetric convolution and is used for increasing the interoperability between pathological cell characteristic information, and TrunkD is a human retina receptor field simulated volume block and is used for increasing the characteristic receptor field range.
The invention is further configured to: the generator framework comprises a convolutional layer E, a TrunkA, a TrunkB, a TrunkC, a TrunkD, a convolutional layer F, an upper sampling layer, a convolutional layer G and an output layer; inputting an image into a convolutional layer E, extracting image characteristic information to obtain nonlinear expression data, enabling nonlinear expression to pass through TrunkA to obtain first characteristic data, fusing the nonlinear expression data with the first characteristic data to obtain second characteristic data, fusing the nonlinear expression data with the second characteristic data to obtain third characteristic data, fusing the nonlinear expression data with the third characteristic data to obtain fourth characteristic data, fusing the nonlinear expression data with the fourth characteristic data to input into a convolutional layer F to obtain fifth characteristic data, alternately adopting adjacent domain interpolation and sub-pixel convolution on the fifth characteristic data on an upper sampling layer to obtain sixth characteristic data, and outputting three-channel color image data after passing through a convolutional layer G.
The invention is further configured to: the discriminator comprises N layers of convolution layers, a pooling layer, a full-connection layer and an output layer; the N convolution layers are used for extracting low-order features, medium-order features and high-order features of the pathological image.
The invention is further configured to: in the first stage, a first iteration number, a first learning rate and a first learning rate adjusting frequency set value are set, the first learning rate is adjusted when the iteration reaches the first learning rate adjusting frequency set value, a virtual high-resolution image is generated based on the final low-resolution image, the virtual high-resolution image is compared with a real final high-resolution image to obtain an absolute value loss, and a BP algorithm is adopted for optimization to obtain a trained generator.
The invention is further configured to: in the second stage, a generator and a discriminator are alternately trained by adopting GAN, a second iteration number, a second learning rate and a second learning rate adjusting frequency set value are set, the second learning rate is adjusted when the iteration reaches the second learning rate adjusting frequency set value, and the discriminator and the generator are respectively trained.
The invention is further configured to: the training discriminator includes: the output probability of the discriminator on the real high-resolution image is set to W1, the output probability on the virtual high-resolution image generated by the generator is set to W2, wherein W1 and W2 respectively comprise a series of data, the first antagonistic loss of the real high-resolution image is BCELoss [ W1-mean (W2, 1], the second antagonistic loss of the virtual high-resolution image is BCELoss [ W2-mean (W1, 0], wherein BCELoss represents a two-classification loss formula, mean (W1) represents averaging of data of W1 series, mean (W2) represents averaging of data of W2 series, averaging of the first antagonistic loss and the second antagonistic loss, and the weight of each layer of the discriminator is adjusted through back propagation.
The invention is further configured to: the training generator includes: respectively outputting the characteristic graphs between the real high-resolution image and the virtual high-resolution image by using a VGG neural network, solving VGG loss and absolute value loss, and calculating perception loss, wherein a perception loss function is shown as the following formula: the perceived loss is K1 × absolute loss + VGG loss + K2 × antagonistic loss; where K1/K2 represents the weight of the corresponding parameter.
In a second aspect, the above object of the present invention is achieved by the following technical solutions:
a pathological image acquisition and processing device comprises a processor and a memory, wherein the memory stores a computer program capable of running on the processor, and the processor can realize the deep learning-based pathological image hyper-segmentation modeling method when executing the computer program.
Compared with the prior art, the beneficial technical effects of this application do:
1. the method and the device have the advantages that the low-resolution pathological images are obtained based on the real high-resolution pathological images, the one-to-one correspondence relationship of the images is guaranteed, the virtual high-resolution images are generated based on the low-resolution images and are identified, and the trueness of the virtual high-resolution images is improved;
2. further, the generator is pre-trained based on absolute value loss, and the truth of the virtual image generated by the generator is guaranteed;
3. further, the method and the device are based on a trained generator, a virtual high-resolution image is generated for the low-resolution pathological image, the loss-confrontation optimization discriminator is calculated through the calculation of the perception loss optimization generator, and the generation capacity of the generator and the discrimination capacity of the discriminator are improved.
Drawings
FIG. 1 is a schematic diagram of a generator architecture of a specific embodiment of the present application;
fig. 2 is a schematic diagram of a discriminator architecture according to an embodiment of the present application.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
Detailed description of the preferred embodiment
The application provides a pathological image hyper-resolution modeling method based on deep learning, which comprises the following steps: obtaining a low-resolution image from the pathology true high-resolution image, and generating a high-resolution image-low-resolution image pair; establishing a generator and discriminator framework; based on a low-resolution image pre-training generator, enabling an absolute error value of a virtual high-resolution image and a real high-resolution image generated by the generator to reach a minimum error set value; based on the confrontation loss of the virtual high-resolution image and the real high-resolution image, optimizing the discriminator, and based on the perception loss of the virtual high-resolution image and the real high-resolution image, optimizing the generator until the model training is finished to obtain a hyper-resolution model for carrying out high-resolution reconstruction on the pathological low-resolution image.
Obtaining a low resolution image from the pathology true high resolution image, generating a high resolution image-low resolution image pair, comprising the steps of:
s1, acquiring a certain number of high-resolution images of different pathologies;
s2, cutting the high-resolution image for the first time to obtain a high-resolution sub-image with the size of M × M;
s3, learning each high-resolution sub-image by adopting a kernel GAN algorithm, obtaining a high-resolution noise distribution file corresponding to each high-resolution sub-image, and storing the high-resolution noise distribution file;
s4, performing double-triple down-sampling on each high-resolution sub-image to obtain each low-resolution sub-image with the size of N x N, and correspondingly storing each high-resolution sub-image and each low-resolution sub-image;
s5, reading the high-resolution noise distribution file, disturbing the noise in the noise distribution of the high-resolution subimages, and randomly distributing the noise to the corresponding sub-low-resolution subimages to obtain low-resolution subimages;
s6, performing secondary cropping on each high-resolution sub-image to obtain a real high-resolution image with the size of P x P, and performing primary cropping on a low-resolution sub-image to obtain a low-resolution image with the size of Q x Q; a true high resolution image-a low resolution image pair is formed.
Wherein, a certain number of high-resolution images of different pathologies are real high-resolution images of different pathologies in nature, and real names are not added in order to distinguish the real high-resolution images after processing and cutting.
M, N, P, Q are different positive integers.
In step S5, the high resolution sub-image noise distribution of the high resolution sub-image a1 is read, and assuming that there are three cells B1/B2/B3 in the high resolution sub-image a1 and there are corresponding three-cell noise distributions C1/C2/C3, when the noise distributions are laid out in the sub-low resolution sub-images, the noise distributions C1/C2/C3 are not distributed on B1/B2/B3, but are randomly performed, that is, the noise distribution C1 may be distributed on B2 or B3, and the noise distribution C2 may be distributed on B1 or B3, so that the noise is randomly distributed, and the reality and universality of the low resolution images can be increased.
In one particular embodiment of the present application, M equals 1944, N equals 486, P equals 216, and Q equals 54.
The second cropping of the high resolution sub-image of size M x M, the first cropping of the low resolution sub-image is performed in a 9 x 9 manner.
Establishing a generator framework:
based on a convolutional neural network, four backbone networks of TrunkA, TrunkB, TrunkC and TrunkD are adopted, a generator framework is set, wherein the TrunkA and the TrunkC comprise grouping convolution and point convolution, common convolution is divided into two steps, the first step is the grouping convolution, and the second step is the point convolution. Firstly, the image is input into 1x1 convolution to keep the image nonlinear, then the depth separable convolution is accessed to carry out the grouping convolution, so that the parameter number is reduced compared with the conventional convolution, in the embodiment, the parameter number is reduced by 98%, and finally the grouping convolution is combined by using the point convolution, so that the parameter number is reduced, and the network depth is increased.
TrunkB is a symmetric convolution and is used for increasing the interoperability between pathological cell characteristic information and acquiring more pathological cell characteristic information through convolution layers with different channel numbers and different characteristic diagram sizes. In a specific embodiment of the present application, the number of channels is 64- >32- >16- >32- >64, which is a process for automatically removing redundant feature maps and automatically learning useful feature maps; the image size 54- >108- >54 is a learning process of a multi-scale feature map, a feature map with a larger size can obtain richer global information, and a feature map with a smaller size is used for obtaining richer local information.
TrunkD is a human retinal receptive field-mimicking rolling mass used to increase the range of characteristic receptive fields. By using a multi-branch structure, each branch captures a receptive field, and the information of the receptive fields is fused through the last connecting layer, so that the effect of simulating human vision is achieved.
In one embodiment of the present application, the generator architecture, as shown in fig. 1, includes a convolutional layer E, a split convolutional block TrunkA, a symmetric convolutional block TrunkB, a split convolutional block TrunkC, a large-field convolutional block TrunkD, a convolutional layer F, an upsampling layer, a convolutional layer G, and an output layer; inputting the image into a convolutional layer E, extracting high-dimensional and low-dimensional feature information of the image to obtain nonlinear expression data, obtaining first feature data after the nonlinear expression passes through TrunkA, fusing the nonlinear expression data with the first feature data, and inputting TrunkB to obtain second feature data; fusing the nonlinear expression data and the second characteristic data, and inputting the fused data into TrunkC to obtain third characteristic data; fusing the nonlinear expression data with the third characteristic data, and inputting TrunkD to obtain fourth characteristic data; fusing the nonlinear expression data and the fourth characteristic data, and inputting the fused data into the convolutional layer F to obtain fifth characteristic data; for the fifth feature data in the upper sampling layer, adjacent domain interpolation and sub-pixel convolution are alternately adopted, information exchange between an image space and network depth is improved, parameter quantity and time complexity are reduced, and sixth feature data are obtained; and outputting RGB three-channel color image data after passing through the convolution layer G.
Establishing a discriminator framework:
as shown in fig. 2, the discriminator architecture includes: ten convolutional layers, one pooling layer, one full-link layer, and one output layer.
The first convolution layer and the second convolution layer are used for extracting low-order features including colors and the like of the pathological image; the third convolution layer and the fourth convolution layer are used for extracting medium-order features including edges and the like of the pathological image; the fifth to tenth convolution layers are used to extract high-order features of the pathology image, including cell texture and the like. The pooling layer is used for keeping the size of the final output feature graph as W, the full-connection layer is used for combining the high-order features, and the output layer is used for outputting the probability that the image is a real image or a virtual image.
A pre-training generator:
setting the total number of iterations to be 100 ten thousand, the learning rate to be 0.0002 and the learning rate adjustment frequency to be 25 ten thousand, namely adjusting the learning rate once after every 25 ten thousand iterations; the optimizer strategy selects cosine annealing.
Inputting the low-resolution image into a generator, outputting a virtual high-resolution image, calculating the absolute value loss between the real high-resolution image and the virtual high-resolution image, continuously optimizing the generator by adopting a BP (back propagation) algorithm to reduce the absolute value loss between the virtual high-resolution image and the real high-resolution image, and stopping optimization when the absolute value loss reaches a minimum error set value to obtain a trained generator model.
By pre-training the generator, the generator learns the approximate distribution of high resolution, and the generated virtual high-resolution image has higher similarity with the real high-resolution image.
Then, based on the trained generator model, the generator and the discriminator are trained again, and the aim of obtaining a high-resolution image from a low-resolution image is achieved.
Specifically, a GAN algorithm is adopted, the total iteration frequency is set to be 40 ten thousand, the learning rate is 0.0001, the learning rate adjustment frequency is 5 ten thousand, namely, the learning rate is adjusted once after every 5 ten thousand iterations; the optimizer strategy selects to reach the set batch to reduce the learning rate once, and in this embodiment, after reaching the set batch, the learning rate is reduced by half.
Training a discriminator: firstly, the true probability of the discriminator output is defined to be 1, and the virtual probability is defined to be 0. The discriminator discriminates the true high-resolution image, and the output probabilities W1 are set to 0.9, 0.88 and 0.86. Inputting the virtual high-resolution image generated by the trained generator based on the low-resolution image into a discriminator for discrimination, and setting the output probability W2 to be 0.04, 0.07 and 0.09, wherein the first confrontation loss of the real high-resolution image is as follows:
BCELoss[W1-mean(W2),1]；
mean (W2) 0.06, then [ W1-mean (W2) ] (0.9-0.07, 0.88-0.06, 0.86-0.06] (0.83, 0.82, 0.8);
the second penalty for the virtual high resolution image is: BCELoss [ W2-mean (W1),0 ].
In the formula, BCELoss represents the loss of binary classification, and mean represents the average.
And averaging the first countermeasure loss and the second countermeasure loss, and adjusting the weight of each layer of the discriminator through back propagation.
Training generator: the generator here is a generator trained in the first phase.
And (3) outputting the characteristic diagrams between the real high-resolution image and the virtual high-resolution image by adopting a VGG neural network, and solving the VGG loss and the perception loss. The characteristic weights in the VGG network are from a pathological data set, key information can be highlighted when the loss of the characteristic diagram is solved, and the texture characteristic definition in the GAN training stage is improved.
In a specific embodiment of the present application, the discriminator discriminates the real high-resolution image, the output probability W1 is 0.9, the generator generates the virtual high-resolution image based on the low-resolution image, the discriminator discriminates the virtual high-resolution image, the output probability W2 is 0.1, and the countermeasure loss of the real high-resolution image is: true probability- [ W2-mean (W1) ];
solving the absolute value loss L1 of the virtual high-resolution image and the real high-resolution image;
the perceived loss is K1 × absolute loss + VGG loss + K2 × antagonistic loss.
In the formula, K1/K2 represents the weight of the corresponding parameter.
In one specific embodiment of the present application, K1 is 10 and K2 is 0.005.
The perceptual loss is calculated and a hyper-resolution model is obtained when the discriminator reaches Nash equilibrium.
Detailed description of the invention
An embodiment of the present invention of the present application provides a pathological image collection processing apparatus, including: a processor, a memory and a computer program, such as a pre-training generator program, a training generator and a discriminator program, stored in the memory and executable on the processor, the processor implementing the method of the first embodiment when executing the computer program.
Illustratively, the computer program may be partitioned into one or more modules/units that are stored in the memory and executed by the processor to implement the invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used for describing the execution process of the computer program in the acquisition processing device. For example, the computer program may be divided into a plurality of modules, each module having the following specific functions:
1. the image processing module is used for converting the pathological high-resolution image into a low-resolution image;
2. the generator pre-training module is used for pre-training the generator;
3. and the training module is used for training the generator and the discriminator.
The acquisition processing device can be a desktop computer, a notebook, a palm computer, a cloud server and other computing devices. The acquisition processing device may include, but is not limited to, a processor, a memory. It will be understood by those skilled in the art that the above examples are merely examples of the acquisition processing device, and do not constitute a limitation of the acquisition processing device, and may include more or less components than those shown, or combine some components, or different components, for example, the acquisition processing device may further include an input-output device, a network access device, a bus, and the like.
The Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. The general processor can be a microprocessor or the processor can be any conventional processor and the like, the processor is a control center of the collection processing device, and various interfaces and lines are utilized to connect various parts of the whole collection processing device.
The memory may be used to store the computer programs and/or modules, and the processor may implement various functions of the acquisition and processing device by running or executing the computer programs and/or modules stored in the memory and invoking data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.
The embodiments of the present invention are preferred embodiments of the present invention, and the scope of the present invention is not limited by these embodiments, so: all equivalent changes made according to the structure, shape and principle of the invention are covered by the protection scope of the invention.
Claims (10)
1. A pathological image hyper-resolution modeling method based on deep learning is characterized in that: selecting a pathological high-resolution image, converting the pathological high-resolution image into a low-resolution image through an algorithm to form a high-resolution image-low-resolution image pair, setting a generator framework and a discriminator framework, generating a virtual high-resolution image based on the generator in a first stage based on an absolute value loss pre-training generator to enable the absolute error value of the virtual high-resolution image generated by the generator based on the low-resolution image and the absolute error value of a real high-resolution image to reach a minimum error set value, generating the virtual high-resolution image based on the pre-trained generator in a second stage, adopting a GAN (global information network) alternative training method to calculate the confrontation loss between the virtual high-resolution image and the real high-resolution image, optimizing the discriminator based on the confrontation loss, calculating the perception loss between the virtual high, the hyper-score model is obtained when the discriminator reaches Nash equilibrium.
2. The deep learning-based pathological image hyper-segmentation modeling method according to claim 1, characterized in that: cutting a certain number of pathological high-resolution images to obtain high-resolution sub-images with set sizes, and adopting KernelGAN algorithm to the high-resolution sub-images，Outputting a high-resolution subimage noise distribution file, carrying out bicubic downsampling on the high-resolution subimage to obtain a sub-low-resolution subimage, randomly distributing noise disturbance in the noise distribution of the high-resolution subimage to the sub-low-resolution subimage to obtain a low-resolution subimage, and splitting the high-resolution subimage and the low-resolution subimage to obtain a real high-resolution subimage for subsequent processingResolution image, low resolution image pair.
3. The deep learning-based pathological image hyper-segmentation modeling method according to claim 1, characterized in that: based on a convolutional neural network, four main networks are adopted to separate a convolution block TrunkA, a symmetric convolution block TrunkB, a separation convolution block TrunkC and a large-field convolution block TrunkD, and a generator framework is set, wherein the TrunkA and the TrunkC comprise grouping convolution and point convolution and are used for reducing the parameter quantity and increasing the network depth; TrunkB is a symmetric convolution and is used for increasing the interoperability between pathological cell characteristic information, and TrunkD is a human retina receptor field simulated volume block and is used for increasing the characteristic receptor field range.
4. The deep learning-based pathological image hyper-segmentation modeling method according to claim 3, characterized in that: the generator framework comprises a convolutional layer E, a TrunkA, a TrunkB, a TrunkC, a TrunkD, a convolutional layer F, an upper sampling layer, a convolutional layer G and an output layer; inputting an image into a convolutional layer E, extracting image characteristic information to obtain nonlinear expression data, enabling nonlinear expression to pass through TrunkA to obtain first characteristic data, fusing the nonlinear expression data with the first characteristic data to obtain second characteristic data, fusing the nonlinear expression data with the second characteristic data to obtain third characteristic data, fusing the nonlinear expression data with the third characteristic data to obtain fourth characteristic data, fusing the nonlinear expression data with the fourth characteristic data to input into a convolutional layer F to obtain fifth characteristic data, alternately adopting adjacent domain interpolation and sub-pixel convolution on the fifth characteristic data on an upper sampling layer to obtain sixth characteristic data, and outputting three-channel color image data after passing through a convolutional layer G.
5. The deep learning-based pathological image hyper-segmentation modeling method according to claim 1, characterized in that: the discriminator comprises N layers of convolution layers, a pooling layer, a full-connection layer and an output layer; the N convolution layers are used for extracting low-order features, medium-order features and high-order features of the pathological image.
6. The deep learning-based pathological image hyper-segmentation modeling method according to claim 1, characterized in that: in the first stage, a first iteration number, a first learning rate and a first learning rate adjusting frequency set value are set, the first learning rate is adjusted when the iteration reaches the first learning rate adjusting frequency set value, a virtual high-resolution image is generated based on the final low-resolution image, the virtual high-resolution image is compared with a real final high-resolution image to obtain an absolute value loss, and a BP algorithm is adopted for optimization to obtain a trained generator.
7. The deep learning-based pathological image hyper-segmentation modeling method according to claim 1, characterized in that: in the second stage, a generator and a discriminator are alternately trained by adopting GAN, a second iteration number, a second learning rate and a second learning rate adjusting frequency set value are set, the second learning rate is adjusted when the iteration reaches the second learning rate adjusting frequency set value, and the discriminator and the generator are respectively trained.
8. The deep learning-based pathological image hyper-segmentation modeling method according to claim 7, characterized in that: the training discriminator includes: the output probability of the discriminator on the real high-resolution image is set to W1, the output probability on the virtual high-resolution image generated by the generator is set to W2, wherein W1 and W2 respectively comprise a series of data, the first antagonistic loss of the real high-resolution image is BCELoss [ W1-mean (W2, 1], the second antagonistic loss of the virtual high-resolution image is BCELoss [ W2-mean (W1, 0], wherein BCELoss represents a two-classification loss formula, mean (W1) represents averaging of data of W1 series, mean (W2) represents averaging of data of W2 series, averaging of the first antagonistic loss and the second antagonistic loss, and the weight of each layer of the discriminator is adjusted through back propagation.
9. The deep learning-based pathological image hyper-segmentation modeling method according to claim 7, characterized in that: the training generator includes: respectively outputting the characteristic graphs between the real high-resolution image and the virtual high-resolution image by using a VGG neural network, solving VGG loss and absolute value loss, and calculating perception loss, wherein a perception loss function is shown as the following formula: the perceived loss is K1 × absolute loss + VGG loss + K2 × antagonistic loss; where K1/K2 represents the weight of the corresponding parameter.
10. A pathological image acquisition and processing device, comprising a processor, a memory, said memory storing a computer program executable on said processor, said processor being capable of implementing the method according to any one of claims 1-9 when executing said computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011636550.4A CN112785498B (en) | 2020-12-31 | 2020-12-31 | Pathological image superscore modeling method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011636550.4A CN112785498B (en) | 2020-12-31 | 2020-12-31 | Pathological image superscore modeling method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112785498A true CN112785498A (en) | 2021-05-11 |
| CN112785498B CN112785498B (en) | 2023-06-02 |
Family
ID=75754932
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011636550.4A Active CN112785498B (en) | 2020-12-31 | 2020-12-31 | Pathological image superscore modeling method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112785498B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117196947A (en) * | 2023-09-06 | 2023-12-08 | 南通大学 | High-efficiency compression reconstruction model construction method for high-resolution image |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180075581A1 (en) * | 2016-09-15 | 2018-03-15 | Twitter, Inc. | Super resolution using a generative adversarial network |
| CN110070174A (en) * | 2019-04-10 | 2019-07-30 | 厦门美图之家科技有限公司 | A kind of stabilization training method generating confrontation network |
| CN110458767A (en) * | 2019-07-15 | 2019-11-15 | 循声科技(重庆)有限公司 | A kind of blur ed image restoration method under poisson noise |
| CN111369433A (en) * | 2019-11-12 | 2020-07-03 | 天津大学 | Three-dimensional image super-resolution reconstruction method based on separable convolution and attention |
| CN111583109A (en) * | 2020-04-23 | 2020-08-25 | 华南理工大学 | Image super-resolution method based on generation countermeasure network |
-
2020
- 2020-12-31 CN CN202011636550.4A patent/CN112785498B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180075581A1 (en) * | 2016-09-15 | 2018-03-15 | Twitter, Inc. | Super resolution using a generative adversarial network |
| CN110070174A (en) * | 2019-04-10 | 2019-07-30 | 厦门美图之家科技有限公司 | A kind of stabilization training method generating confrontation network |
| CN110458767A (en) * | 2019-07-15 | 2019-11-15 | 循声科技(重庆)有限公司 | A kind of blur ed image restoration method under poisson noise |
| CN111369433A (en) * | 2019-11-12 | 2020-07-03 | 天津大学 | Three-dimensional image super-resolution reconstruction method based on separable convolution and attention |
| CN111583109A (en) * | 2020-04-23 | 2020-08-25 | 华南理工大学 | Image super-resolution method based on generation countermeasure network |
Non-Patent Citations (3)
| Title |
|---|
| BELL-KLIGLER ET AL: "Blind Super-Resolution Kernel Estimation using an Internal-GAN", 《ARXIV》 * |
| BIAO LI ET AL: "s-LWSR: Super Lightweight Super-Resolution Network", 《 IEEE TRANSACTIONS ON IMAGE PROCESSING》 * |
| TAIZHANG SHANG ET AL: "Perceptual Extreme Super Resolution Network with Receptive Field Block", 《CVPR 2020》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117196947A (en) * | 2023-09-06 | 2023-12-08 | 南通大学 | High-efficiency compression reconstruction model construction method for high-resolution image |
| CN117196947B (en) * | 2023-09-06 | 2024-03-22 | 南通大学 | High-efficiency compression reconstruction model construction method for high-resolution image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112785498B (en) | 2023-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113240580B (en) | Lightweight image super-resolution reconstruction method based on multi-dimensional knowledge distillation | |
| CN111369440B (en) | Model training and image super-resolution processing method, device, terminal and storage medium | |
| CN110674829B (en) | Three-dimensional target detection method based on graph convolution attention network | |
| US11983850B2 (en) | Image processing method and apparatus, device, and storage medium | |
| WO2021018163A1 (en) | Neural network search method and apparatus | |
| CN113033570B (en) | Image semantic segmentation method for improving void convolution and multilevel characteristic information fusion | |
| CN109949219B (en) | Reconstruction method, device and equipment of super-resolution image | |
| CN111860398B (en) | Remote sensing image target detection method and system and terminal equipment | |
| CN112381097A (en) | Scene semantic segmentation method based on deep learning | |
| CN110223304B (en) | Image segmentation method and device based on multipath aggregation and computer-readable storage medium | |
| CN113177555B (en) | Target processing method and device based on cross-level, cross-scale and cross-attention mechanism | |
| CN109447897B (en) | Real scene image synthesis method and system | |
| CN111340077A (en) | Disparity map acquisition method and device based on attention mechanism | |
| US20230153965A1 (en) | Image processing method and related device | |
| CN116958324A (en) | Training method, device, equipment and storage medium of image generation model | |
| CN114612902A (en) | Image semantic segmentation method, device, equipment, storage medium and program product | |
| CN112257727A (en) | Feature image extraction method based on deep learning self-adaptive deformable convolution | |
| CN109829857B (en) | Method and device for correcting inclined image based on generation countermeasure network | |
| CN113158970B (en) | Action identification method and system based on fast and slow dual-flow graph convolutional neural network | |
| CN110599495A (en) | Image segmentation method based on semantic information mining | |
| CN112785498B (en) | Pathological image superscore modeling method based on deep learning | |
| CN107729885B (en) | Face enhancement method based on multiple residual error learning | |
| CN110866866A (en) | Image color-matching processing method and device, electronic device and storage medium | |
| WO2023122896A1 (en) | Data processing method and apparatus | |
| CN114140317A (en) | Image animation method based on cascade generation confrontation network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |