CN114972026A - Image processing method and storage medium - Google Patents
Image processing method and storage medium Download PDFInfo
- Publication number
- CN114972026A CN114972026A CN202210585099.0A CN202210585099A CN114972026A CN 114972026 A CN114972026 A CN 114972026A CN 202210585099 A CN202210585099 A CN 202210585099A CN 114972026 A CN114972026 A CN 114972026A
- Authority
- CN
- China
- Prior art keywords
- image
- neural network
- sample image
- network model
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 18
- 238000003062 neural network model Methods 0.000 claims abstract description 179
- 238000012545 processing Methods 0.000 claims abstract description 133
- 238000000034 method Methods 0.000 claims abstract description 82
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 claims abstract description 73
- 238000012549 training Methods 0.000 claims abstract description 58
- 238000004458 analytical method Methods 0.000 claims description 50
- 230000008569 process Effects 0.000 claims description 41
- 238000010586 diagram Methods 0.000 claims description 21
- 230000004913 activation Effects 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 15
- 230000011218 segmentation Effects 0.000 claims description 11
- 238000001514 detection method Methods 0.000 claims description 10
- 238000002372 labelling Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 description 26
- 230000015654 memory Effects 0.000 description 20
- 230000000694 effects Effects 0.000 description 9
- 210000004556 brain Anatomy 0.000 description 7
- 230000000903 blocking effect Effects 0.000 description 5
- 230000003902 lesion Effects 0.000 description 5
- 238000012952 Resampling Methods 0.000 description 4
- 238000013145 classification model Methods 0.000 description 2
- 238000004195 computer-aided diagnosis Methods 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 238000002059 diagnostic imaging Methods 0.000 description 2
- 208000024806 Brain atrophy Diseases 0.000 description 1
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 210000001015 abdomen Anatomy 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000002591 computed tomography Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000007787 long-term memory Effects 0.000 description 1
- 210000004072 lung Anatomy 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000002601 radiography Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 210000003625 skull Anatomy 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The present application relates to an image processing method and a storage medium. The method comprises the following steps: inputting the first medical image into a preset first neural network model to obtain a second medical image of which the resolution is greater than that of the first medical image; the first neural network model is obtained by training the first initial neural network model according to the processed sample image blocks corresponding to the sample images and the gold standard images corresponding to the sample images; the processed sample image block is obtained by performing first expansion processing and/or second expansion processing on each sample image block of the sample image. By adopting the method, the expansibility of the training of the first initial neural network model can be improved, and further, the expansibility of processing the first medical image by using the trained first neural network model to obtain the second medical image is improved.
Description
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to an image processing method and a storage medium.
Background
Medical imaging plays an important role in clinical diagnosis as a means of obtaining an image of the inside of a tissue in a human body or a part of a human body in a non-invasive manner. Computer Aided Diagnosis (CAD) analyzes medical images through a neural network model to give a preliminary diagnosis opinion, so that a doctor diagnoses a lesion region according to the preliminary diagnosis opinion.
However, training a neural network model with a high accuracy requires a large amount of medical data, and even if a sufficient amount of medical data can be acquired, due to the particularity of the medical data, for example, due to different imaging modes, different acquisition parameter settings, and the like, different medical data are acquired, and the performance of the model is greatly influenced by the different medical data.
Therefore, the traditional training method of the neural network model has the problem of poor expansibility.
Disclosure of Invention
In view of the above, it is necessary to provide an image processing method and a storage medium capable of improving the expandability of the neural network model in view of the above technical problems.
In a first aspect, the present application provides an image processing method, including:
acquiring a first medical image;
inputting the first medical image into a preset first neural network model to obtain a second medical image; wherein the resolution of the second medical image is greater than the resolution of the first medical image; the first neural network model is obtained by training a first initial neural network model according to the processed sample image blocks corresponding to the sample images and the gold standard images corresponding to the sample images; the processed sample image blocks are obtained by performing first expansion processing and/or second expansion processing on each sample image block of the sample image; the first expansion processing comprises at least one of adding blank image blocks into each sample image block, adjusting the arrangement sequence of each sample image block, replacing target image blocks in each sample image block and deleting at least one sample image block except the target image blocks in each sample image block; and the second expansion processing comprises adding blank image blocks into the feature map of each sample image block.
In one embodiment, if the processed sample image blocks are obtained by performing a first extension process on each sample image block of the sample image, the training process of the first neural network model includes:
performing the first expansion processing on each sample image block to obtain the processed sample image block;
inputting the processed image block into the first initial neural network model to obtain a third medical image;
and obtaining a value of a first loss function according to the third medical image and the gold standard image, and training the first initial neural network model according to the value of the first loss function to obtain the first neural network model.
In one embodiment, if the first expansion processing is replacement processing of a target image block in each sample image block, and the performing the first expansion processing on each sample image block to obtain the processed sample image block includes:
randomly selecting a plurality of first target image blocks from each sample image block;
replacing the plurality of first target image blocks with corresponding second target image blocks to obtain the processed sample image blocks; the resolution of the second target image block is higher than the resolution of the first target image block.
In one embodiment, if the first expansion processing is to delete at least one sample image block, except for the target image block, in each sample image block, and the first expansion processing is performed on each sample image block to obtain the processed sample image block, the method includes:
selecting a third target image block corresponding to a focus area in the sample image from each sample image block;
and deleting at least one sample image block except the third target image block in each sample image block to obtain the processed sample image block.
In one embodiment, if the processed sample image blocks are obtained by performing a second extension process on each sample image block of the sample image, the training process of the first neural network model includes:
inputting target sample image blocks in the sample image blocks into a coding layer of the first initial neural network model to obtain a feature map corresponding to the target sample image blocks;
adding blank image blocks into the feature map corresponding to each target sample image block according to the sequence of each sample image block to serve as the feature map corresponding to the sample image blocks except the target sample image block, and obtaining a processed feature map;
inputting the processed characteristic diagram into a decoding layer of the first initial neural network model to obtain a fourth medical image;
and obtaining a value of a second loss function according to the fourth medical image and the gold standard image, and training the first initial neural network model according to the value of the second loss function to obtain the first neural network model.
In one embodiment, the method further comprises:
inputting image blocks corresponding to the second medical image into a preset second neural network model to obtain an analysis result of the second medical image; the analysis result of the second medical image comprises any one of a classification result, a segmentation result and a detection result.
In one embodiment, the training process of the second neural network model comprises:
acquiring a gold standard analysis result corresponding to the gold standard image; the gold standard analysis result comprises any one of a gold standard classification result, a gold standard segmentation result and a gold standard detection result;
inputting image blocks corresponding to the second medical image into a second initial neural network model to obtain an analysis result of the second medical image;
obtaining a value of a third loss function according to the analysis result of the second medical image and the gold standard analysis result;
and performing cascade training on the first initial neural network model and the second initial neural network model according to the value of the third loss function to obtain the first neural network model and the second neural network model.
In one embodiment, the method further comprises:
obtaining a standard template image according to the gold standard analysis result; the standard template image is used for representing the labeling information of the gold standard analysis result;
acquiring a class activation graph output by the convolution layer of the second initial neural network model;
and performing constraint adjustment on the similar activation graph by using the standard template image to obtain the second neural network model.
In one embodiment, the activation-like graph is obtained by adjusting the feature graph output by the convolutional layer according to the weight of the fully-connected layer of the second initial neural network model.
In one embodiment, the method further comprises:
resampling the sample image to obtain a resampled sample image;
and carrying out block processing on the resampled sample images to obtain each sample image block.
In a second aspect, the present application further provides an image processing apparatus, comprising:
the first acquisition module is used for acquiring a first medical image;
the first processing module is used for inputting the first medical image into a preset first neural network model to obtain a second medical image; wherein the resolution of the second medical image is greater than the resolution of the first medical image; the first neural network model is obtained by training a first initial neural network model according to the processed sample image blocks corresponding to the sample images and the gold standard images corresponding to the sample images; the processed sample image blocks are obtained by performing first expansion processing and/or second expansion processing on each sample image block of the sample image; the first expansion processing comprises at least one of adding blank image blocks into each sample image block, adjusting the arrangement sequence of each sample image block, replacing target image blocks in each sample image block and deleting at least one sample image block except the target image blocks in each sample image block; and the second expansion processing comprises adding blank image blocks into the feature map of each sample image block.
In a third aspect, the present application further provides a computer device, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the following steps when executing the computer program:
acquiring a first medical image;
inputting the first medical image into a preset first neural network model to obtain a second medical image; wherein the resolution of the second medical image is greater than the resolution of the first medical image; the first neural network model is obtained by training a first initial neural network model according to the processed sample image blocks corresponding to the sample images and the gold standard images corresponding to the sample images; the processed sample image blocks are obtained by performing first expansion processing and/or second expansion processing on each sample image block of the sample image; the first expansion processing comprises at least one of adding blank image blocks into each sample image block, adjusting the arrangement sequence of each sample image block, replacing target image blocks in each sample image block and deleting at least one sample image block except the target image blocks in each sample image block; and the second expansion processing comprises adding blank image blocks into the feature map of each sample image block.
In a fourth aspect, the present application further provides a computer readable storage medium having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring a first medical image;
inputting the first medical image into a preset first neural network model to obtain a second medical image; wherein the resolution of the second medical image is greater than the resolution of the first medical image; the first neural network model is obtained by training a first initial neural network model according to the processed sample image blocks corresponding to the sample images and the gold standard images corresponding to the sample images; the processed sample image blocks are obtained by performing first expansion processing and/or second expansion processing on each sample image block of the sample image; the first expansion processing comprises at least one of adding blank image blocks into each sample image block, adjusting the arrangement sequence of each sample image block, replacing target image blocks in each sample image block and deleting at least one sample image block except the target image blocks in each sample image block; and the second expansion processing comprises adding blank image blocks into the feature map of each sample image block.
In a fifth aspect, the present application further provides a computer program product comprising a computer program which, when executed by a processor, performs the steps of:
acquiring a first medical image;
inputting the first medical image into a preset first neural network model to obtain a second medical image; wherein the resolution of the second medical image is greater than the resolution of the first medical image; the first neural network model is obtained by training a first initial neural network model according to the processed sample image blocks corresponding to the sample images and the gold standard images corresponding to the sample images; the processed sample image blocks are obtained by performing first expansion processing and/or second expansion processing on each sample image block of the sample image; the first expansion processing comprises at least one of adding blank image blocks into each sample image block, adjusting the arrangement sequence of each sample image block, replacing target image blocks in each sample image block and deleting at least one sample image block except the target image blocks in each sample image block; and the second expansion processing comprises adding blank image blocks into the feature map of each sample image block.
In the image processing method and the storage medium, the first neural network model is obtained by training the first initial neural network model according to the processed sample image blocks corresponding to the sample image and the gold standard image corresponding to the sample image, the processed sample image blocks are obtained by performing the first expansion processing and/or the second expansion processing on each sample image block of the sample image, the first expansion processing comprises at least one processing of adding blank image blocks into each sample image block, adjusting the arrangement sequence of each sample image block, performing replacement processing on the target image blocks in each sample image block and deleting at least one sample image block except the target image block in each sample image block, the second expansion processing comprises adding image block blanks into the feature images of each sample image block, thus excluding redundant information in the sample image, the first initial neural network model can learn deeper and more universal characteristics of the sample image through the first extension processing and/or the second extension processing performed on each sample image block of the sample image, and the richness of the obtained processed sample image blocks is improved by combining different first extension processing and second extension processing, so that the training expansibility of the first initial neural network model is improved, and the processing of the first medical image by using the trained first neural network model is further improved to obtain the expansibility of the second medical image.
Drawings
FIG. 1 is a diagram of an exemplary environment in which a method for image processing is implemented;
FIG. 2 is a flow diagram that illustrates a method for image processing according to one embodiment;
FIG. 3 is a schematic diagram illustrating a process of training a first initial neural network model using sample patches that are not extended in one embodiment;
FIG. 4 is a schematic diagram illustrating a process of adding blank image blocks to each sample image block to train a first initial neural network model according to an embodiment;
FIG. 5 is a diagram illustrating a process of training a first initial neural network model by adjusting an order of arrangement of sample image blocks according to an embodiment;
FIG. 6 is a schematic diagram illustrating a process of training a first initial neural network model by performing replacement processing on a target image block in each sample image block in one embodiment;
fig. 7 is a schematic diagram illustrating a reconstruction result of a first neural network model for first medical images with different layer thicknesses according to an embodiment;
fig. 8 is a schematic diagram of a reconstruction result of a first medical image obtained by scanning with different scanning apparatuses according to an embodiment of the first neural network model;
FIG. 9 is a flowchart illustrating an image processing method according to another embodiment;
FIG. 10 is a flowchart illustrating an image processing method according to another embodiment;
fig. 11 is a schematic process diagram illustrating second expansion processing performed on each sample image block of a sample image in an embodiment;
FIG. 12 is a flowchart illustrating an image processing method according to another embodiment;
FIG. 13 is a flowchart of a classification task in one embodiment;
FIG. 14 is a flowchart illustrating an image processing method according to another embodiment;
FIG. 15 is a block diagram showing a configuration of an image processing apparatus according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of and not restrictive on the broad application.
The image processing method provided by the embodiment of the application can be applied to the computer equipment shown in fig. 1. The computer device comprises a processor, a memory connected by a system bus, and a computer program stored in the memory, which when executed by the processor, performs the steps of the method embodiments described below. Optionally, the computer device may further comprise a network interface, a display screen and an input device. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a nonvolatile storage medium storing an operating system and a computer program, and an internal memory. The internal memory provides an environment for the operating system and the computer program to run on the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. Optionally, the computer device may be a server, a personal computer, a personal digital assistant, other terminal devices such as a tablet computer, a mobile phone, and the like, or a cloud or a remote server, and the embodiment of the present application does not limit the specific form of the computer device.
In one embodiment, as shown in fig. 2, an image processing method is provided, which is described by taking the method as an example applied to the computer device in fig. 1, and includes the following steps:
s201, acquiring a first medical image.
The first medical image may be a brain medical image, a lung medical image, an abdomen medical image, a chest medical image, or the like. Alternatively, the first medical image may be an image of any one modality of a Magnetic Resonance Image (MRI), a Computed Tomography image (CT), a Digital Radiography (DR), and a natural image. Alternatively, the computer device may acquire the first medical image from the medical imaging device in real time, or may acquire the first medical image from a PACS (Picture Archiving and Communication Systems) server. Optionally, the computer device may further perform preprocessing such as artifact removal and denoising on the acquired first medical image. In addition, it should be noted that the image processing method provided by the embodiment is not only applicable to two-dimensional medical images, but also applicable to three-dimensional medical images.
S202, inputting the first medical image into a preset first neural network model to obtain a second medical image; wherein the resolution of the second medical image is greater than the resolution of the first medical image; the first neural network model is obtained by training the first initial neural network model according to the processed sample image blocks corresponding to the sample images and the gold standard images corresponding to the sample images; the processed sample image blocks are obtained by performing first expansion processing and/or second expansion processing on each sample image block of the sample image; the first expansion processing comprises at least one of adding blank image blocks into each sample image block, adjusting the arrangement sequence of each sample image block, replacing target image blocks in each sample image block and deleting at least one sample image block except the target image blocks in each sample image block; the second expansion process includes adding a blank image block to the feature map of each sample image block.
In this embodiment, the first expansion processing includes at least one of adding a blank image block into each sample image block, adjusting an arrangement order of each sample image block, performing replacement processing on an object image block in each sample image block, and deleting at least one sample image block other than the object image block in each sample image block. Exemplarily, taking the first expansion process as adding a blank image block into each sample image block as an example, fig. 3 is a schematic diagram of a process of training a first initial neural network model by using sample image blocks which are not subjected to the expansion process in one embodiment, fig. 4 is a schematic diagram of a process of training a first initial neural network model by adding a blank image block into each sample image block in one embodiment, and an image block without any combined image information in fig. 4 is an added blank block. Taking the first expansion process as an example of adjusting the arrangement order of each sample image block, fig. 5 is a schematic diagram of a process of training the first initial neural network model by adjusting the arrangement order of each sample image block in an embodiment, and the sample image block with a frame of the input encoder in fig. 5 is the sample image block with the adjusted arrangement order. Taking the first expansion process as an example of replacing a target image block in each sample image block, in this scenario, the target image block may be a sample image block with higher resolution in each sample image block, or a sample image block at a focus portion in each sample image block, fig. 6 is a schematic diagram of a process of training a first initial neural network model by replacing the target image block in each sample image block in an embodiment, and the sample image block with a border input to the encoder in fig. 6 is an image block after replacement processing of the target image block. Taking the example that the first expansion process is to delete at least one sample image block of each sample image block except the target image block, in this scenario, the target image block may be any image block of the sample image blocks except the image block corresponding to the lesion area.
Optionally, in this embodiment, the first initial neural network model may be any one of a convolutional neural network model, a full convolutional network model, a deep neural network model, a generative confrontation network model, a cyclic neural network model, a deep residual network model, a long-term memory network model, and all existing neural network models suitable for images. Optionally, the computer device may segment the sample image by using a sliding window method to obtain each sample image block of the sample image; or, the computer device may also segment the sample image by using a segmentation template with a preset size to obtain each sample image block of the sample image. Optionally, the computer device may input the processed sample image block corresponding to the sample image into the first initial neural network model, obtain a value of a loss function by using the output of the first initial neural network model and the gold standard image corresponding to the sample image, and train the first initial neural network model according to the value of the loss function to obtain the first neural network model.
Fig. 7 is a schematic diagram of a reconstruction result of a first neural network model for first medical images with different layer thicknesses according to an embodiment, and fig. 8 is a schematic diagram of a reconstruction result of a first medical image scanned by different scanning devices according to an embodiment, and it can be seen from fig. 7 and fig. 8 that the first neural network model has higher robustness in reconstructing the first medical images scanned by different scanning devices with different layer thicknesses.
In the image processing method, the first neural network model is obtained by training the first initial neural network model according to the processed sample image blocks corresponding to the sample image and the gold standard image corresponding to the sample image, the processed sample image blocks are obtained by performing first expansion processing and/or second expansion processing on each sample image block of the sample image, the first expansion processing comprises adding blank image blocks into each sample image block, adjusting the arrangement sequence of each sample image block, performing replacement processing on target image blocks in each sample image block and deleting at least one of the sample image blocks except the target image block in each sample, the second expansion processing comprises adding blank image blocks into the feature image of each sample image block, thus eliminating redundant information in the sample image, and the first expansion processing and/or the second expansion processing are performed on each sample image block of the sample image The method has the advantages that the first initial neural network model can learn deeper and more universal characteristics of the sample image, and the richness of the obtained processed sample image block is improved by combining different first extension processing and second extension processing, so that the expansibility of training the first initial neural network model is improved, and further the expansibility of processing the first medical image by using the trained first neural network model to obtain the second medical image is improved.
In the scenario of training the first neural network model, if the processed sample image blocks are obtained by performing the first extension processing on each sample image block of the sample image, in an embodiment, as shown in fig. 9, the training process of the first neural network model includes:
and S301, performing first expansion processing on each sample image block to obtain a processed image block.
Optionally, in this embodiment, the first expansion processing performed on each sample image block by the computer device may include at least one of adding a blank image block into each sample image block, adjusting an arrangement order of each sample image block, performing replacement processing on a target image block in each sample image block, and deleting at least one sample image block except the target image block in each sample image block. The following will describe in detail that the first expansion processing is replacement processing of a target image block in each sample image block, and the first expansion processing is deletion of at least one sample image block except the target image block in each sample image block, respectively:
the first method comprises the following steps: if the first expansion process is a process of replacing a target image block in each sample image block, the process in S301 includes:
step A: a plurality of first target image blocks are arbitrarily selected from the sample image blocks.
Optionally, in this embodiment, the number of the selected first target image blocks may be determined in combination with actual application requirements, and then the first target image blocks of the number may be selected from each sample image block at will. Optionally, the selected first target image block may be an adjacent image block, or may be a different image block separated by multiple image blocks.
And B, step B: replacing the plurality of first target image blocks with corresponding second target image blocks to obtain processed image blocks; the resolution of the second target image block is higher than the resolution of the first target image block.
Optionally, in this embodiment, after the first target image block is selected, image enhancement processing may be performed on the first target image block to obtain a second target image block with a resolution higher than that of the first target image block, for example, histogram enhancement processing may be performed on the first target image block to obtain a second target image block, and then the selected plurality of first target image blocks are replaced by corresponding second target image blocks to obtain a processed image block.
And the second method comprises the following steps: if the first expansion process is to delete at least one sample image block of each sample image block except the target image block, the step S301 includes:
step C: and selecting a third target image block corresponding to the focus area in the sample image from the sample image blocks.
Optionally, the computer device may select an image block corresponding to the lesion area in the sample image from each sample image block of the sample image, and determine the image block corresponding to the selected lesion area as the third target image block. Exemplarily, taking the sample image as a brain image as an example, the computer device may determine an image block corresponding to a brain atrophy area in the brain image as the third target image block.
Step D: and deleting at least one sample image block except the third target image block in each sample image block to obtain a processed image block.
Optionally, in this embodiment, after selecting the third target image block from the sample image blocks of the sample image, the computer device may delete at least one sample image block, except the third target image block, from the sample image blocks to obtain a processed image block, that is, the computer device may delete an image block, except a focal region, from the sample image blocks to obtain a processed image block. It should be noted that a focus region in the medical image may affect the processing accuracy of the neural network model on the medical image, the focus region in the medical image is reserved, and in the process of training the first initial neural network model by using a sample image block including the focus region, the first initial neural network model can learn the features of the focus region, so that the first initial neural network model can process the focus region, thereby reducing the processing accuracy of the neural network model on the medical image.
It should be noted that, due to the anisotropy of the medical image, the resolution of the general cross section is higher, and the resolutions of the sagittal bit and the coronal bit are lower, in a scene where the medical image is subjected to the super-resolution reconstruction, the layers with high resolution may be deleted, so that the network knows that the layers do not need to be reconstructed.
And S302, inputting the processed image block into the first initial neural network model to obtain a third medical image.
Optionally, the computer device may sequentially input the processed image blocks into the first initial neural network model according to the order of the processed image blocks, so as to obtain a third medical image; or, the computer device may also input all the processed image blocks into the first initial neural network model at the same time to obtain the third medical image.
And S303, obtaining a value of a first loss function according to the third medical image and the gold standard image, and training the first initial neural network model according to the value of the first loss function to obtain a first neural network model.
In this embodiment, the computer device may obtain a value of the first loss function according to the third medical image and the gold standard image corresponding to the sample medical image, adjust a parameter of the first initial neural network model according to the value of the first loss function, and repeatedly execute the operation until the value of the first loss function reaches a minimum value or the value of the first loss function reaches a stable value, and use the first initial neural network model at this time as the first neural network model.
In this embodiment, by performing the first extension processing on each sample image block of the sample image, redundant information in the sample image is eliminated, so that the first initial neural network model can learn deeper and more universal characteristics of the sample image, and richness of the processed sample image blocks is improved, thereby improving extensibility of training of the first initial neural network model.
In the scenario of training the first neural network model, if the processed sample image blocks are obtained by performing the second extension process on each sample image block of the sample image, in an embodiment, as shown in fig. 10, the training process of the first neural network model includes:
and S401, inputting the target sample image blocks in the sample image blocks into the coding layer of the first initial neural network model to obtain the feature maps corresponding to the target sample image blocks.
Optionally, in this embodiment, the computer device may select a part of the sample image blocks from the sample image blocks as target sample image blocks, input the selected target sample image blocks into the coding layer of the first initial neural network model, and perform feature extraction on the target sample image blocks to obtain a feature map corresponding to each target sample image block.
And S402, adding blank image blocks into the feature map corresponding to each target sample image block according to the sequence of each sample image block to serve as the feature map corresponding to the sample image blocks except the target sample image block, and obtaining the processed feature map.
Optionally, in this embodiment, according to the sequence of each sample image block, the computer device may complement the corresponding place in the feature map corresponding to each target sample image block with a blank image block, and replace the feature map corresponding to the sample image blocks except the target sample image block with the blank image block, so as to obtain the processed feature map. It should be noted that the number of blank image blocks added corresponds to the number of sample image blocks of each sample image block except the target sample image block, and is not random. For example, as shown in fig. 11, a first column of image blocks after the brain image in fig. 11 are sample image blocks corresponding to the brain image, a second column of image blocks are target sample image blocks selected from the sample image blocks, the first column of image blocks after the encoder are feature maps corresponding to the target sample image blocks, and the second column of image blocks after the encoder are feature maps corresponding to all the sample image blocks, where the feature maps corresponding to the target sample image blocks correspond to the blank image blocks are complemented by blank image blocks according to the sequence of the sample image blocks, that is, the image blocks in the column are feature maps corresponding to all the sample image blocks.
And S403, inputting the processed characteristic diagram into a decoding layer of the first initial neural network model to obtain a fourth medical image.
Specifically, the computer device inputs the obtained processed feature map into a decoding layer of the first initial neural network model, and processes the input processed sample image by using the decoding layer of the first initial neural network model to obtain a fourth medical image corresponding to the sample image.
S404, obtaining a value of a second loss function according to the fourth medical image and the gold standard image, and training the first initial neural network model according to the value of the second loss function to obtain a first neural network model.
In this embodiment, the computer device obtains a value of the second loss function according to the obtained fourth medical image and the gold standard image corresponding to the sample image, adjusts parameters of the first initial neural network model according to the value of the second loss function, and repeatedly executes the operation until the value of the second loss function reaches a minimum value or the value of the second loss function reaches a stable value, and then takes the first initial neural network model at this time as the first neural network model.
In this embodiment, each sample image block of the sample image is input into the coding layer of the first initial neural network model to obtain a feature map corresponding to each sample image block, the feature maps corresponding to each sample image block are sorted according to the sequence of each sample image block to obtain a sorted feature map, and a blank image block is added into the sorted feature map to enrich the processed sample image blocks, so that the processed sample image blocks are input into the decoding layer of the first initial neural network model, and the first initial neural network model can learn deeper and more universal features of the sample image, thereby improving the extensibility of the training of the first initial neural network model.
In some scenarios, the obtained second medical image may be further analyzed to obtain an analysis result of the second medical image. In one embodiment, the method further comprises: inputting image blocks corresponding to the second medical image into a preset second neural network model to obtain an analysis result of the second medical image; the analysis result of the second medical image includes any one of a classification result, a segmentation result and a detection result.
In this embodiment, after obtaining the second medical image with a resolution greater than that of the first medical image, the second medical image may be further analyzed, for example, the second medical image may be classified, segmented, detected, and the like. Optionally, the second neural network model in this embodiment may be a classification model, a segmentation model, or a detection model. Optionally, the computer device may input image blocks corresponding to the second medical image into a preset second neural network model according to the arrangement order, so as to obtain an analysis result of the second medical image; or, the image blocks corresponding to the second medical image may be simultaneously input into a preset second neural network model to obtain an analysis result of the second medical image. Optionally, the second neural network model may be obtained through the following training manner:
step E, obtaining a gold standard analysis result corresponding to the variable standard image; the gold standard analysis result comprises any one of a gold standard classification result, a gold standard segmentation result and a gold standard detection result.
It can be understood that, when the second neural network model is used to classify the second medical image, the obtained gold standard analysis result is a gold standard classification result; when the second neural network model is used for segmenting the second medical image, the obtained gold standard analysis result is a gold standard segmentation result; when the second neural network model is used for detecting the second medical image, the obtained gold standard analysis result is a gold standard detection result.
Step F: and inputting the image block corresponding to the second medical image into the second initial neural network model to obtain an analysis result of the second medical image.
G: and obtaining a value of a third loss function according to the analysis result of the second medical image and the analysis result of the gold standard.
Step H: and performing cascade training on the first initial neural network model and the second initial neural network model according to the value of the third loss function to obtain the first neural network model and the second neural network model.
Optionally, the computer device may return to adjust parameters of the first initial neural network model and parameters of the second initial neural network model according to a value of the third loss function, and perform cascade training on the first initial neural network model and the second initial neural network model until the value of the third loss function reaches a minimum value or the value of the third loss function reaches a stable value, to obtain the first neural network model and the second neural network model.
In this embodiment, the computer device inputs the image block corresponding to the second medical image with the improved resolution into the preset second neural network model, and the analysis result of the second medical image can be obtained through the second neural network model.
In the above scenario of training the second neural network model, in an embodiment, as shown in fig. 12, the method further includes:
s501, obtaining a standard template image according to a gold standard analysis result; and the standard template image is used for representing the labeling information of the gold standard analysis result.
The standard template image is an image of labeling information for representing the gold standard analysis result. Optionally, the annotation information corresponding to the gold standard analysis result may be determined according to the gold standard analysis result, and the annotation information corresponding to the gold standard analysis result is highlighted in the second medical image to obtain the standard template image. Taking the second neural network model as the classification model as an example, the labeling information corresponding to the gold standard analysis result is the region range where the focus is located, so that the region range where the focus region is located can be highlighted in the standard template image.
S502, a class activation graph output by the convolution layer of the second initial neural network model is obtained.
The class activation map is obtained by mapping the output of the second initial neural network model back to the second medical image, and can be used for characterizing which part of the second medical image has a larger influence on the final analysis result. Optionally, the activation-like graph is obtained by adjusting a feature graph output by the convolutional layer of the second neural network model according to the weight of the fully-connected layer of the second initial neural network model. For example, as shown in fig. 13, fig. 13 is a schematic diagram of a classification task, and the class activation map in fig. 13 is a class activation map obtained by adjusting the feature map of the convolutional layer output according to the weight of the fully-connected layer of the second initial neural network model in this embodiment.
And S503, performing constraint adjustment on the class activation graph by using the standard template image to obtain a second neural network model.
Optionally, the constraint activation graph and the same class in the standard template image are in the same region where errors are allowed, errors caused by inaccuracy of capture of important features in an analysis task of the network due to different data sources and the like are reduced, and the constraint adjustment is performed on the constraint activation graph by using the standard template image, so that the second neural network model is obtained. For example, with continuing reference to fig. 13, an image pointed by the classification result in fig. 13 is the standard template image in this embodiment, and errors caused by inaccuracy in capturing important features in the classification task of the second initial neural network model due to different data sources can be reduced by constraining a highlight portion in the class activation map shown in fig. 13 to be in the same area where the error is allowed by the same class image in the standard template image, so as to improve expandability of the second initial neural network model on each central data.
In this embodiment, according to a gold standard analysis result corresponding to the gold standard image, label information representing the gold standard analysis result can be obtained, and a class activation graph output by a convolution layer of the second initial neural network model is obtained, so that the standard template image can be used to perform constraint adjustment on the class activation graph output by the convolution layer of the second neural network model, the class activation graph and an image of the same class in the standard template image are in the same region within an error allowable range, errors caused by inaccurate capture of important features in an analysis task of the network due to different data sources and the like are reduced, and thus the expandability of the second initial neural network model on each piece of central data is improved.
In the above scenario where the first expansion processing and/or the second expansion processing is performed on each sample image block of the sample image, each sample image block of the sample image needs to be obtained first. In one embodiment, as shown in fig. 14, the method further includes:
and S601, resampling the sample image to obtain a resampled sample image.
Optionally, in this embodiment, the sample image may be resampled to 1 × 1 × 1mm in the standard resolution space 3 . It should be noted that, the resampling processing is performed on the sample image, and the resampled sample image is input into the neural network model, so that the learning difficulty of the neural network model can be reduced, and the training efficiency of the neural network model can be improved. Optionally, if the sample image is a brain image, the skull and other non-brain parenchyma regions in the sample image can be removed.
And S602, carrying out blocking processing on the re-sampled sample image to obtain each sample image block.
Optionally, the computer device may perform blocking processing on the resampled sample image by using a preset image block size to obtain each sample image block corresponding to the sample image; or, the computer device may also perform blocking processing on the resampled sample image by using a sliding window method to obtain each sample image block corresponding to the sample image.
In this embodiment, the sample images are resampled to unify the sample images into images with the same resolution, and then the resampled sample images are subjected to blocking processing, so that consistency of each sample image block corresponding to the obtained sample images is ensured.
It should be understood that, although the steps in the flowcharts related to the above embodiments are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not limited to being performed in the exact order illustrated and, unless explicitly stated herein, may be performed in other orders. Moreover, at least a part of the steps in the flowcharts related to the above embodiments may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a part of the steps or stages in other steps.
Based on the same inventive concept, the embodiment of the present application further provides an image processing apparatus for implementing the image processing method. The implementation scheme for solving the problem provided by the apparatus is similar to the implementation scheme described in the method, so specific limitations in one or more embodiments of the image processing apparatus provided below may refer to the limitations on the image processing method in the foregoing, and details are not described here again.
In one embodiment, as shown in fig. 15, there is provided an image processing apparatus including: the device comprises a first acquisition module and a first processing module, wherein:
the first acquisition module is used for acquiring a first medical image.
The first processing module is used for inputting the first medical image into a preset first neural network model to obtain a second medical image; wherein the resolution of the second medical image is greater than the resolution of the first medical image; the first neural network model is obtained by training the first initial neural network model according to the processed sample image blocks corresponding to the sample images and the gold standard images corresponding to the sample images; the processed sample image blocks are obtained by performing first expansion processing and/or second expansion processing on each sample image block of the sample image; the first expansion processing comprises at least one of adding blank image blocks into each sample image block, adjusting the arrangement sequence of each sample image block, replacing target image blocks in each sample image block and deleting at least one sample image block except the target image blocks in each sample image block; the second expansion process includes adding a blank image block to the feature map of each sample image block.
The image processing apparatus provided in this embodiment may perform the method embodiments, and the implementation principle and technical effects are similar, which are not described herein again.
On the basis of the foregoing embodiment, optionally, if the processed sample image block is obtained by performing the first expansion processing on each sample image block of the sample image, the apparatus further includes: the second processing module, the second acquisition module and the first training module, wherein:
and the second processing module is used for performing first expansion processing on each sample image block to obtain a processed sample image block.
And the second acquisition module is used for inputting the processed image block into the first initial neural network model to obtain a third medical image.
And the first training module is used for obtaining a value of a first loss function according to the third medical image and the gold standard image, and training the first initial neural network model according to the value of the first loss function to obtain a first neural network model.
The image processing apparatus provided in this embodiment may perform the method embodiments, and the implementation principle and technical effects are similar, which are not described herein again.
On the basis of the foregoing embodiment, optionally, if the first expansion processing is replacement processing performed on a target image block in each sample image block, the second processing module includes: a first selection unit and a replacement unit, wherein:
and the first selection unit is used for randomly selecting a plurality of first target image blocks from each sample image block.
The replacing unit is used for replacing the plurality of first target image blocks with the corresponding second target image blocks to obtain processed sample image blocks; the resolution of the second target image block is higher than the resolution of the first target image block.
The image processing apparatus provided in this embodiment may perform the method embodiments, and the implementation principle and technical effects are similar, which are not described herein again.
On the basis of the foregoing embodiment, optionally, if the first expansion processing is to delete at least one sample image block of each sample image block except for the target image block, the second processing module includes: a second selecting unit and a deleting unit, wherein:
and the second selection unit is used for selecting a third target image block corresponding to the lesion area in the sample image from all the sample image blocks.
And the deleting unit is used for deleting at least one sample image block except the third target image block in each sample image block to obtain a processed sample image block.
The image processing apparatus provided in this embodiment may perform the method embodiments, and the implementation principle and technical effects are similar, which are not described herein again.
On the basis of the foregoing embodiment, optionally, if the processed sample image block is obtained by performing the second expansion processing on each sample image block of the sample image, the apparatus further includes: the third acquisition module, the sequencing module, the fourth acquisition module and the second training module, wherein:
and the third acquisition module is used for inputting the target sample image blocks in the sample image blocks into the coding layer of the first initial neural network model to obtain the characteristic maps corresponding to the target sample image blocks.
And the sorting module is used for adding blank image blocks into the feature map corresponding to each target sample image block according to the sequence of each sample image block as the feature map corresponding to the sample image blocks except the target sample image block to obtain the processed feature map.
And the fourth acquisition module is used for inputting the processed characteristic diagram into a decoding layer of the first initial neural network model to obtain a fourth medical image.
And the second training module is used for obtaining a value of a second loss function according to the fourth medical image and the gold standard image, and training the first initial neural network model according to the value of the second loss function to obtain the first neural network model.
The image processing apparatus provided in this embodiment may perform the method embodiments, and the implementation principle and technical effects are similar, which are not described herein again.
On the basis of the foregoing embodiment, optionally, the apparatus further includes: a fifth obtaining module, wherein:
the fifth acquisition module is used for inputting the image block corresponding to the second medical image into a preset second neural network model to obtain an analysis result of the second medical image; the analysis result of the second medical image includes any one of a classification result, a segmentation result and a detection result.
The image processing apparatus provided in this embodiment may perform the method embodiments, and the implementation principle and technical effects are similar, which are not described herein again.
On the basis of the foregoing embodiment, optionally, the apparatus further includes: a sixth acquisition module, a seventh acquisition module, an eighth acquisition module, and a third training module, wherein:
a sixth obtaining module, configured to obtain a gold standard analysis result corresponding to the gold standard image; the gold standard analysis result comprises any one of a gold standard classification result, a gold standard segmentation result and a gold standard detection result.
And the seventh acquisition module is used for inputting the image blocks corresponding to the second medical image into the second initial neural network model to obtain the analysis result of the second medical image.
And the eighth acquisition module is used for acquiring a value of a third loss function according to the analysis result of the second medical image and the analysis result of the gold standard.
And the third training module is used for carrying out cascade training on the first initial neural network model and the second initial neural network model according to the value of the third loss function to obtain the first neural network model and the second neural network model.
The image processing apparatus provided in this embodiment may perform the method embodiments, and the implementation principle and technical effects are similar, which are not described herein again.
On the basis of the foregoing embodiment, optionally, the apparatus further includes: a ninth obtaining module, a tenth obtaining module and an adjusting module, wherein:
the ninth acquisition module is used for acquiring a standard template image according to the gold standard analysis result; and the standard template image is used for representing the labeling information of the gold standard analysis result.
And the tenth acquisition module is used for acquiring the activation-like graph output by the convolution layer of the second initial neural network model.
And the adjusting module is used for performing constraint adjustment on the class activation graph by adopting the standard template image so as to obtain a second neural network model.
Optionally, the class activation graph is obtained by adjusting a feature graph output by the convolutional layer according to the weight of the fully-connected layer of the second initial neural network model.
The image processing apparatus provided in this embodiment may perform the method embodiments, and the implementation principle and technical effects are similar, which are not described herein again.
On the basis of the foregoing embodiment, optionally, the apparatus further includes: a third processing module and a fourth processing module, wherein:
and the third processing module is used for resampling the sample image to obtain a resampled sample image.
And the fourth processing module is used for carrying out blocking processing on the re-sampled sample image to obtain each sample image block.
The image processing apparatus provided in this embodiment may perform the method embodiments, and the implementation principle and technical effects are similar, which are not described herein again.
The respective modules in the image processing apparatus described above may be wholly or partially implemented by software, hardware, and a combination thereof. The modules can be embedded in a hardware form or independent of a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, comprising a memory having a computer program stored therein and a processor that when executing the computer program performs the steps of:
acquiring a first medical image;
inputting the first medical image into a preset first neural network model to obtain a second medical image; wherein the resolution of the second medical image is greater than the resolution of the first medical image; the first neural network model is obtained by training the first initial neural network model according to the processed sample image blocks corresponding to the sample images and the gold standard images corresponding to the sample images; the processed sample image blocks are obtained by performing first expansion processing and/or second expansion processing on each sample image block of the sample image; the first expansion processing comprises at least one of adding blank image blocks into each sample image block, adjusting the arrangement sequence of each sample image block, replacing target image blocks in each sample image block and deleting at least one sample image block except the target image blocks in each sample image block; the second expansion process includes adding a blank image block to the feature map of each sample image block.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, performs the steps of:
acquiring a first medical image;
inputting the first medical image into a preset first neural network model to obtain a second medical image; wherein the resolution of the second medical image is greater than the resolution of the first medical image; the first neural network model is obtained by training the first initial neural network model according to the processed sample image blocks corresponding to the sample images and the gold standard images corresponding to the sample images; the processed sample image blocks are obtained by performing first expansion processing and/or second expansion processing on each sample image block of the sample image; the first expansion processing comprises at least one of adding blank image blocks into each sample image block, adjusting the arrangement sequence of each sample image block, replacing target image blocks in each sample image block and deleting at least one sample image block except the target image blocks in each sample image block; the second expansion process includes adding a blank image block to the feature map of each sample image block.
In one embodiment, a computer program product is provided, comprising a computer program which when executed by a processor performs the steps of:
acquiring a first medical image;
inputting the first medical image into a preset first neural network model to obtain a second medical image; wherein the resolution of the second medical image is greater than the resolution of the first medical image; the first neural network model is obtained by training the first initial neural network model according to the processed sample image blocks corresponding to the sample images and the gold standard images corresponding to the sample images; the processed sample image blocks are obtained by performing first expansion processing and/or second expansion processing on each sample image block of the sample image; the first expansion processing comprises at least one of adding blank image blocks into each sample image block, adjusting the arrangement sequence of each sample image block, replacing target image blocks in each sample image block and deleting at least one sample image block except the target image blocks in each sample image block; the second expansion process includes adding a blank image block to the feature map of each sample image block.
It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above may be implemented by hardware instructions of a computer program, which may be stored in a non-volatile computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. Any reference to memory, databases, or other media used in the embodiments provided herein can include at least one of non-volatile and volatile memory. The nonvolatile Memory may include a Read-Only Memory (ROM), a magnetic tape, a floppy disk, a flash Memory, an optical Memory, a high-density embedded nonvolatile Memory, a resistive Random Access Memory (ReRAM), a Magnetic Random Access Memory (MRAM), a Ferroelectric Random Access Memory (FRAM), a Phase Change Memory (PCM), a graphene Memory, and the like. Volatile Memory can include Random Access Memory (RAM), external cache Memory, and the like. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), for example. The databases involved in the embodiments provided herein may include at least one of relational and non-relational databases. The non-relational database may include, but is not limited to, a block chain based distributed database, and the like. The processors referred to in the various embodiments provided herein may be, without limitation, general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, or the like.
All possible combinations of the technical features in the above embodiments may not be described for the sake of brevity, but should be considered as being within the scope of the present disclosure as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, and these are all within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.
Claims (10)
1. An image processing method, characterized in that the method comprises:
acquiring a first medical image;
inputting the first medical image into a preset first neural network model to obtain a second medical image; wherein the resolution of the second medical image is greater than the resolution of the first medical image; the first neural network model is obtained by training a first initial neural network model according to the processed sample image blocks corresponding to the sample images and the gold standard images corresponding to the sample images; the processed sample image blocks are obtained by performing first expansion processing and/or second expansion processing on each sample image block of the sample image; the first expansion processing comprises at least one of adding blank image blocks into each sample image block, adjusting the arrangement sequence of each sample image block, replacing target image blocks in each sample image block and deleting at least one sample image block except the target image blocks in each sample image block; and the second expansion processing comprises adding blank image blocks into the feature map of each sample image block.
2. The method according to claim 1, wherein if the processed sample patches are obtained by performing a first extension process on each sample patch of the sample image, the training process of the first neural network model includes:
performing the first expansion processing on each sample image block to obtain the processed sample image block;
inputting the processed image block into the first initial neural network model to obtain a third medical image;
and obtaining a value of a first loss function according to the third medical image and the gold standard image, and training the first initial neural network model according to the value of the first loss function to obtain the first neural network model.
3. The method according to claim 2, wherein if the first expansion processing is replacement processing of a target image block in each of the sample image blocks, the performing the first expansion processing on each of the sample image blocks to obtain the processed sample image block includes:
randomly selecting a plurality of first target image blocks from each sample image block;
replacing the plurality of first target image blocks with corresponding second target image blocks to obtain the processed sample image blocks; the resolution of the second target image block is higher than the resolution of the first target image block.
4. The method according to claim 2, wherein if the first expansion process is to delete at least one sample tile except for a target tile in each sample tile, the performing the first expansion process on each sample tile to obtain the processed sample tile includes:
selecting a third target image block corresponding to a focus area in the sample image from each sample image block;
and deleting at least one sample image block except the third target image block in each sample image block to obtain the processed sample image block.
5. The method according to claim 1, wherein if the processed sample patches are obtained by performing a second extension process on each sample patch of the sample image, the training process of the first neural network model includes:
inputting target sample image blocks in the sample image blocks into a coding layer of the first initial neural network model to obtain a feature map corresponding to the target sample image blocks;
adding blank image blocks into the feature map corresponding to each target sample image block according to the sequence of each sample image block to serve as the feature map corresponding to the sample image blocks except the target sample image block, and obtaining a processed feature map;
inputting the processed characteristic diagram into a decoding layer of the first initial neural network model to obtain a fourth medical image;
and obtaining a value of a second loss function according to the fourth medical image and the gold standard image, and training the first initial neural network model according to the value of the second loss function to obtain the first neural network model.
6. The method according to any one of claims 1-5, further comprising:
inputting image blocks corresponding to the second medical image into a preset second neural network model to obtain an analysis result of the second medical image; the analysis result of the second medical image comprises any one of a classification result, a segmentation result and a detection result.
7. The method of claim 6, wherein the training process of the second neural network model comprises:
acquiring a gold standard analysis result corresponding to the gold standard image; the gold standard analysis result comprises any one of a gold standard classification result, a gold standard segmentation result and a gold standard detection result;
inputting image blocks corresponding to the second medical image into a second initial neural network model to obtain an analysis result of the second medical image;
obtaining a value of a third loss function according to the analysis result of the second medical image and the gold standard analysis result;
and performing cascade training on the first initial neural network model and the second initial neural network model according to the value of the third loss function to obtain the first neural network model and the second neural network model.
8. The method of claim 7, further comprising:
obtaining a standard template image according to the gold standard analysis result; the standard template image is used for representing the labeling information of the gold standard analysis result;
acquiring a class activation graph output by the convolution layer of the second initial neural network model;
and performing constraint adjustment on the similar activation graph by using the standard template image to obtain the second neural network model.
9. The method of claim 8, wherein the activation-like graph is obtained by adjusting a feature graph output by the convolutional layer according to weights of fully-connected layers of the second initial neural network model.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210585099.0A CN114972026A (en) | 2022-05-27 | 2022-05-27 | Image processing method and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210585099.0A CN114972026A (en) | 2022-05-27 | 2022-05-27 | Image processing method and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114972026A true CN114972026A (en) | 2022-08-30 |
Family
ID=82955170
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210585099.0A Pending CN114972026A (en) | 2022-05-27 | 2022-05-27 | Image processing method and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114972026A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116071375A (en) * | 2023-03-10 | 2023-05-05 | 福建自贸试验区厦门片区Manteia数据科技有限公司 | Image segmentation method and device, storage medium and electronic equipment |
-
2022
- 2022-05-27 CN CN202210585099.0A patent/CN114972026A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116071375A (en) * | 2023-03-10 | 2023-05-05 | 福建自贸试验区厦门片区Manteia数据科技有限公司 | Image segmentation method and device, storage medium and electronic equipment |
| CN116071375B (en) * | 2023-03-10 | 2023-09-26 | 福建自贸试验区厦门片区Manteia数据科技有限公司 | Image segmentation method and device, storage medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220406410A1 (en) | System and method for creating, querying, and displaying a miba master file | |
| CN111369542B (en) | Vessel marking method, image processing system, and storage medium | |
| US10722210B2 (en) | Method for memorable image generation for anonymized three-dimensional medical image workflows | |
| JP2023550844A (en) | Liver CT automatic segmentation method based on deep shape learning | |
| EP3874457B1 (en) | Three-dimensional shape reconstruction from a topogram in medical imaging | |
| CN112885453A (en) | Method and system for identifying pathological changes in subsequent medical images | |
| Gaggion et al. | Improving anatomical plausibility in medical image segmentation via hybrid graph neural networks: applications to chest x-ray analysis | |
| CN110717905B (en) | Brain image detection method, computer device, and storage medium | |
| CN110210519B (en) | Classification method, computer device, and storage medium | |
| WO2023044605A1 (en) | Three-dimensional reconstruction method and apparatus for brain structure in extreme environments, and readable storage medium | |
| CN110223279B (en) | Image processing method and device and electronic equipment | |
| CN111161269A (en) | Image segmentation method, computer device and readable storage medium | |
| KR101885562B1 (en) | Method for mapping region of interest in first medical image onto second medical image and apparatus using the same | |
| CN111918611B (en) | Method for controlling abnormal display of chest X-ray image, recording medium and apparatus | |
| CN114972026A (en) | Image processing method and storage medium | |
| CN113160199B (en) | Image recognition method and device, computer equipment and storage medium | |
| CN111209946B (en) | Three-dimensional image processing method, image processing model training method and medium | |
| CN111091504A (en) | Image deviation field correction method, computer device, and storage medium | |
| CN116630239A (en) | Image analysis method, device and computer equipment | |
| WO2022163513A1 (en) | Learned model generation method, machine learning system, program, and medical image processing device | |
| CN113327221B (en) | Image synthesis method, device, electronic equipment and medium for fusing ROI (region of interest) | |
| CN114723723A (en) | Medical image processing method, computer device and storage medium | |
| CN110874614B (en) | Brain image classification method, computer device, and readable storage medium | |
| CN110415239B (en) | Image processing method, image processing apparatus, medical electronic device, and medium | |
| US20230306608A1 (en) | Image processing apparatus, method and program, and learning apparatus, method and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |