CN110648288B - Choledochoscope image enhancement method based on residual transposition deconvolution neural network - Google Patents
Choledochoscope image enhancement method based on residual transposition deconvolution neural network Download PDFInfo
- Publication number
- CN110648288B CN110648288B CN201910793892.8A CN201910793892A CN110648288B CN 110648288 B CN110648288 B CN 110648288B CN 201910793892 A CN201910793892 A CN 201910793892A CN 110648288 B CN110648288 B CN 110648288B
- Authority
- CN
- China
- Prior art keywords
- image
- choledochoscope
- bit
- image enhancement
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 38
- 230000017105 transposition Effects 0.000 title description 17
- 238000012549 training Methods 0.000 claims abstract description 64
- 238000012545 processing Methods 0.000 claims abstract description 24
- 238000003384 imaging method Methods 0.000 claims abstract description 18
- 230000002708 enhancing effect Effects 0.000 claims abstract description 6
- 238000012795 verification Methods 0.000 claims description 62
- 238000012360 testing method Methods 0.000 claims description 35
- 210000003445 biliary tract Anatomy 0.000 claims description 29
- 230000000694 effects Effects 0.000 claims description 18
- 230000011218 segmentation Effects 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 8
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 7
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 206010004637 Bile duct stone Diseases 0.000 abstract description 2
- 201000009331 Choledocholithiasis Diseases 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 6
- 238000002350 laparotomy Methods 0.000 description 3
- HVYWMOMLDIMFJA-DPAQBDIFSA-N cholesterol Chemical compound C1C=C2C[C@@H](O)CC[C@]2(C)[C@@H]2[C@@H]1[C@@H]1CC[C@H]([C@H](C)CCCC(C)C)[C@@]1(C)CC2 HVYWMOMLDIMFJA-DPAQBDIFSA-N 0.000 description 2
- 208000001130 gallstones Diseases 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 210000001835 viscera Anatomy 0.000 description 2
- 210000000683 abdominal cavity Anatomy 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 210000000941 bile Anatomy 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 210000001124 body fluid Anatomy 0.000 description 1
- 239000010839 body fluid Substances 0.000 description 1
- 235000012000 cholesterol Nutrition 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000002503 metabolic effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 231100000915 pathological change Toxicity 0.000 description 1
- 230000036285 pathological change Effects 0.000 description 1
- 244000144985 peep Species 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000028327 secretion Effects 0.000 description 1
- 239000004575 stone Substances 0.000 description 1
Images
Classifications
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Abstract
The invention provides a cholangioscope image enhancement method based on a residual transpose deconvolution neural network, which is used for enhancing the imaging quality of a cholangioscope image to be processed and comprises the following steps: carrying out motion blur removing processing and denoising processing on the choledochoscope image to be processed to obtain a choledochoscope image to be enhanced; and inputting the choledochoscope image to be enhanced into a final image enhancement model after training for enhancement processing to obtain a final choledochoscope image. The final image enhancement model is trained by a training method of matching the low-digit image with the high-digit image, and after the training is finished, the final image enhancement model can increase the image digit of the choledochoscope image to be processed, so that the imaging quality of the choledochoscope image to be processed is further enhanced on the existing basis, and the success rate of the choledocholithiasis operation can be further improved.
Description
Technical Field
The invention belongs to computer-assisted medicine, relates to a choledochoscope image enhancement method, and particularly relates to a choledochoscope image enhancement method based on a residual transposition deconvolution neural network.
Background
Biliary calculi are clinically a common disease type, and the causes of the biliary calculi are many, mainly caused by the metabolic disturbance of cholesterol and the reduction of bile secretion of patients. At present, the main clinical treatment mode for the biliary tract calculus is a choledochoscope combined laparoscopic laparotomy.
The choledochoscope has high image quality and a working channel for operation, can greatly improve the treatment effect of a patient, has less interference to the abdominal cavity and little damage, can directly see the visceral organs under the stable condition of keeping the body environment, is favorable for knowing the shape, the size, the peripheral adhesion condition and the like of the visceral organs, and can accurately and timely judge the degree and the size of pathological changes of various organs. Through the combined laparoscopic laparotomy of choledochoscope, can reduce the operation wound, reduce the unnecessary inspection, and can directly peep and get the stone through the basket, the security obviously improves. In the choledochoscope combined laparoscopic laparotomy, the imaging quality of a choledochoscope image directly influences the success rate of the operation.
However, at present, the research for improving the imaging quality of the choledochoscope image is applied to the performance improvement of hardware facilities such as an endoscope lens, and the research for further improving the imaging quality of the choledochoscope image through an enhancement model on the existing imaging result is very little, so that the enhancement effect of the imaging quality of the choledochoscope image is insufficient, and the success rate of the choledocholithiasis operation is affected.
Disclosure of Invention
In order to solve the problems, the invention provides an image enhancement method capable of further improving the imaging quality of a choledochoscope image on the existing imaging result, and adopts the following technical scheme:
the invention provides a choledochoscope image enhancement method based on a residual transpose deconvolution neural network, which is used for enhancing the imaging quality of a choledochoscope image to be processed and is characterized by comprising the following steps of:
s1, performing motion blur removing processing and denoising processing on a choledochoscope image to be processed to obtain a choledochoscope image to be enhanced;
s2, inputting the choledochoscope image to be enhanced into a final image enhancement model after training for enhancement processing to obtain a final choledochoscope image;
the final training process of the image enhancement model comprises the following steps:
step T1, constructing a deconvolution residual error transposition network model as an initial image enhancement model;
t2, selecting a plurality of biliary tract endoscope images similar to the biliary tract endoscope image to be processed as a training image and a verification image;
step T3, dividing the training image and the verification image into a low-digit image and a high-digit image corresponding to the low-digit image according to different bit depths, and taking the low-digit image and the high-digit image as a training set and a verification set of the initial image enhancement model;
step T4, inputting the training set into the initial image enhancement model for training to obtain an image enhancement model to be verified;
step T5, inputting the verification set into an image enhancement model to be verified for verification to obtain a model verification result;
and T6, storing the image enhancement model to be verified with the optimal enhancement effect as a final image enhancement model according to the model verification result.
The choledochoscope image enhancement method based on the residual error transposition deconvolution neural network provided by the invention can also have the technical characteristics, and the test process of the final image enhancement model comprises the following steps:
a1, selecting a plurality of biliary tract endoscope images similar to a biliary tract endoscope image to be processed as test images;
step A2, dividing the test image into a low-bit image and a high-bit image corresponding to the low-bit image according to different bit depths, and taking the low-bit image and the high-bit image as a test set of a final image enhancement model;
and step A3, inputting the test set into the final image enhancement model to obtain the actual enhancement effect and robustness of the final enhancement model.
The choledochoscope image enhancement method based on the residual error transposition deconvolution neural network, provided by the invention, can also have the technical characteristics that the images in the training set, the verification set and the test set are divided into a low-bit image and a high-bit image according to the bit depths of 8-bit class and 16-bit class.
The choledochoscope image enhancement method based on the residual transpose deconvolution neural network can also have the technical characteristics that in the step T1, an initial image enhancement model comprises a characteristic top layer and a characteristic bottom layer, and the initial image enhancement model adopts a convolution kernel of 3 x 3 as a deconvolution operator for collecting characteristic information of the characteristic top layer and the characteristic bottom layer.
The choledochoscope image enhancement method based on the residual error transposition deconvolution neural network, provided by the invention, can also have the technical characteristics that in the step T1, a residual block of an initial image enhancement model is arranged between the characteristic top layer and the characteristic bottom layer, and the characteristic top layer and the characteristic bottom layer are combined through jump connection, so that the residual block is skipped when the characteristic information of the characteristic top layer and the characteristic bottom layer is combined through the jump connection.
The choledochoscope image enhancement method based on the residual error transposition deconvolution neural network provided by the invention can also have the technical characteristics that in the step T1, the parameter settings of the initial image enhancement model are respectively as follows: the optimizer chooses Adam with beta1 set to 0.5, beta2 set to 0.9, batch size set to 5, and range of learning rates set to 1 × 10 -4 To 1X 10 -5 Step size is set to 1 × 10 -5 。
The choledochoscope image enhancement method based on the residual error transposition deconvolution neural network provided by the invention can also have the technical characteristics that in the step T4, the method comprises the following substeps:
step T4-1, segmenting images in the training set to obtain image blocks with the size of 96 pixels multiplied by 96 pixels;
step T4-2, taking the segmentation image blocks of each low-bit image and the segmentation image blocks of the high-bit images corresponding to each low-bit image in the training set as a training subset;
and T4-3, sequentially inputting each training subset into the initial image enhancement model for training to obtain the image enhancement model to be verified.
The choledochoscope image enhancement method based on the residual error transposition deconvolution neural network provided by the invention can also have the technical characteristics that in the step T5, the method comprises the following substeps:
step T5-1, dividing the image in the verification set to obtain an image block with the size of 96 pixels multiplied by 96 pixels;
step T5-2, taking the segmentation image block of each low-bit image and the segmentation image block of the high-bit image corresponding to each low-bit image in the verification set as a verification subset;
and T5-3, sequentially inputting each verification subset into the image enhancement model to be verified for verification, and obtaining a model verification result.
The choledochoscope image enhancement method based on the residual transpose deconvolution neural network, which is provided by the invention, can also have the technical characteristics that in the step S2, the method comprises the following substeps:
s2-1, selecting a biliary tract region on the choledochoscope image to be enhanced,
step S2-2, taking the center of the biliary tract area as an origin point, taking 96 pixels as step length to carry out interception, intercepting the biliary tract area into a plurality of image blocks to be enhanced with the size of 96 pixels multiplied by 96 pixels,
and S2-3, inputting the image block to be enhanced into the final image enhancement model after training for enhancement processing, and obtaining a final choledochoscope image.
The choledochoscope image enhancement method based on the residual error transposition deconvolution neural network, provided by the invention, can also have the technical characteristics that in the step S2-2, when a biliary tract area is intercepted by taking 96 pixels as a step length, an area with the edge less than 96 pixels in size in the biliary tract area is removed.
Action and Effect of the invention
According to the choledochoscope image enhancement method based on the residual error transposition deconvolution neural network, a deconvolution residual error transposition network model is constructed to serve as an initial image enhancement model, a plurality of choledochoscope images similar to the choledochoscope image to be processed are selected to serve as a training image and a verification image, the training image and the verification image are divided into a low-bit image and a high-bit image corresponding to the low-bit image according to different bit depths to serve as a training set and a verification set of the initial image enhancement model, the training set is input into the initial image enhancement model to be trained, the image enhancement model to be verified can be obtained, the verification set is input into the image enhancement model to be verified, a model verification result can be obtained, the image enhancement model to be verified with the optimal enhancement effect can be stored according to the model verification result and serves as a final image enhancement model, therefore, compared with the existing Laplace of a Laplace pyramid filter, a cosine transform and the like, the final image enhancement model can be trained through a training method of matching the low-bit image enhancement model with the low-bit image enhancement model and the high-bit image enhancement model, and multiple aspects can be more accurately displayed.
Furthermore, the choledochoscope image to be processed is subjected to motion blur removing processing and denoising processing to obtain a choledochoscope image to be enhanced, and then the choledochoscope image to be enhanced is input into a final image enhancement model after training is completed to be enhanced, so that a final choledochoscope image with the optimal enhancement effect can be obtained.
Drawings
Fig. 1 is a schematic structural diagram of a deconvolution residual transposed network model of a choledochoscope image enhancement method based on a residual transposed deconvolution neural network according to an embodiment of the present invention;
FIG. 2 is a diagram of a jump connection processing procedure of a choledochoscope image enhancement method based on a residual transpose deconvolution neural network according to an embodiment of the present invention;
FIG. 3 is a flowchart of training, verifying and testing an initial image enhancement model of a choledochoscope image enhancement method based on a residual transpose deconvolution neural network according to an embodiment of the present invention;
FIG. 4 is a flowchart of the cholangioscope image enhancement method based on the residual transpose deconvolution neural network according to the embodiment of the present invention;
FIG. 5 is an image enhancement flowchart of a choledochoscope image to be processed of the choledochoscope image enhancement method based on the residual transposition deconvolution neural network according to the embodiment of the present invention;
fig. 6 is a schematic diagram of intercepting biliary tract regions of the choledochoscope image enhancement method based on the residual transpose deconvolution neural network according to the embodiment of the present invention.
Detailed Description
The conception, the specific structure and the technical effects of the present invention will be further described with reference to the accompanying drawings to fully understand the objects, the features and the effects of the present invention.
Fig. 1 is a schematic structural diagram of a deconvolution residual transposed network model of a choledochoscope image enhancement method based on a residual transposed deconvolution neural network according to an embodiment of the present invention;
fig. 2 is a diagram of a jump connection processing procedure of a choledochoscope image enhancement method based on a residual transpose deconvolution neural network according to an embodiment of the present invention.
As shown in fig. 1 and fig. 2, the choledochoscope image enhancement method based on the residual transpose deconvolution neural network of the present invention adopts a deconvolution residual transpose network model to perform image enhancement on a choledochoscope image to be processed.
In this embodiment, the parameter settings of the deconvolution residual transposed network model are respectively as follows: the optimizer chooses Adam with beta1 set to 0.5, beta2 set to 0.9, batch size set to 5, and range of learning rates set to 1 × 10 -4 To 1 x 10 -5 Step size is set to 1 × 10 -5 。
The specific steps of constructing the deconvolution residual error transposed network model are as follows:
and step T1, constructing a deconvolution residual transposed network model according to the parameters to serve as an initial image enhancement model, and replacing the traditional convolution with deconvolution operation to realize the enhancement performance of the image.
In step T1, the initial image enhancement model includes a feature top layer and a feature bottom layer, and in this embodiment, the initial image enhancement model uses a convolution kernel of 3 × 3 as a deconvolution operator to collect feature information of the feature top layer and the feature bottom layer.
In step T1, a residual block of the initial image enhancement model is located between the feature top layer and the feature bottom layer, and the feature top layer and the feature bottom layer are combined through a skip connection, so that the residual block can be skipped when the skip connection combines the feature information of the feature top layer and the feature bottom layer.
Fig. 3 is a flowchart of training, verifying and testing an initial image enhancement model of a choledochoscope image enhancement method based on a residual transpose deconvolution neural network according to an embodiment of the present invention.
As shown in fig. 3, the initial image enhancement model is trained, validated, and tested after being constructed.
And T2, selecting a plurality of biliary tract endoscope images similar to the biliary tract endoscope image to be processed as a training image, a verification image and a test image.
Biliary tract endoscope images are selected from public image data sets such as Sintel, UST-HK, ESPL and the like, and each biliary tract endoscope image comprises two bit depths. In this embodiment, 5000 training pictures, 4000 verification pictures, and 1000 test images are used.
And T3, dividing the training image, the verification image and the test image into a low-bit image and a high-bit image corresponding to the low-bit image according to different bit depths, and taking the low-bit image and the high-bit image as a training set, a verification set and a test set of the initial image enhancement model.
In this embodiment, the images in the training set, the verification set, and the test set are classified into a low-bit number image and a high-bit number image according to the bit depths of the 8-bit class and the 16-bit class.
The training process of the final image enhancement model comprises the following steps:
and T4, inputting the training set into the initial image enhancement model for training to obtain the image enhancement model to be verified.
In step T4, the following substeps are included:
and step T4-1, segmenting the images in the training set to obtain image blocks with the size of 96 pixels multiplied by 96 pixels, so that the training time and memory requirements of an initial image enhancement model can be reduced, the choledochoscope images to be enhanced can be enhanced in a blocking mode when the model is applied, the resolution of the original choledochoscope images to be enhanced does not need to be changed, and image distortion can be avoided.
And T4-2, taking the segmentation image blocks of each low-bit image and the segmentation image blocks of the high-bit images corresponding to each low-bit image in the training set as a training subset.
And T4-3, sequentially inputting each training subset into the initial image enhancement model for training to obtain the image enhancement model to be verified.
The verification process of the final image enhancement model comprises the following steps:
and T5, inputting the verification set into the image enhancement model to be verified for verification, and obtaining a model verification result.
In step T5, the following substeps are included:
and step T5-1, dividing the image in the verification set to obtain an image block with the size of 96 pixels multiplied by 96 pixels, so that the time and memory requirements for the verification of the to-be-verified image enhancement model can be reduced.
And T5-2, taking the segmented image blocks of each low-bit image and the segmented image blocks of the high-bit images corresponding to each low-bit image in the verification set as a verification subset.
And T5-3, sequentially inputting each verification subset into the image enhancement model to be verified for verification, and obtaining a model verification result.
And T6, storing the image enhancement model to be verified with the optimal enhancement effect as a final image enhancement model according to the model verification result, thereby completing the construction of the deconvolution residual transposed network capable of enhancing the imaging quality of the choledochoscope image to be processed.
And the model verification result comprises the accuracy and the loss rate of the image enhancement model to be verified corresponding to each verification subset, and the image enhancement model to be verified corresponding to the moment with the highest accuracy and the lowest loss rate is selected as the final image enhancement model.
The testing process of the final image enhancement model comprises the following steps:
and T7, inputting the test set into the final image enhancement model for testing to obtain the actual enhancement effect and robustness of the final enhancement model.
In step T7, the following substeps are included:
and step T7-1, dividing the image in the test set to obtain an image block with the size of 96 pixels multiplied by 96 pixels, so that the time and memory requirements of the final image enhancement model test can be reduced.
And T7-2, taking the segmentation image block of each low-bit image and the segmentation image block of the high-bit image corresponding to each low-bit image in the test set as a test subset.
And T7-3, sequentially inputting each test subset into the final image enhancement model for testing, and obtaining the actual enhancement effect and robustness of the final enhancement model.
Fig. 4 is a flowchart of the cholangioscope image enhancement method based on the residual transpose deconvolution neural network according to the embodiment of the present invention;
FIG. 5 is an image enhancement flowchart of a choledochoscope image to be processed of the choledochoscope image enhancement method based on the residual transposition deconvolution neural network according to the embodiment of the present invention;
as shown in fig. 4 and 5, in this embodiment, the image enhancement is performed on the to-be-processed choledochoscope image by using the final image enhancement model, so as to enhance the imaging quality of the to-be-processed choledochoscope image, which specifically includes the following steps:
s1, performing motion blur removing processing and denoising processing on a choledochoscope image to be processed to obtain a choledochoscope image to be enhanced.
Because the choledochoscope image to be processed is imaged in a relatively unstable environment, a certain motion blur phenomenon exists in an imaging result, and meanwhile, the imaging process is in a human body, and an endoscope lens is inevitably influenced by body fluid and the like, so that a certain noise exists in the imaging result, in the step S1, before the choledochoscope image to be processed is processed, the choledochoscope image to be processed needs to be preprocessed, firstly, the motion blur removing processing is carried out on the choledochoscope image to be processed, and then, the image denoising algorithm is used for denoising the choledochoscope image to be processed, so that the influence of the motion blur phenomenon and the noise on an image enhancement result is eliminated.
And S2, inputting the choledochoscope image to be enhanced into the final image enhancement model after training for enhancement processing to obtain a final choledochoscope image, namely the choledochoscope enhanced image after the imaging quality is enhanced.
Fig. 6 is a schematic diagram of intercepting biliary tract regions of the choledochoscope image enhancement method based on the residual transpose deconvolution neural network according to the embodiment of the present invention.
As shown in fig. 6, in step S2, the following substeps are included:
and S2-1, selecting a biliary tract region on the choledochoscope image to be enhanced.
And S2-2, taking the center of the biliary tract area as an original point, taking 96 pixels as a step length for intercepting, intercepting the biliary tract area into a plurality of image blocks to be enhanced with the size of 96 pixels multiplied by 96 pixels, and eliminating the areas with the edge of the biliary tract area less than the size of 96 pixels, so that the processing efficiency of the choledochoscope image to be enhanced can be improved, and the enhancement of the choledochoscope image to be enhanced is more targeted and targeted.
And S2-3, inputting the image block to be enhanced into the final image enhancement model after training for enhancement processing, and obtaining a final choledochoscope image.
The image enhancement result of the final image enhancement model according to the present embodiment is compared with the image enhancement results of the prior art models such as laplacian pyramid filter and discrete cosine transform, and the comparison result is shown in table 1.
TABLE 1
According to the comparison result in table 1, the model 1 is used for enhancing the image through the laplacian pyramid filter, the model 2 is used for enhancing the image through discrete cosine transform, the peak signal-to-noise ratio and the structural similarity of the cholangioscope enhanced image after the final image enhancement model of the embodiment is enhanced are both improved to a greater extent, so that the cholangioscope enhanced image is enhanced more obviously, the contrast of the cholangioscope region and other regions is also greatly improved, and the biliary calculus operation is facilitated.
Examples effects and effects
According to the choledochoscope image enhancement method based on the residual error transposition deconvolution neural network, the deconvolution residual error transposition network model is constructed to serve as an initial image enhancement model, a plurality of choledochoscope images similar to the choledochoscope image to be processed are selected to serve as a training image and a verification image, the training image and the verification image are divided into a low-bit image and a high-bit image corresponding to the low-bit image according to different bit depths to serve as a training set and a verification set of the initial image enhancement model, the training set is input into the initial image enhancement model for training, the image enhancement model to be verified can be obtained, the verification set is input into the image enhancement model to be verified for verification, a model verification result can be obtained, the image enhancement model to be verified with the optimal enhancement effect can be stored according to the model verification result, and as a final image enhancement model, the image bit number of an input image can be increased through the final image enhancement model, so that the imaging quality of the image is improved, meanwhile, the choledochoscope image to be processed is subjected to motion blur removing processing and denoising processing, so that a choledochoscope image to be enhanced is obtained, the choledochoscope image to be enhanced is input into the final image enhancement model after training for enhancement, and a final choledochoscope image with the optimal enhancement effect can be obtained.
In this embodiment, after the final image enhancement model is determined, a test is performed, specifically, a plurality of cholangioscope images similar to the cholangioscope image to be processed are selected as test images, the test images are divided into a low-order image and a high-order image corresponding to the low-order image according to different bit depths, the low-order image and the high-order image are used as a test set of the final image enhancement model, and the test set is input into the final image enhancement model.
In this embodiment, since the initial image enhancement model uses a convolution kernel of 3 × 3 as a deconvolution operator, it is convenient to collect feature information of the top feature layer and the bottom feature layer of the initial image enhancement model, so that the image can be enhanced more purposefully.
In this embodiment, since the feature top layer and the feature bottom layer of the initial image enhancement model are combined through the jump connection, the residual block can be skipped when the feature information of the feature top layer and the feature bottom layer is combined through the jump connection, so that the neural network can be converged better, and the memory requirement can be reduced.
In this embodiment, since the images in the training set, the verification set, and the test set are all segmented to obtain image blocks of 96 pixels × 96 pixels, time and memory requirements for model training, verification, or testing can be reduced, and the choledochoscope image to be enhanced can be enhanced in blocks when the model is applied, so that the resolution of the original choledochoscope image to be enhanced does not need to be changed, and image distortion can be avoided.
In this embodiment, a cholangiographic region is selected from the cholangiographic image to be enhanced, the center of the cholangiographic region is used as an origin, the cholangiographic region is intercepted by using 96 pixels as a step length, the cholangiographic region is intercepted into a plurality of image blocks to be enhanced, which are 96 pixels × 96 pixels in size, and the image blocks to be enhanced are input into a final image enhancement model after training for enhancement, so that the processing efficiency of the cholangiographic image to be enhanced can be improved, and the cholangiographic image to be enhanced is enhanced more specifically and purposefully.
The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions available to those skilled in the art through logic analysis, reasoning and limited experiments based on the prior art according to the concept of the present invention should be within the scope of protection defined by the claims.
Claims (8)
1. A cholangioscope image enhancement method based on a residual transpose deconvolution neural network is used for enhancing the imaging quality of a cholangioscope image to be processed, and is characterized by comprising the following steps:
s1, performing motion blur removing processing and denoising processing on the choledochoscope image to be processed to obtain a choledochoscope image to be enhanced;
s2, inputting the choledochoscope image to be enhanced into a final image enhancement model after training for enhancement processing to obtain a final choledochoscope image;
the training process of the final image enhancement model comprises the following steps:
step T1, constructing a deconvolution residual error transposed network model as an initial image enhancement model;
t2, selecting a plurality of biliary tract endoscope images similar to the biliary tract endoscope image to be processed as a training image and a verification image;
step T3, dividing the training image and the verification image into a low-bit image and a high-bit image corresponding to the low-bit image according to different bit depths, and taking the low-bit image and the high-bit image as a training set and a verification set of an initial image enhancement model;
step T4, inputting the training set into the initial image enhancement model for training to obtain an image enhancement model to be verified;
step T5, inputting the verification set into the image enhancement model to be verified for verification to obtain a model verification result;
step T6, saving the image enhancement model to be verified with the optimal enhancement effect as the final image enhancement model according to the model verification result,
the initial image enhancement model comprises a feature top layer and a feature bottom layer,
the initial image enhancement model adopts a convolution kernel of 3 x 3 as a deconvolution operator for collecting the feature information of the feature top layer and the feature bottom layer,
between the feature top layer and the feature bottom layer is a residual block of the initial image enhancement model,
and the characteristic top layer and the characteristic bottom layer are combined through a jump connection, so that the jump connection skips the residual block when the characteristic information of the characteristic top layer and the characteristic bottom layer is combined.
2. The choledochoscope image enhancement method based on residual transpose deconvolution neural network of claim 1, characterized in that:
wherein the testing process of the final image enhancement model comprises the following steps:
a1, selecting a plurality of biliary tract endoscope images similar to the biliary tract endoscope image to be processed as test images;
step A2, dividing the test image into a low-bit image and a high-bit image corresponding to the low-bit image according to different bit depths, and taking the low-bit image and the high-bit image as a test set of the final image enhancement model;
and A3, inputting the test set into the final image enhancement model to obtain the actual enhancement effect and robustness of the final enhancement model.
3. The choledochoscope image enhancement method based on the residual transpose deconvolution neural network of claim 2, characterized in that:
and dividing the images in the training set, the verification set and the test set into a low-bit image and a high-bit image according to the bit depths of 8-bit classes and 16-bit classes.
4. The choledochoscope image enhancement method based on residual transpose deconvolution neural network of claim 1, characterized in that:
in step T1, the parameter settings of the initial image enhancement model are respectively: the optimizer chooses Adam with beta1 set to 0.5, beta2 set to 0.9, batch size set to 5, and range of learning rates set to 1 × 10 -4 To 1X 10 -5 Step size is set to 1 × 10 -5 。
5. The choledochoscope image enhancement method based on residual transpose deconvolution neural network of claim 1, characterized in that:
wherein, in step T4, the following substeps are included:
step T4-1, dividing the images in the training set to obtain image blocks with the size of 96 pixels multiplied by 96 pixels;
t4-2, taking the segmentation image blocks of each low-bit image and the segmentation image blocks of the high-bit images corresponding to each low-bit image in a training set as a training subset;
and T4-3, sequentially inputting each training subset into the initial image enhancement model for training to obtain the image enhancement model to be verified.
6. The choledochoscope image enhancement method based on residual transpose deconvolution neural network of claim 1, characterized in that:
wherein, in step T5, the following substeps are included:
step T5-1, dividing the image in the verification set to obtain an image block with the size of 96 pixels multiplied by 96 pixels;
t5-2, taking the segmented image blocks of each low-bit image and the segmented image blocks of the high-bit image corresponding to each low-bit image in a verification set as a verification subset;
and T5-3, sequentially inputting each verification subset into the image enhancement model to be verified for verification, and obtaining a model verification result.
7. The choledochoscope image enhancement method based on the residual transpose deconvolution neural network of claim 1, characterized in that:
wherein, in step S2, the following substeps are included:
s2-1, selecting a biliary tract area on the choledochoscope image to be enhanced,
s2-2, taking the center of the biliary tract area as an origin point, taking 96 pixels as step length to intercept, intercepting the biliary tract area into a plurality of image blocks to be enhanced with the size of 96 pixels multiplied by 96 pixels,
and S2-3, inputting the image blocks to be enhanced into a final image enhancement model after training for enhancement processing, and obtaining a final choledochoscope image.
8. The choledochoscope image enhancement method based on the residual transpose deconvolution neural network of claim 7, characterized in that:
in step S2-2, when the biliary tract region is extracted with 96 pixels as a step length, the region with the edge less than 96 pixels in size in the biliary tract region is removed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910793892.8A CN110648288B (en) | 2019-08-27 | 2019-08-27 | Choledochoscope image enhancement method based on residual transposition deconvolution neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910793892.8A CN110648288B (en) | 2019-08-27 | 2019-08-27 | Choledochoscope image enhancement method based on residual transposition deconvolution neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110648288A CN110648288A (en) | 2020-01-03 |
| CN110648288B true CN110648288B (en) | 2023-02-03 |
Family
ID=68990441
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910793892.8A Active CN110648288B (en) | 2019-08-27 | 2019-08-27 | Choledochoscope image enhancement method based on residual transposition deconvolution neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110648288B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105827971A (en) * | 2016-03-31 | 2016-08-03 | 维沃移动通信有限公司 | Image processing method and mobile terminal |
| CN106981053A (en) * | 2017-03-02 | 2017-07-25 | 天津大学 | A kind of underwater picture Enhancement Method based on Weighted Fusion |
| CN109285128A (en) * | 2018-09-06 | 2019-01-29 | 中国石油大学(华东) | A kind of cholelithiasis CT medical image data Enhancement Method based on deep learning |
| CN109544585A (en) * | 2018-12-19 | 2019-03-29 | 中国石油大学(华东) | A kind of cholelithiasis CT medical image data Enhancement Method based on lightweight convolutional neural networks |
| WO2019102476A2 (en) * | 2017-11-26 | 2019-05-31 | Yeda Research And Development Co. Ltd. | Signal enhancement and manipulation using a signal-specific deep network |
-
2019
- 2019-08-27 CN CN201910793892.8A patent/CN110648288B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105827971A (en) * | 2016-03-31 | 2016-08-03 | 维沃移动通信有限公司 | Image processing method and mobile terminal |
| CN106981053A (en) * | 2017-03-02 | 2017-07-25 | 天津大学 | A kind of underwater picture Enhancement Method based on Weighted Fusion |
| WO2019102476A2 (en) * | 2017-11-26 | 2019-05-31 | Yeda Research And Development Co. Ltd. | Signal enhancement and manipulation using a signal-specific deep network |
| CN109285128A (en) * | 2018-09-06 | 2019-01-29 | 中国石油大学(华东) | A kind of cholelithiasis CT medical image data Enhancement Method based on deep learning |
| CN109544585A (en) * | 2018-12-19 | 2019-03-29 | 中国石油大学(华东) | A kind of cholelithiasis CT medical image data Enhancement Method based on lightweight convolutional neural networks |
Non-Patent Citations (1)
| Title |
|---|
| 基于卷积神经网络的图像分类研究;葛程等;《软件导刊》;20180717(第10期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110648288A (en) | 2020-01-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhang et al. | A fast medical image super resolution method based on deep learning network | |
| Lefkimmiatis | Universal denoising networks: a novel CNN architecture for image denoising | |
| CN110827216B (en) | Multi-generator generation countermeasure network learning method for image denoising | |
| CN110706166B (en) | Image super-resolution reconstruction method and device for sharpening label data | |
| Raj et al. | Computer aided detection of brain tumor in magnetic resonance images | |
| CN109949235A (en) | A kind of chest x-ray piece denoising method based on depth convolutional neural networks | |
| CN106886977A (en) | A kind of many figure autoregistrations and anastomosing and splicing method | |
| WO2021136368A1 (en) | Method and apparatus for automatically detecting pectoralis major region in molybdenum target image | |
| CN113436076B (en) | Image super-resolution reconstruction method with characteristics gradually fused and electronic equipment | |
| CN111626927B (en) | Binocular image super-resolution method, system and device adopting parallax constraint | |
| CN111178499B (en) | Medical image super-resolution method based on generation countermeasure network improvement | |
| CN106157249A (en) | Based on the embedded single image super-resolution rebuilding algorithm of optical flow method and sparse neighborhood | |
| KR20110049570A (en) | Image enhancement method using neural network model based on edge component classification | |
| CN114240797B (en) | OCT image denoising method, device, equipment and medium | |
| CN105654425A (en) | Single-image super-resolution reconstruction method applied to medical X-ray image | |
| Jiang et al. | CT image super resolution based on improved SRGAN | |
| Liu et al. | Facial image inpainting using multi-level generative network | |
| CN110648288B (en) | Choledochoscope image enhancement method based on residual transposition deconvolution neural network | |
| CN114283058A (en) | Image super-resolution reconstruction method based on countermeasure network and maximum mutual information optimization | |
| CN112819739A (en) | Scanning electron microscope image processing method and system | |
| CN111325700A (en) | Multi-dimensional fusion algorithm and system based on color images | |
| Prakoso et al. | Enhancement methods of brain MRI images: A Review | |
| CN115797176A (en) | Image super-resolution reconstruction method | |
| Tian et al. | Retinal fundus image superresolution generated by optical coherence tomography based on a realistic mixed attention GAN | |
| Wen et al. | Do noises bother human and neural networks in the same way? a medical image analysis perspective |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |