CN110648288A - Choledochoscope image enhancement method based on residual transposition deconvolution neural network - Google Patents
Choledochoscope image enhancement method based on residual transposition deconvolution neural network Download PDFInfo
- Publication number
- CN110648288A CN110648288A CN201910793892.8A CN201910793892A CN110648288A CN 110648288 A CN110648288 A CN 110648288A CN 201910793892 A CN201910793892 A CN 201910793892A CN 110648288 A CN110648288 A CN 110648288A
- Authority
- CN
- China
- Prior art keywords
- image
- choledochoscope
- image enhancement
- bit
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 40
- 230000017105 transposition Effects 0.000 title claims description 16
- 238000012549 training Methods 0.000 claims abstract description 66
- 238000012545 processing Methods 0.000 claims abstract description 25
- 238000003384 imaging method Methods 0.000 claims abstract description 18
- 230000002708 enhancing effect Effects 0.000 claims abstract description 6
- 238000012795 verification Methods 0.000 claims description 63
- 238000012360 testing method Methods 0.000 claims description 35
- 210000003445 biliary tract Anatomy 0.000 claims description 27
- 230000000694 effects Effects 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 8
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 7
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 206010004637 Bile duct stone Diseases 0.000 abstract description 2
- 201000009331 Choledocholithiasis Diseases 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 6
- 238000002350 laparotomy Methods 0.000 description 3
- HVYWMOMLDIMFJA-DPAQBDIFSA-N cholesterol Chemical compound C1C=C2C[C@@H](O)CC[C@]2(C)[C@@H]2[C@@H]1[C@@H]1CC[C@H]([C@H](C)CCCC(C)C)[C@@]1(C)CC2 HVYWMOMLDIMFJA-DPAQBDIFSA-N 0.000 description 2
- 208000001130 gallstones Diseases 0.000 description 2
- 230000009191 jumping Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 210000001835 viscera Anatomy 0.000 description 2
- 210000000683 abdominal cavity Anatomy 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 210000000941 bile Anatomy 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 210000001124 body fluid Anatomy 0.000 description 1
- 239000010839 body fluid Substances 0.000 description 1
- 235000012000 cholesterol Nutrition 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000002503 metabolic effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 231100000915 pathological change Toxicity 0.000 description 1
- 230000036285 pathological change Effects 0.000 description 1
- 244000144985 peep Species 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000028327 secretion Effects 0.000 description 1
- 239000004575 stone Substances 0.000 description 1
Images
Classifications
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Abstract
The invention provides a cholangioscope image enhancement method based on a residual transpose deconvolution neural network, which is used for enhancing the imaging quality of a cholangioscope image to be processed and comprises the following steps: carrying out motion blur removing processing and denoising processing on the choledochoscope image to be processed to obtain a choledochoscope image to be enhanced; and inputting the choledochoscope image to be enhanced into a final image enhancement model after training for enhancement processing to obtain a final choledochoscope image. The final image enhancement model is trained by a training method of matching the low-digit image with the high-digit image, and after the training is finished, the image digit of the choledochoscope image to be processed can be increased by the final image enhancement model, so that the imaging quality of the choledochoscope image to be processed is further enhanced on the basis of the prior art, and the success rate of the choledocholithiasis operation can be further improved.
Description
Technical Field
The invention belongs to computer-assisted medicine, relates to a choledochoscope image enhancement method, and particularly relates to a choledochoscope image enhancement method based on a residual transposition deconvolution neural network.
Background
Biliary calculi are clinically a common disease type, and the causes of the biliary calculi are many, mainly caused by the metabolic disturbance of cholesterol and the reduction of bile secretion of patients. At present, the main clinical treatment mode for the biliary tract calculus is a choledochoscope combined laparoscopic laparotomy.
The choledochoscope has high image quality and a working channel for operation, can greatly improve the treatment effect of a patient, has less interference to the abdominal cavity and little damage, can directly see the visceral organs under the stable condition of keeping the body environment, is favorable for knowing the shape, the size, the peripheral adhesion condition and the like of the visceral organs, and can accurately and timely judge the degree and the size of pathological changes of various organs. Through the combined laparoscopic laparotomy of choledochoscope, can reduce the operation wound, reduce the unnecessary inspection, and can directly peep and get the stone through the basket, the security obviously improves. In the choledochoscope combined laparoscopic laparotomy, the imaging quality of a choledochoscope image directly influences the success rate of the operation.
However, at present, the research for improving the imaging quality of the choledochoscope image is applied to the performance improvement of hardware facilities such as an endoscope lens, and the research for further improving the imaging quality of the choledochoscope image through an enhancement model on the existing imaging result is very little, so that the enhancement effect of the imaging quality of the choledochoscope image is insufficient, and the success rate of the choledocholithiasis operation is affected.
Disclosure of Invention
In order to solve the problems, the invention provides an image enhancement method capable of further improving the imaging quality of a choledochoscope image on the existing imaging result, and adopts the following technical scheme:
the invention provides a choledochoscope image enhancement method based on a residual error transposition deconvolution neural network, which is used for enhancing the imaging quality of a choledochoscope image to be processed and is characterized by comprising the following steps:
step S1, performing motion blur removing processing and denoising processing on the choledochoscope image to be processed to obtain a choledochoscope image to be enhanced;
step S2, inputting the choledochoscope image to be enhanced into the final image enhancement model after training for enhancement processing, and obtaining the final choledochoscope image;
the final training process of the image enhancement model comprises the following steps:
step T1, constructing a deconvolution residual error transposition network model as an initial image enhancement model;
step T2, selecting a plurality of biliary tract endoscope images similar to the biliary tract endoscope image to be processed as a training image and a verification image;
step T3, dividing the training image and the verification image into a low-bit image and a high-bit image corresponding to the low-bit image according to different bit depths, and taking the low-bit image and the high-bit image as a training set and a verification set of the initial image enhancement model;
step T4, inputting the training set into the initial image enhancement model for training to obtain an image enhancement model to be verified;
step T5, inputting the verification set into the image enhancement model to be verified for verification, and obtaining a model verification result;
and step T6, saving the image enhancement model to be verified with the optimal enhancement effect as the final image enhancement model according to the model verification result.
The choledochoscope image enhancement method based on the residual error transposition deconvolution neural network provided by the invention can also have the technical characteristics, and the test process of the final image enhancement model comprises the following steps:
step A1, selecting a plurality of biliary tract endoscope images similar to the biliary tract endoscope image to be processed as test images;
step A2, dividing the test image into a low-bit image and a high-bit image corresponding to the low-bit image according to different bit depths, and taking the low-bit image and the high-bit image as a test set of a final image enhancement model;
and step A3, inputting the test set into the final image enhancement model to obtain the actual enhancement effect and robustness of the final enhancement model.
The choledochoscope image enhancement method based on the residual error transposition deconvolution neural network, provided by the invention, can also have the technical characteristics that the images in the training set, the verification set and the test set are divided into a low-bit image and a high-bit image according to the bit depths of 8-bit class and 16-bit class.
The choledochoscope image enhancement method based on the residual transpose deconvolution neural network provided by the invention can also have the technical characteristics that in step T1, the initial image enhancement model comprises a characteristic top layer and a characteristic bottom layer, and the initial image enhancement model adopts a convolution kernel of 3 x 3 as a deconvolution operator for collecting characteristic information of the characteristic top layer and the characteristic bottom layer.
The choledochoscope image enhancement method based on the residual transpose deconvolution neural network provided by the invention can also have the technical characteristics that in step T1, a residual block of an initial image enhancement model is arranged between the characteristic top layer and the characteristic bottom layer, and the characteristic top layer and the characteristic bottom layer are combined through jump connection, so that the residual block is skipped when the jump connection is combined with the characteristic information of the characteristic top layer and the characteristic bottom layer.
The choledochoscope image enhancement method based on the residual transpose deconvolution neural network provided by the invention can also have the technical characteristics that in the step T1, the parameter settings of the initial image enhancement model are respectively as follows: the optimizer chooses Adam with beta1 set to 0.5, beta2 set to 0.9, batch size set to 5, and range of learning rates set to 1 × 10-4To 1X 10-5Step size is set to 1 × 10-5。
The choledochoscope image enhancement method based on the residual transpose deconvolution neural network provided by the invention can also have the technical characteristics that in the step T4, the method comprises the following substeps:
step T4-1, dividing the image in the training set to obtain an image block with the size of 96 pixels multiplied by 96 pixels;
step T4-2, taking the segmentation image blocks of each low-bit image and the segmentation image blocks of the high-bit images corresponding to each low-bit image in the training set as a training subset;
and step T4-3, sequentially inputting each training subset into the initial image enhancement model for training to obtain the image enhancement model to be verified.
The choledochoscope image enhancement method based on the residual transpose deconvolution neural network provided by the invention can also have the technical characteristics that in the step T5, the method comprises the following substeps:
step T5-1, dividing the image in the verification set to obtain an image block with the size of 96 pixels multiplied by 96 pixels;
step T5-2, taking the divided image blocks of each low-bit image and the divided image blocks of the high-bit images corresponding to each low-bit image in the verification set as a verification subset;
and step T5-3, sequentially inputting each verification subset into the image enhancement model to be verified for verification, and obtaining a model verification result.
The cholangioscope image enhancement method based on the residual transpose deconvolution neural network provided by the invention can also have the technical characteristics that in the step S2, the cholangioscope image enhancement method comprises the following substeps:
step S2-1, selecting biliary tract area on the choledochoscope image to be enhanced,
step S2-2, taking the center of the biliary tract area as the origin, taking 96 pixels as the step size to intercept, intercepting the biliary tract area into a plurality of image blocks to be enhanced with the size of 96 pixels multiplied by 96 pixels,
and step S2-3, inputting the image blocks to be enhanced into the final image enhancement model after training for enhancement processing, and obtaining the final choledochoscope image.
The choledochoscope image enhancement method based on the residual error transposition deconvolution neural network, provided by the invention, can also have the technical characteristic that in the step S2-2, when a biliary tract area is intercepted by taking 96 pixels as a step length, an area with the edge less than 96 pixels in size in the biliary tract area is removed.
Action and Effect of the invention
According to the choledochoscope image enhancement method based on the residual error transposition deconvolution neural network, as a deconvolution residual error transposition network model is constructed to be used as an initial image enhancement model, a plurality of choledochoscope images similar to the choledochoscope image to be processed are selected to be used as a training image and a verification image, the training image and the verification image are divided into a low-bit image and a high-bit image corresponding to the low-bit image according to different bit depths to be used as a training set and a verification set of the initial image enhancement model, the training set is input into the initial image enhancement model for training, the image enhancement model to be verified can be obtained, the verification set is input into the image enhancement model to be verified for verification, a model verification result can be obtained, the image enhancement model to be verified with the optimal enhancement effect can be stored according to the model verification result to be used as a final image enhancement model, therefore, compared with the existing Laplacian pyramid filter, discrete cosine transform and other models, the final image enhancement model is trained by a training method of matching the low-bit image and the high-bit image, and the final image enhancement model after training can enhance the images in multiple aspects and at multiple angles, so that the image characteristics can be displayed more accurately.
Furthermore, the choledochoscope image to be processed is subjected to motion blur removing processing and denoising processing to obtain a choledochoscope image to be enhanced, and then the choledochoscope image to be enhanced is input into a final image enhancement model after training is completed to be enhanced, so that a final choledochoscope image with the optimal enhancement effect can be obtained.
Drawings
Fig. 1 is a schematic structural diagram of a deconvolution residual transposed network model of a choledochoscope image enhancement method based on a residual transposed deconvolution neural network according to an embodiment of the present invention;
FIG. 2 is a diagram of a jump connection processing procedure of a choledochoscope image enhancement method based on a residual transpose deconvolution neural network according to an embodiment of the present invention;
FIG. 3 is a flowchart of training, verifying and testing an initial image enhancement model of a choledochoscope image enhancement method based on a residual transpose deconvolution neural network according to an embodiment of the present invention;
FIG. 4 is a flowchart of the cholangioscope image enhancement method based on the residual transpose deconvolution neural network according to the embodiment of the present invention;
FIG. 5 is an image enhancement flowchart of a choledochoscope image to be processed of the choledochoscope image enhancement method based on the residual transposition deconvolution neural network according to the embodiment of the present invention;
fig. 6 is a schematic diagram of intercepting biliary tract regions of the choledochoscope image enhancement method based on the residual transpose deconvolution neural network according to the embodiment of the present invention.
Detailed Description
The conception, the specific structure and the technical effects of the present invention will be further described with reference to the accompanying drawings to fully understand the objects, the features and the effects of the present invention.
Fig. 1 is a schematic structural diagram of a deconvolution residual transposed network model of a choledochoscope image enhancement method based on a residual transposed deconvolution neural network according to an embodiment of the present invention;
fig. 2 is a diagram of a jump connection processing procedure of a choledochoscope image enhancement method based on a residual transpose deconvolution neural network according to an embodiment of the present invention.
As shown in fig. 1 and fig. 2, the choledochoscope image enhancement method based on the residual transpose deconvolution neural network of the present invention adopts a deconvolution residual transpose network model to perform image enhancement on a choledochoscope image to be processed.
In this embodiment, the parameter settings of the deconvolution residual transposed network model are respectively as follows: the optimizer chooses Adam with beta1 set to 0.5, beta2 set to 0.9, batch size set to 5, and range of learning rates set to 1 × 10-4To 1X 10-5Step size is set to 1 × 10-5。
The specific steps of constructing the deconvolution residual error transposed network model are as follows:
and step T1, constructing a deconvolution residual error transposed network model according to the parameters to serve as an initial image enhancement model, and replacing the traditional convolution with the deconvolution operation to realize the enhancement performance of the image.
In step T1, the initial image enhancement model includes a feature top layer and a feature bottom layer, and in this embodiment, the initial image enhancement model uses a3 × 3 convolution kernel as a deconvolution operator to collect feature information of the feature top layer and the feature bottom layer.
In step T1, a residual block of the initial image enhancement model is located between the feature top layer and the feature bottom layer, and the feature top layer and the feature bottom layer are combined through a skip connection, so that the skip connection can skip the residual block when combining the feature information of the feature top layer and the feature bottom layer.
Fig. 3 is a flowchart of training, verifying and testing an initial image enhancement model of a choledochoscope image enhancement method based on a residual transpose deconvolution neural network according to an embodiment of the present invention.
As shown in fig. 3, the initial image enhancement model is trained, validated, and tested after being constructed.
And step T2, selecting a plurality of biliary tract endoscope images similar to the biliary tract endoscope image to be processed as a training image, a verification image and a test image.
Biliary tract endoscopic images are selected from public image data sets such as Sintel, UST-HK, ESPL, etc., and each of these biliary tract endoscopic images includes two bit depths. In this embodiment, 5000 training pictures, 4000 verification pictures, and 1000 test images are used.
And step T3, dividing the training image, the verification image and the test image into a low-bit image and a high-bit image corresponding to the low-bit image according to different bit depths, and taking the low-bit image and the high-bit image as a training set, a verification set and a test set of the initial image enhancement model.
In the present embodiment, the images in the training set, the verification set, and the test set are classified into a low-bit number image and a high-bit number image according to the bit depths of 8-bit class and 16-bit class.
The training process of the final image enhancement model comprises the following steps:
and step T4, inputting the training set into the initial image enhancement model for training to obtain the image enhancement model to be verified.
In step T4, the following sub-steps are included:
and step T4-1, images in the training set are segmented, and image blocks with the size of 96 pixels multiplied by 96 pixels are obtained, so that the training time and memory requirements of an initial image enhancement model can be reduced, the choledochoscope image to be enhanced can be enhanced in a blocking mode when the model is applied, the resolution of the original choledochoscope image to be enhanced does not need to be changed, and image distortion can be avoided.
And step T4-2, taking the segmented image blocks of each low-bit image and the segmented image blocks of the high-bit images corresponding to each low-bit image in the training set as a training subset.
And step T4-3, sequentially inputting each training subset into the initial image enhancement model for training to obtain the image enhancement model to be verified.
The verification process of the final image enhancement model comprises the following steps:
and step T5, inputting the verification set into the image enhancement model to be verified for verification, and obtaining a model verification result.
In step T5, the following sub-steps are included:
and step T5-1, dividing the image in the verification set to obtain an image block with the size of 96 pixels multiplied by 96 pixels, thereby reducing the time and memory requirements for the verification of the to-be-verified image enhancement model.
And step T5-2, taking the divided image blocks of each low-bit image and the divided image blocks of the high-bit images corresponding to each low-bit image in the verification set as a verification subset.
And step T5-3, sequentially inputting each verification subset into the image enhancement model to be verified for verification, and obtaining a model verification result.
And step T6, saving the image enhancement model to be verified with the optimal enhancement effect as a final image enhancement model according to the model verification result, thereby completing the construction of the deconvolution residual error transposition network capable of enhancing the imaging quality of the choledochoscope image to be processed.
And the model verification result comprises the accuracy and the loss rate of the image enhancement model to be verified corresponding to each verification subset, and the image enhancement model to be verified corresponding to the moment with the highest accuracy and the lowest loss rate is selected as the final image enhancement model.
The testing process of the final image enhancement model comprises the following steps:
and step T7, inputting the test set into the final image enhancement model for testing, and acquiring the actual enhancement effect and robustness of the final enhancement model.
In step T7, the following sub-steps are included:
and step T7-1, dividing the image in the test set to obtain an image block with the size of 96 pixels multiplied by 96 pixels, thereby reducing the time and memory requirements of the final image enhancement model test.
And step T7-2, taking the divided image blocks of each low-bit image and the divided image blocks of the high-bit images corresponding to each low-bit image in the test set as a test subset.
And step T7-3, sequentially inputting each test subset into the final image enhancement model for testing, and obtaining the actual enhancement effect and robustness of the final enhancement model.
FIG. 4 is a flowchart of the cholangioscope image enhancement method based on the residual transpose deconvolution neural network according to the embodiment of the present invention;
FIG. 5 is an image enhancement flowchart of a choledochoscope image to be processed of the choledochoscope image enhancement method based on the residual transposition deconvolution neural network according to the embodiment of the present invention;
as shown in fig. 4 and 5, in this embodiment, the image enhancement is performed on the to-be-processed choledochoscope image by using the final image enhancement model, so as to enhance the imaging quality of the to-be-processed choledochoscope image, which specifically includes the following steps:
and step S1, performing motion blur removing processing and denoising processing on the choledochoscope image to be processed to obtain a choledochoscope image to be enhanced.
Because the choledochoscope image to be processed is imaged in a relatively unstable environment, a certain motion blur phenomenon exists in an imaging result, and meanwhile, the imaging process is in a human body, and an endoscope lens is inevitably influenced by body fluid and the like, so that a certain noise exists in the imaging result, in step S1, before the choledochoscope image to be processed is processed, the choledochoscope image to be processed needs to be preprocessed, firstly, the motion blur removing processing is carried out on the choledochoscope image to be processed, and then, the image denoising algorithm is used for denoising the choledochoscope image to be processed, so that the influence of the motion blur phenomenon and the noise on an image enhancement result is eliminated.
And step S2, inputting the choledochoscope image to be enhanced into the final image enhancement model after training for enhancement processing, and obtaining the final choledochoscope image, namely the choledochoscope enhanced image after the imaging quality is enhanced.
Fig. 6 is a schematic diagram of intercepting biliary tract regions of the choledochoscope image enhancement method based on the residual transpose deconvolution neural network according to the embodiment of the present invention.
As shown in fig. 6, in step S2, the following sub-steps are included:
step S2-1, select the biliary tract area on the choledochoscope image to be enhanced.
And step S2-2, taking the center of the biliary tract area as an original point, taking 96 pixels as a step length for intercepting, intercepting the biliary tract area into a plurality of image blocks to be enhanced with the size of 96 pixels multiplied by 96 pixels, and eliminating the areas with the edge of the biliary tract area less than 96 pixels, so that the processing efficiency of the choledochoscope image to be enhanced can be improved, and the enhancement of the choledochoscope image to be enhanced is more targeted and targeted.
And step S2-3, inputting the image blocks to be enhanced into the final image enhancement model after training for enhancement processing, and obtaining the final choledochoscope image.
The image enhancement result of the final image enhancement model according to the present embodiment is compared with the image enhancement results of the prior art models such as laplacian pyramid filter and discrete cosine transform, and the comparison result is shown in table 1.
TABLE 1
According to the comparison result in table 1, the model 1 is used for enhancing the image through the laplacian pyramid filter, the model 2 is used for enhancing the image through discrete cosine transform, the peak signal-to-noise ratio and the structural similarity of the cholangioscope enhanced image after the final image enhancement model of the embodiment is enhanced are both improved to a greater extent, so that the cholangioscope enhanced image is enhanced more obviously, the contrast of the cholangioscope region and other regions is also greatly improved, and the biliary calculus operation is facilitated.
Examples effects and effects
According to the choledochoscope image enhancement method based on the residual error transposition deconvolution neural network, as a deconvolution residual error transposition network model is constructed to be used as an initial image enhancement model, a plurality of choledochoscope images similar to the choledochoscope image to be processed are selected to be used as a training image and a verification image, the training image and the verification image are divided into a low-bit image and a high-bit image corresponding to the low-bit image according to different bit depths to be used as a training set and a verification set of the initial image enhancement model, the training set is input into the initial image enhancement model for training, the image enhancement model to be verified can be obtained, the verification set is input into the image enhancement model to be verified for verification, a model verification result can be obtained, the image enhancement model to be verified with the optimal enhancement effect can be stored according to the model verification result to be used as a final image enhancement model, through the final image enhancement model, the image bit number of an input image can be improved, so that the imaging quality of the image is improved, meanwhile, the choledochoscope image to be processed is subjected to motion blur removing processing and denoising processing, the choledochoscope image to be enhanced is obtained, the choledochoscope image to be enhanced is input into the final image enhancement model after training is completed for enhancement processing, and the final choledochoscope image with the optimal enhancement effect can be obtained.
In this embodiment, after the final image enhancement model is determined, a test is performed, specifically, a plurality of cholangioscope images similar to the cholangioscope image to be processed are selected as test images, the test images are divided into a low-order image and a high-order image corresponding to the low-order image according to different bit depths, the low-order image and the high-order image are used as a test set of the final image enhancement model, and the test set is input into the final image enhancement model.
In this embodiment, since the initial image enhancement model uses a convolution kernel of 3 × 3 as a deconvolution operator, it is convenient to collect feature information of the top feature layer and the bottom feature layer of the initial image enhancement model, so that the image can be enhanced more purposefully.
In this embodiment, the feature top layer and the feature bottom layer of the initial image enhancement model are combined through jumping connection, so that the residual block can be skipped when the feature information of the feature top layer and the feature bottom layer is combined through jumping connection, thereby enabling the neural network to better converge, and reducing the memory requirement.
In this embodiment, since the images in the training set, the verification set, and the test set are all segmented to obtain image blocks of 96 pixels × 96 pixels, time and memory requirements for model training, verification, or testing can be reduced, and the choledochoscope image to be enhanced can be enhanced in a block-wise manner when the model is applied, so that the resolution of the original choledochoscope image to be enhanced does not need to be changed, and image distortion can be avoided.
In this embodiment, a cholangiographic region is selected from the cholangiographic image to be enhanced, the center of the cholangiographic region is used as an origin, the cholangiographic region is intercepted by using 96 pixels as a step length, the cholangiographic region is intercepted into a plurality of image blocks to be enhanced, which are 96 pixels × 96 pixels in size, and the image blocks to be enhanced are input into a final image enhancement model after training for enhancement, so that the processing efficiency of the cholangiographic image to be enhanced can be improved, and the cholangiographic image to be enhanced is enhanced more specifically and purposefully.
The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions available to those skilled in the art through logic analysis, reasoning and limited experiments based on the prior art according to the concept of the present invention should be within the scope of protection defined by the claims.
Claims (10)
1. A cholangioscope image enhancement method based on a residual transpose deconvolution neural network is used for enhancing the imaging quality of a cholangioscope image to be processed, and is characterized by comprising the following steps:
step S1, performing motion blur removing processing and denoising processing on the choledochoscope image to be processed to obtain a choledochoscope image to be enhanced;
step S2, inputting the choledochoscope image to be enhanced into a final image enhancement model after training for enhancement processing to obtain a final choledochoscope image;
the training process of the final image enhancement model comprises the following steps:
step T1, constructing a deconvolution residual error transposition network model as an initial image enhancement model;
step T2, selecting a plurality of biliary tract endoscope images similar to the biliary tract endoscope image to be processed as a training image and a verification image;
step T3, dividing the training image and the verification image into a low-bit image and a high-bit image corresponding to the low-bit image according to different bit depths, and taking the low-bit image and the high-bit image as a training set and a verification set of an initial image enhancement model;
step T4, inputting the training set into the initial image enhancement model for training to obtain an image enhancement model to be verified;
step T5, inputting the verification set into the image enhancement model to be verified for verification, and obtaining a model verification result;
and T6, saving the image enhancement model to be verified with the optimal enhancement effect as the final image enhancement model according to the model verification result.
2. The choledochoscope image enhancement method based on residual transpose deconvolution neural network of claim 1, characterized in that:
wherein the testing process of the final image enhancement model comprises the following steps:
step A1, selecting a plurality of biliary tract endoscope images similar to the choledochoscope image to be processed as test images;
step A2, dividing the test image into a low-bit image and a high-bit image corresponding to the low-bit image according to different bit depths, and taking the low-bit image and the high-bit image as a test set of the final image enhancement model;
and step A3, inputting the test set into the final image enhancement model, and acquiring the actual enhancement effect and robustness of the final enhancement model.
3. The cholangioscope image enhancement method based on residual transpose deconvolution neural network of claim 2, characterized in that:
and dividing the images in the training set, the verification set and the test set into a low-bit image and a high-bit image according to the bit depths of 8-bit classes and 16-bit classes.
4. The choledochoscope image enhancement method based on residual transpose deconvolution neural network of claim 1, characterized in that:
wherein, in step T1, the initial image enhancement model includes a feature top layer and a feature bottom layer,
and the initial image enhancement model adopts a convolution kernel of 3 multiplied by 3 as a deconvolution operator and is used for collecting the characteristic information of the characteristic top layer and the characteristic bottom layer.
5. The choledochoscope image enhancement method based on the residual transpose deconvolution neural network of claim 4, characterized in that:
wherein, in step T1, a residual block of the initial image enhancement model is between the feature top layer and the feature bottom layer,
and the characteristic top layer and the characteristic bottom layer are combined through a jump connection, so that the jump connection skips the residual block when the characteristic information of the characteristic top layer and the characteristic bottom layer is combined.
6. The choledochoscope image enhancement method based on residual transpose deconvolution neural network of claim 1, characterized in that:
in step T1, the parameter settings of the initial image enhancement model are respectively: the optimizer chooses Adam with beta1 set to 0.5, beta2 set to 0.9, batch size set to 5, and range of learning rates set to 1 × 10-4To 1X 10-5Step size is set to 1 × 10-5。
7. The choledochoscope image enhancement method based on residual transpose deconvolution neural network of claim 1, characterized in that:
in step T4, the following sub-steps are included:
step T4-1, dividing the images in the training set to obtain image blocks with the size of 96 pixels multiplied by 96 pixels;
step T4-2, taking the segmented image blocks of each low-bit image and the segmented image blocks of the high-bit images corresponding to each low-bit image in the training set as a training subset;
and T4-3, sequentially inputting each training subset into the initial image enhancement model for training to obtain the image enhancement model to be verified.
8. The choledochoscope image enhancement method based on residual transpose deconvolution neural network of claim 1, characterized in that:
in step T5, the following sub-steps are included:
step T5-1, dividing the image in the verification set to obtain an image block with the size of 96 pixels multiplied by 96 pixels;
step T5-2, taking the divided image blocks of each low-bit number image and the divided image blocks of the high-bit number image corresponding to each low-bit number image in the verification set as a verification subset;
and T5-3, sequentially inputting each verification subset into the to-be-verified image enhancement model for verification, and obtaining a model verification result.
9. The choledochoscope image enhancement method based on residual transpose deconvolution neural network of claim 1, characterized in that:
in step S2, the method includes the following substeps:
step S2-1, selecting a biliary tract region on the choledochoscope image to be enhanced,
step S2-2, taking the center of the biliary tract area as the origin, taking 96 pixels as the step size to intercept, taking the biliary tract area as a plurality of image blocks to be enhanced with the size of 96 pixels multiplied by 96 pixels,
and step S2-3, inputting the image blocks to be enhanced into the final image enhancement model after training for enhancement processing, and obtaining the final choledochoscope image.
10. The cholangioscope image enhancement method based on residual transpose deconvolution neural network of claim 9, characterized in that:
in step S2-2, when the biliary tract region is extracted with 96 pixels as a step, the region with the edge less than 96 pixels in size is removed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910793892.8A CN110648288B (en) | 2019-08-27 | 2019-08-27 | Choledochoscope image enhancement method based on residual transposition deconvolution neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910793892.8A CN110648288B (en) | 2019-08-27 | 2019-08-27 | Choledochoscope image enhancement method based on residual transposition deconvolution neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110648288A true CN110648288A (en) | 2020-01-03 |
| CN110648288B CN110648288B (en) | 2023-02-03 |
Family
ID=68990441
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910793892.8A Active CN110648288B (en) | 2019-08-27 | 2019-08-27 | Choledochoscope image enhancement method based on residual transposition deconvolution neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110648288B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105827971A (en) * | 2016-03-31 | 2016-08-03 | 维沃移动通信有限公司 | Image processing method and mobile terminal |
| CN106981053A (en) * | 2017-03-02 | 2017-07-25 | 天津大学 | A kind of underwater picture Enhancement Method based on Weighted Fusion |
| CN109285128A (en) * | 2018-09-06 | 2019-01-29 | 中国石油大学(华东) | A kind of cholelithiasis CT medical image data Enhancement Method based on deep learning |
| CN109544585A (en) * | 2018-12-19 | 2019-03-29 | 中国石油大学(华东) | A kind of cholelithiasis CT medical image data Enhancement Method based on lightweight convolutional neural networks |
| WO2019102476A2 (en) * | 2017-11-26 | 2019-05-31 | Yeda Research And Development Co. Ltd. | Signal enhancement and manipulation using a signal-specific deep network |
-
2019
- 2019-08-27 CN CN201910793892.8A patent/CN110648288B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105827971A (en) * | 2016-03-31 | 2016-08-03 | 维沃移动通信有限公司 | Image processing method and mobile terminal |
| CN106981053A (en) * | 2017-03-02 | 2017-07-25 | 天津大学 | A kind of underwater picture Enhancement Method based on Weighted Fusion |
| WO2019102476A2 (en) * | 2017-11-26 | 2019-05-31 | Yeda Research And Development Co. Ltd. | Signal enhancement and manipulation using a signal-specific deep network |
| CN109285128A (en) * | 2018-09-06 | 2019-01-29 | 中国石油大学(华东) | A kind of cholelithiasis CT medical image data Enhancement Method based on deep learning |
| CN109544585A (en) * | 2018-12-19 | 2019-03-29 | 中国石油大学(华东) | A kind of cholelithiasis CT medical image data Enhancement Method based on lightweight convolutional neural networks |
Non-Patent Citations (1)
| Title |
|---|
| 葛程等: "基于卷积神经网络的图像分类研究", 《软件导刊》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110648288B (en) | 2023-02-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhang et al. | A fast medical image super resolution method based on deep learning network | |
| CN110827216B (en) | Multi-generator generation countermeasure network learning method for image denoising | |
| US20220207728A1 (en) | Quality assessment in video endoscopy | |
| Raj et al. | Computer aided detection of brain tumor in magnetic resonance images | |
| CN106886977A (en) | A kind of many figure autoregistrations and anastomosing and splicing method | |
| CN109949235A (en) | A kind of chest x-ray piece denoising method based on depth convolutional neural networks | |
| KR101070981B1 (en) | Image Enhancement Method Using Neural Network Model Based on Edge Component Classification | |
| CN106157249A (en) | Based on the embedded single image super-resolution rebuilding algorithm of optical flow method and sparse neighborhood | |
| CN111091524A (en) | Prostate transrectal ultrasound image segmentation method based on deep convolutional neural network | |
| CN105654425A (en) | Single-image super-resolution reconstruction method applied to medical X-ray image | |
| CN111178499A (en) | Medical image super-resolution method based on generation countermeasure network improvement | |
| Jiang et al. | CT image super resolution based on improved SRGAN | |
| CN113793348A (en) | Retinal vessel segmentation method and device | |
| CN115018728A (en) | Image fusion method and system based on multi-scale transformation and convolution sparse representation | |
| CN110648288B (en) | Choledochoscope image enhancement method based on residual transposition deconvolution neural network | |
| CN110189332B (en) | Prostate magnetic resonance image segmentation method and system based on weight map design | |
| CN111325700A (en) | Multi-dimensional fusion algorithm and system based on color images | |
| Wen et al. | Do noises bother human and neural networks in the same way? a medical image analysis perspective | |
| Tian et al. | Retinal fundus image superresolution generated by optical coherence tomography based on a realistic mixed attention GAN | |
| CN114066908A (en) | Method and system for brain tumor image segmentation | |
| Bouffard et al. | Multi-step reinforcement learning for medical image super-resolution | |
| Chilukuri et al. | Analysing Of Image Quality Computation Models Through Convolutional Neural Network | |
| CN111462004A (en) | Image enhancement method and device, computer equipment and storage medium | |
| Wang et al. | X-ray Image Blind Denoising in Hybrid Noise Based on Convolutional Neural Networks | |
| Khattar et al. | Investigation of Denoising Techniques for Removal of Hair and Noise from Dermoscopic Images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |