CN116309435A - Medical image specular reflection restoration method and system - Google Patents
Medical image specular reflection restoration method and system Download PDFInfo
- Publication number
- CN116309435A CN116309435A CN202310244931.5A CN202310244931A CN116309435A CN 116309435 A CN116309435 A CN 116309435A CN 202310244931 A CN202310244931 A CN 202310244931A CN 116309435 A CN116309435 A CN 116309435A
- Authority
- CN
- China
- Prior art keywords
- specular reflection
- image
- pixels
- repaired
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 40
- 239000011159 matrix material Substances 0.000 claims abstract description 35
- 230000008439 repair process Effects 0.000 claims abstract description 25
- 238000012545 processing Methods 0.000 claims abstract description 14
- 238000012216 screening Methods 0.000 claims abstract description 3
- 229920006395 saturated elastomer Polymers 0.000 claims description 22
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000001514 detection method Methods 0.000 claims description 7
- 230000000007 visual effect Effects 0.000 abstract description 6
- 230000006870 function Effects 0.000 description 27
- 239000004973 liquid crystal related substance Substances 0.000 description 10
- 230000000694 effects Effects 0.000 description 6
- 238000003745 diagnosis Methods 0.000 description 5
- 238000001914 filtration Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000002059 diagnostic imaging Methods 0.000 description 3
- 238000000926 separation method Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000015556 catabolic process Effects 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000006731 degradation reaction Methods 0.000 description 2
- 230000000877 morphologic effect Effects 0.000 description 2
- FFBHFFJDDLITSX-UHFFFAOYSA-N benzyl N-[2-hydroxy-4-(3-oxomorpholin-4-yl)phenyl]carbamate Chemical compound OC1=C(NC(=O)OCC2=CC=CC=C2)C=CC(=C1)N1CCOCC1=O FFBHFFJDDLITSX-UHFFFAOYSA-N 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 229910052751 metal Inorganic materials 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Quality & Reliability (AREA)
- Radiology & Medical Imaging (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a medical image specular reflection restoration method, which comprises the following steps: s1: decomposing the acquired medical video sequence into single-frame images; s2: screening out a specular reflection area in a single frame image according to the brightness and saturation double threshold values to generate a mask matrix; s3: repairing the pixels of the specular reflection area by adopting a frequency domain selection reconstruction algorithm according to the mask matrix; s4: and merging the repaired images according to the processing sequence to obtain the video sequence with the repaired specular reflection. The invention can effectively repair pixels in the specular reflection area in the medical image, so that the image is more natural and has better visual effect.
Description
Technical Field
The invention relates to the field of image processing, in particular to a medical image specular reflection restoration method and system.
Background
With the rapid development of medical imaging technology and computer technology, high-quality medical imaging technology plays an increasingly critical role in assisting in diagnosis and treatment of clinical diseases, and the appearance of the medical imaging technology provides favorable conditions for doctors to improve diagnosis efficiency and reduce operation risks. However, due to the influence of the image acquisition equipment and the acquisition environment, various degradation and degradation problems of medical images inevitably occur, such as the areas with smooth surfaces, high reflection coefficients, such as human organ tissues, metal surgical instruments and the like, specular reflection areas are extremely easy to occur after imaging, the image characteristics and textures in the original areas can be covered by the large-range specular reflection areas, and meanwhile, because the brightness in the areas is saturated, the attention of doctors can be dispersed, and the observation and diagnosis of the doctors are influenced. Therefore, repairing the specular reflection area is beneficial to improving the visual level of the picture, optimizes the visual experience in the diagnosis process of doctors, and has practical significance and application value.
Current specular reflection repair methods generally include two stages, detection and repair: firstly, a mask matrix of a specular reflection area is obtained through detection, and then a restoration algorithm is adopted to restore the reflection area. The main ideas at present are mainly divided into two main categories: (1) An algorithm for specular reflection separation based on a bicolor reflection model; (2) an algorithm based on image restoration. The former is to separate specular reflection components, extract diffuse reflection components, and is suitable for the repair task of processing reflection areas which are unsaturated and exist with image information. The latter regards the specular reflection area as a missing area, and the reflection repair equivalent translates into a reconstruction of the missing area pixels, i.e. the missing pixels are reconstructed with known pixels outside the target area. Considering that the reflectance is generally high in practical medical images, pixels are generally saturated or nearly saturated. It is therefore desirable to provide a robust image restoration algorithm that effectively removes specular reflection regions from medical images.
Disclosure of Invention
The invention mainly aims to provide a medical image specular reflection restoration method capable of effectively eliminating a specular reflection area in a medical image.
The invention provides a medical image specular reflection restoration method, which comprises the following steps:
s1: decomposing the acquired medical video sequence into single-frame images;
s2: screening out a specular reflection area in a single frame image according to the brightness and saturation double threshold values to generate a mask matrix;
s3: repairing the pixels of the specular reflection area according to the mask matrix and by adopting a frequency domain selection reconstruction algorithm;
s4: and merging the repaired images according to the processing sequence to obtain the video sequence with the repaired specular reflection.
The technical scheme is that the step S2 specifically comprises the following steps:
s21: acquiring a brightness component L and a saturation component S of a single frame image;
s22: traversing each pixel, and detecting specular reflection pixels according to the brightness and saturation double thresholds;
s23: generating a mask matrix of the specular reflection area according to the detection result;
s24: the mask matrix is processed to generate a final mask.
By adopting the technical scheme, the brightness component L is a V component of an HSV space, an I component of an HSI space, a Y component of a YUV space or an L component of a CIELAB space; the saturation component S is the S component of HSV or HSI space or the C component of CIELCH space.
In step S22, the brightness threshold includes a first brightness threshold L 1 Second luminance threshold L 2 And a third brightness threshold L 3 The saturation threshold is S 1 The method comprises the steps of carrying out a first treatment on the surface of the Let Ω be the set of mirror-reflected pixels screened, then for any pixel p i E Ω, one of the following criteria needs to be met:
①
②
wherein N (p) i ) r Represented by pixel p i At the center, r is the set of all pixel values of the neighborhood of the radius.
With the technical scheme, the mask matrix is an element 0-1 matrix, 1 corresponds to a normal pixel, and 0 corresponds to a specular reflection pixel.
With the above technical solution, step S3 specifically includes the following steps:
s31: uniformly dividing the target image and the mask matrix into sub-blocks with fixed sizes, and marking the sub-blocks as image blocks to be processed;
s32: constructing a reconstruction region for each image block to be processed, and classifying pixels in the reconstruction region;
s33: calculating the priority of all the image blocks to be processed, and determining the repair sequence and the iteration times;
s34: calculating projection coefficients of the estimated residual error of the reconstruction region in a frequency domain;
s35: selecting a Fourier basis function and estimating an expansion coefficient, and calculating a projection coefficient;
s36: updating estimation and reconstruction residual errors of the reconstruction region according to the projection coefficients;
s37: stopping the iteration step S34-S36 until the iteration times, and outputting a restoration result of the image block to be processed;
s38: and finishing repairing all the image blocks to be processed, and outputting a repaired complete image.
In step S31, all sub-blocks with sub-block sizes of mxn and element numbers of not all 1 in the mask matrix are marked as image blocks to be repaired.
In step S32, the size of the reconstruction area of the image block B to be processed is 2 mx 2N, and the pixels in the reconstruction area are classified into three types, i.e. available, repaired and to-be-repaired, and sequentially stored in different sets A, R and B.
In step S33, the method for determining the repair order of the image block to be processed includes: the calculation formula of the center coordinates (X, Y) of the pixels in the B set and the sub-block priority level T in the image block area to be processed is as follows:
wherein N is p Is the number of pixels belonging to the B set in the image block B to be processed, -1 is less than or equal to a 2 <a 1 <The smaller the T value is, the higher the priority isHigh, f [ m, n ]]Represents a reconstruction region, and m and n are integers.
With the above technical solution, step S2 specifically includes: dividing the reflection types of pixels in a single frame image according to brightness and saturation threshold values, including saturated specular reflection, unsaturated specular reflection and diffuse reflection, extracting a target reflection area according to the reflection types, and obtaining a saturated specular reflection area mask matrix and an unsaturated specular reflection area mask matrix;
the step S3 specifically comprises the following steps: repairing pixels of the saturated specular reflection area according to the mask matrix of the saturated area; and repairing the pixels of the unsaturated specular reflection area according to the mask matrix of the unsaturated specular reflection area.
The invention has the beneficial effects that: according to the invention, the specular reflection area in the single frame image is screened out through the brightness and saturation double threshold values, so that the specular reflection area frequently appearing in the medical image can be effectively removed, the image abrupt sense caused by reflection pixels is eliminated, and the visual experience of doctors is optimized. The technical scheme provided by the invention can reconstruct the specular reflection area by adopting the frequency domain image restoration algorithm, optimize and improve the restoration quality and efficiency on the basis of the original algorithm, and improve the robustness and the practicability.
In addition, the invention also provides a method for fully covering the mirror reflection pixels (saturated and unsaturated), and different repairing methods are respectively adopted for different types of reflection pixels, so that the method has stronger robustness and better repairing effect.
Drawings
The invention will be further described with reference to the accompanying drawings and examples, in which:
FIG. 1 is a flow chart of a method for repairing specular reflection of an endoscopic image in accordance with an embodiment of the present invention;
FIG. 2 is a detailed flow chart of the embodiment of FIG. 1;
FIG. 3 is a flow chart of a method for repairing specular reflection of an endoscopic image in accordance with an embodiment of the present invention;
FIG. 4 is a detailed flow chart of the embodiment of FIG. 3;
FIG. 5 is a comparison of the medical image restoration of an embodiment of the present invention before and after;
FIG. 6 is a schematic diagram of a medical image specular reflection removal system according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The embodiment of the invention discloses a method for repairing mirror reflection of an endoscope image, which comprises the following steps:
s1: an endoscope video sequence is acquired through an image acquisition system and is decomposed into single-frame images;
s2: obtaining a mask matrix of a mirror reflection area of the endoscope image by using a detection algorithm;
s3: repairing the pixels of the specular reflection area by using a frequency domain selection reconstruction algorithm;
s4: sequentially combining the processed images to obtain an endoscope video sequence after mirror reflection repair;
further, detecting the specular reflection pixels in step S2 to obtain a mask matrix includes the steps of:
s21: the endoscopic image is converted into HSV color space, and the luminance component V and the saturation component S are extracted:
V＝max(R,G,B)
s22: setting a first brightness threshold L 1 =0.73, second luminance threshold L 2 =0.85, third luminance threshold L 2 =0.34, first saturation threshold S 1 =0.24. Traversing components V and S, detecting specular reflection pixels according to the following criteria, saved in set Ω:
①
②
wherein, the liquid crystal display device comprises a liquid crystal display device,
representing pixel p i Luminance value of>
Representing pixel p i Is a saturation value of (1); n (p) i ) 5 Representing pixel p i A set of neighborhood pixel values with a center radius of 5; mean is a function of the Mean.
S23: according to the detection result, the position of the specular reflection pixel is 1, the rest positions are 0, and a mask matrix M of the specular reflection area of the endoscope image is obtained;
s24: the mask matrix M is morphologically expanded using a circular structure with 5 x 5 structural elements:
further, the step S3 of repairing the specular reflection area of the endoscope image by the frequency selective reconstruction algorithm comprises the following steps:
s31: dividing an endoscope image and a mask matrix M into image blocks with the size of 16 multiplied by 16, and marking sub-blocks which are not all 1 in the M as image blocks to be repaired;
s32: traversing an image block B, constructing a 32 multiplied by 32 reconstruction area, dividing pixels in L into three types of available, repaired and to-be-repaired, and sequentially storing pixel coordinates in sets A, R and B; the reconstruction region contains the image block and surrounding available pixel areas that provide the information needed for the repair process. In the embodiment of the invention, the reconstruction area is 2 times of the image block, and the size of the reconstruction area can be determined according to the specific situation.
S33: setting alpha 1 ＝-0.2,α 2 = -0.5 (each acting as a contribution to compute image block processing priority for the pixels in adjustment set a, R), compute b-regionThe center coordinates (X, Y) and B priority levels T of the inner B-set pixels (i.e., pixels to be repaired) are formulated as follows:
wherein f [ m, n ] represents a reconstruction region, m, n being integers. The T values of the blocks are ordered to generate a processing sequence of the image blocks.
Setting the maximum iteration number I max =100, minimum iteration number I max =10, η=40, the gradient value Grad of the image block is calculated, and the number of iterations of this sub-block is determined:
I＝(I max -I min )×Grad
gradient template
S34: calculating an estimated residual
Setting sigma 1 ＝10，σ 2 =20, generating spatial domain weights for each pixel within an image block:
calculating projection coefficients of the weighted residual error of the reconstruction region estimation in a frequency domain:
s35: setting an orthogonality defect compensation factor g=0.5, and generating frequency domain weights by using a second-order butterworth low-pass filter:
d(k,l)＝[(k-M/2) 2 +(l-N2) 2 ] 1/2
the following target model is optimized, a Fourier basis function is selected, and corresponding expansion coefficients are estimated:
wherein k and l are frequency coordinates, w f [k,l]Representing the frequency weight in the k, l coordinates; the basis function is
The estimation of the expansion coefficient is +.>
S36: updating the estimation and estimation residual error of the reconstruction region:
s37: stopping the iterative steps S34-S36 when the iterative times are reached, and outputting the repairing result of the image block;
s38: and (5) finishing repairing all the image blocks, and outputting a repaired complete endoscope image.
The effectiveness of the technical solution proposed by the present invention is verified by several sets of endoscopic pictures. The left column of the picture of fig. 5 has a distinct specular reflection area at the same time, and in particular, there are consecutive specular reflection areas of larger area in left 2 and left 3. The reflective area significantly reduces the overall visual experience of the endoscopic image, which may affect the diagnosis of the physician in an actual endoscopic examination. The right column of the image in fig. 3 is an endoscopic image repaired by the technical scheme, and from the result, the specular reflection area is basically removed, the area after pixel replacement and the adjacent area are integrated, and the boundary transition is natural. Although the restoration is a process of generating an original image without pixels, the replaced pixels are different from the actual pixels, but the visual effect of the image is obviously improved, so that the technical scheme has practical significance and application value.
In the above embodiment, the existing frequency domain selection reconstruction algorithm (Frequency Selective Reconstruction) (see further embodiments of the invention, detailed below) is mainly 3-point improved:
1. and determining the processing priority of each sub-image block to be repaired.
The calculation priority in the prior art is the sum of Euclidean distances from the coordinates of all pixels in the calculation set A, R to the geometric center coordinates of the image block, and the center coordinates of the whole pixel to be repaired are defaulted as the center coordinates of the image block;
improved algorithm of the above embodiment: (1) The central coordinate of the whole pixel to be repaired is the average of the coordinates of all the pixels to be repaired, so that the setting is more reasonable; (2) Introducing parameter a 1 And a 2 To balance the contribution (importance) of the pixels in the set A, R to the T-value calculation, which is not considered in the prior art algorithm, but rather a simple euclidean distance cumulative summation is directly performed.
2. Iterative number determination of algorithm
The operation efficiency of the frequency domain selective reconstruction algorithm is critical. The algorithm is an iterative algorithm, and a certain number of iterations are needed to achieve a good repairing effect. Because the algorithm repairs the image blocks, the observable image blocks with weak textures (smooth images and small pixel edge changes) need less iteration times, more iterations can not bring about the improvement of the repairing effect, but can increase the time consumption; and the image blocks with strong textures (the surface of the image has large fluctuation and the pixel value changes severely) need more iteration times, and the repair effect can be influenced by fewer iteration times.
The existing algorithm: the same iteration times are set for all the image blocks, so that the image blocks with strong/weak textures are considered, the iteration times can overflow for the image blocks with weak textures, and a plurality of useless time cost is increased.
The above embodiment improves the algorithm: and taking the gradient value as a standard for measuring the texture intensity of the image block, thereby constructing a one-to-one correspondence relation between the gradient value and the iteration times. By setting the minimum and maximum iteration times and setting the iteration times according to the calculated gradient values of each image block, the time consumption can be reduced while the repairing effect is ensured.
3. Spatial and frequency domain weighting functions
First is a spatial domain weighting function: the spatial domain weighting function takes Euclidean distance from a pixel in a reconstruction area to the center of the pixel to be repaired as a function variable, wherein the spatial domain weighting value reflects the importance degree of different pixels to the repair.
The existing algorithm: an exponential function is used as the weighting function. Because the pixels in R have weaker importance to repair than the pixels in A, the original algorithm simply adds an attenuation factor on the basis of the pixel weight calculation function in A.
The embodiment algorithm described above: the spatial domain weight is distributed to the pixels in A, R by adopting the Gaussian functions with two different scales, and the pixel distribution mode has higher degree of freedom than the original algorithm and has more pertinence to the distribution of the weight in A, R.
The frequency domain weighting function is as follows: the effect of the frequency domain weighting function is to repair more important low frequency information because a high probability of high frequency information loss or spurious high frequencies in the image block to be repaired, giving more weight to the high frequency information, results in repair errors.
The existing algorithm: a frequency domain weighting function is calculated using otf.
Improved algorithm of the above embodiment: a butterworth low pass filter is used as the frequency domain weighting function. The overall shape of the two functions is similar, but the function curve of the improved algorithm is smoother, and the weight changes more gradually and naturally along with the frequency.
In another embodiment of the present invention, as shown in fig. 3, the method for repairing specular reflection of a medical image according to the present invention includes the following steps:
s301: acquiring a video sequence through an image acquisition system, and decomposing the video sequence into single-frame images;
s302: dividing the reflection types of pixels in a single frame image according to brightness and saturation threshold values, including saturated specular reflection, unsaturated specular reflection and diffuse reflection, extracting a target reflection area according to the reflection types, and obtaining a saturated specular reflection area mask and an unsaturated specular reflection area mask;
s303: repairing the pixel value of the target reflection area by using an algorithm combining image repairing and mirror surface separating, and repairing the pixels of the saturated mirror surface reflection area according to the saturated area mask; repairing pixels of the unsaturated specular reflection area according to the unsaturated specular reflection area mask;
s304: sequentially combining the repaired images to obtain a video sequence with the repaired mirror reflection;
further, the step S302 of dividing the pixel reflection type by using the detection algorithm includes the following steps:
s3021: acquiring brightness and saturation components L and S of an image;
s3022: traversing image pixels, and dividing the reflection type of the pixels according to brightness and saturation threshold values;
s3023: obtaining a mask of a mirror reflection area to be repaired;
s3024: carrying out morphological processing on the mask map to obtain a mask map of a final specular reflection area;
in step S3021, the luminance component L may be a V component of the HSV space, an I component of the HSI space, a Y component of the YUV space, or an L channel of the CIELAB space; the saturation component may be from the S of HSV and the C channel of CIELCH space;
the threshold in step S3022 includes a luminance threshold: lthr1, lthr2 ε (0, 1)]And saturation threshold: sthr1, sthr 2E (0, 1)]The method comprises the steps of carrying out a first treatment on the surface of the The reflection type includes saturated specular reflection omega 1 Unsaturated specular reflection omega 2 And diffuse reflection omega 3 ；
As shown in fig. 4, in a specific embodiment of the present invention, lthr1=0.73, lthr2=0.85, sthr1=0.24, sthr2=0.35, and the specific classification procedure is as follows:
set up the collection Ω 1 ,Ω 2 ,Ω 3 Storing 3 types of reflection type pixels, p for any pixel of the L component i ，N(p i ) Is a neighborhood of the method, and is classified as follows:
①
then p is i ∈Ω 1 ；
②
And satisfy->
Then p is i ∈Ω 1 The method comprises the steps of carrying out a first treatment on the surface of the Otherwise, p i ∈Ω 2 ；
③
Satisfy->
Then p is i ∈Ω 2 The method comprises the steps of carrying out a first treatment on the surface of the Otherwise, p is i ∈Ω 3 ；
④
Then p is i ∈Ω 3 。
In step S3023, the mask is a 0-1 matrix, and the 0 element corresponds to the specular reflection pixel. Wherein M is 1 Is a saturated specular reflection area mask; m is M 2 Is an unsaturated specular reflection area mask.
Pair M in step S3024 1 And M 2 And carrying out morphological expansion treatment to weaken the unnatural phenomenon of the boundary of the repair area. The specific structural elements are 5×5 circular structures:
further, step S303 repairs Ω 1 ,Ω 2 The pixel specifically comprises the following steps:
s3031: restoration of Ω using frequency domain selective reconstruction algorithm 1 A pixel;
s3032: repairing omega by adopting specular reflection separation algorithm combined with pixel clustering 2 A pixel;
s3033: based on the mask M after the expansion process 1 And M 2 The original image corresponding position pixels are replaced by the repair pixels, and other non-reflection area pixels remain unchanged.
The frequency selective reconstruction algorithm in step S3031 may use either the frequency selective reconstruction algorithms of the above embodiments S31 to S37 or the following frequency selective reconstruction algorithm, and specifically includes the following steps:
s30311: let I c C epsilon { r, g, b } represents a channel of RGB space of the image to be repaired, I is c Equally divided into sub-blocks of size W×H (e.g. 4×4), for M 1 Performing completely consistent sub-block division processing;
s30312: the processing sequence of the sub-blocks is determined according to the local density characteristics of the reflection pixels in the sub-blocks. For M 1 Carrying out Gaussian filtering, wherein the processing sequence of the sub-blocks is determined by the accumulated value of the pixel values of all the sub-blocks after filtering, and the accumulated value of the pixel values of all the sub-blocks after filtering is calculated, the larger the accumulated value is, the higher the priority is, and the filtering radius is selected to be max (W, H), such as Gaussian filtering with radius of 8;
s30313: for sub-block b i Outer push area L i Comprises b i And its neighborhood pixels, of size X X Y (X>W,Y>H) E.g. 8X 8, L i The region comprises three types of pixels which can be used, repaired and to be repaired, and the three types of pixels are sequentially stored in the sets A, R and B, wherein the pixels in the sets A and R dominate the repair of the pixels in the set B;
s30314: residual projection coefficients are calculated. Let f [ x, y ]]∈L i Consider the following sparse model:
wherein, the liquid crystal display device comprises a liquid crystal display device,
is an estimate L i Is estimated as (1)>
Is a Fourier basis function, +.>
Is the expansion coefficient, and the set K is formed by storing index tuples of all base functions in the sparse model. Let->
Represents f [ x, y ]]And the residual with the ith estimate. In the iterative process, the frequency domain reconstruction algorithm estimates the projection coefficients of the residuals on the fourier basis by minimizing the weighted residual energy function:
wherein, the liquid crystal display device comprises a liquid crystal display device,
projection coefficients representing residual weighting values on a Fourier basis, w [ x, y]Is a spatial domain weighting function:
wherein, the liquid crystal display device comprises a liquid crystal display device,
for weight attenuation factors, e.g. selectable +.>
w[x,y]Value of (c) represents L i The contribution and importance of each pixel in the repairing process are that the element B of the set is the pixel to be repaired, the element B does not participate in the repairing process, the weight is 0, the closer the pixel is to the center of the ion block, the higher the correlation degree of the pixel and the sub-block to be repaired is, the greater the weight is, the higher the reliability degree of the pixel A of the set is than that of the pixel R of the set is, and the weight is greater.
S30315: maximizing the model estimation basis functions and expansion coefficients as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,
s30316: updating L i Estimation of (a)
And residual:
s30317: and (3) circularly iterating the steps S314-S317, if the iteration times reach a preset value (such as 100 times), stopping the iteration process to obtain a repaired sub-block b i ；
S30318: repair I c Outputting the repaired image by all the sub-blocks;
further, the step S3032 specular reflection separation algorithm includes the following steps;
s30321: let I be the RGB image to be repaired, according to the bicolor reflection model there are:
I(x)＝w d Λ(x)+w s Γ
wherein Λ (x) and Γ are diffuse reflection and specular reflection chromaticity values, respectively, w d And w s Weight of corresponding component, w d Corresponding to diffuse reflection component Λ (x), w s Corresponding to specular reflection component Γ; calculation I max (x)，I min (x)，I range (x)：
I min (x)＝min(I r (x),I g (x),I b (x))＝w d Λ min (x)+w s (x)Γ
I max (x)＝max(I r (x),I g (x),I b (x))＝w d Λ max (x)+w s (x)Γ
I range (x)＝I max (x)-I min (x)＝w d (Λ max (x)-Λ min (x))
Wherein Λ (x) = [ Λ ] r (x),Λ g (x),Λ b (x)]，Λ min (x)＝min(Λ r (x),Λ g (x),Λ b (x))，Λ max (x)＝max(Λ r (x),Λ g (x),Λ b (x))。
S30322: calculating pseudo-feature-free images
Wherein->
Representation I min Calculate the mean value of I psf (x) Is defined by the following formula (i):
s30323: constructing maximum and minimum chrominance vectors for all pixels p
And based on the chromaticity space distance measurement, adopting a K-means algorithm to perform pixel clustering.
S30324: pixel intensity ratio I for each cluster ratio (x)＝I max (x)/I range (x) Non-strictly increasing sorting is carried out, and the intensity ratio of quantile positions is selected as I ratio (x) The separated diffuse reflection component is calculated as follows:
the medical image specular reflection restoration system according to the embodiment of the present invention is mainly used for implementing the restoration method of the above embodiment, as shown in fig. 6, and includes:
the single-frame decomposition module is used for decomposing the acquired medical video sequence into single-frame images;
the mask acquisition module screens out a specular reflection area in a single frame image according to the brightness and saturation double threshold values to generate a mask matrix;
the repair module is used for repairing the pixels of the specular reflection area according to the mask matrix and by adopting a frequency domain selection reconstruction algorithm;
and the video merging module merges the repaired images according to the processing sequence to obtain a video sequence with the repaired specular reflection.
Each module is mainly used for implementing the above method embodiments, and is not described herein in detail.
In another embodiment of the present invention, a medical image specular reflection restoration system includes:
the single-frame decomposition module is used for decomposing the acquired medical video sequence into single-frame images;
the mask acquisition module is used for dividing the reflection types of pixels in the single frame image according to the brightness and saturation threshold values, including saturated specular reflection, unsaturated specular reflection and diffuse reflection, extracting a target reflection area according to the reflection types, and acquiring a saturated specular reflection area mask and an unsaturated specular reflection area mask;
the repair module is used for repairing the pixels of the saturated mirror reflection area according to the saturated area mask; repairing pixels of the unsaturated specular reflection area according to the unsaturated specular reflection area mask;
and the video merging module is used for merging the repaired images according to the processing sequence to obtain a video sequence with the repaired specular reflection.
Each module is mainly used for implementing the above method embodiments, and is not described herein in detail.
The present invention also provides a computer readable storage medium such as a flash memory, a hard disk, a multimedia card, a card memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an App application store, etc., on which a computer program is stored that when executed by a processor performs a corresponding function. The computer readable storage medium of the present embodiment is for implementing the medical image specular reflection restoration method of the method embodiment when executed by a processor.
The above embodiment is exemplified by an endoscopic image, but not limited to an endoscopic image, and medical images acquired by other devices can be processed by the method of the present invention as long as the characteristics of the images processed by the present invention are met. The embodiment of the present invention repairs the specular reflection occurring in the endoscope image, which is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art who is within the scope of the present invention should be covered by the scope of the present invention by equally replacing or changing the technical scheme and the inventive concept thereof.
Claims (10)
1. A medical image specular reflection restoration method, comprising the steps of:
s1: decomposing the acquired medical video sequence into single-frame images;
s2: screening out a specular reflection area in a single frame image according to the brightness and saturation double threshold values to generate a mask matrix;
s3: repairing the pixels of the specular reflection area according to the mask matrix and by adopting a frequency domain selection reconstruction algorithm;
s4: and merging the repaired images according to the processing sequence to obtain the video sequence with the repaired specular reflection.
2. The medical image specular reflection restoration method according to claim 1, wherein step S2 specifically comprises the steps of:
s21: acquiring a brightness component L and a saturation component S of a single frame image;
s22: traversing each pixel, and detecting specular reflection pixels according to the brightness and saturation double thresholds;
s23: generating a mask matrix of the specular reflection area according to the detection result;
s24: the mask matrix is processed to generate a final mask.
3. The medical image specular reflection restoration method according to claim 1, wherein the luminance component L is a V component of HSV space, an I component of HSI space, a Y component of YUV space, or an L component of CIELAB space; the saturation component S is the S component of HSV or HSI space or the C component of CIELCH space.
4. The medical map of claim 2The image mirror reflection repairing method is characterized in that in step S22, the brightness threshold value comprises a first brightness threshold value L 1 Second luminance threshold L 2 And a third brightness threshold L 3 The saturation threshold is S 1 The method comprises the steps of carrying out a first treatment on the surface of the Let Ω be the set of mirror-reflected pixels screened, then for any pixel p i E Ω, one of the following criteria needs to be met:
①
②
wherein N (p) i ) r Represented by pixel p i At the center, r is the set of all pixel values of the neighborhood of the radius.
5. A medical image specular reflection repair method according to claim 2, wherein the mask matrix is an element 0-1 matrix, 1 corresponding to a normal pixel, and 0 corresponding to a specular reflection pixel.
6. The medical image specular reflection restoration method according to claim 1, wherein the step S3 specifically includes the steps of:
s31: uniformly dividing the target image and the mask matrix into sub-blocks with fixed sizes, and marking the sub-blocks as image blocks to be processed;
s32: constructing a reconstruction region for each image block to be processed, and classifying pixels in the reconstruction region;
s33: calculating the priority of all the image blocks to be processed, and determining the repair sequence and the iteration times;
s34: calculating projection coefficients of the estimated residual error of the reconstruction region in a frequency domain;
s35: selecting a Fourier basis function and estimating an expansion coefficient, and calculating a projection coefficient;
s36: updating estimation and reconstruction residual errors of the reconstruction region according to the projection coefficients;
s37: stopping the iteration step S34-S36 until the iteration times, and outputting a restoration result of the image block to be processed;
s38: and finishing repairing all the image blocks to be processed, and outputting a repaired complete image.
7. The method according to claim 1, wherein in step S31, the sub-blocks have a size of mxn, and all sub-blocks with element numbers of 1 in the mask matrix are marked as image blocks to be repaired.
8. The method according to claim 1, wherein in step S32, the size of the reconstruction area of the image block B to be processed is 2 mx 2N, and pixels in the reconstruction area are classified into three types, i.e., available, repaired and to be repaired, and are sequentially stored in different sets A, R and B.
9. The method for repairing specular reflection of medical image according to claim 8, wherein in step S33, the method for determining the repairing order of the image blocks to be processed is: the calculation formula of the center coordinates (X, Y) of the pixels in the B set and the sub-block priority level T in the image block area to be processed is as follows:
wherein N is p Is the number of pixels belonging to the B set in the image block B to be processed, -1 is less than or equal to a 2 <a 1 <The smaller the value of T is, the higher the priority is, f [ m, n]Represents a reconstruction region, and m and n are integers.
10. The medical image specular reflection restoration method according to claim 1, wherein step S2 specifically comprises: dividing the reflection types of pixels in a single frame image according to brightness and saturation threshold values, including saturated specular reflection, unsaturated specular reflection and diffuse reflection, extracting a target reflection area according to the reflection types, and obtaining a saturated specular reflection area mask matrix and an unsaturated specular reflection area mask matrix;
the step S3 specifically comprises the following steps: repairing pixels of the saturated specular reflection area according to the mask matrix of the saturated area; and repairing the pixels of the unsaturated specular reflection area according to the mask matrix of the unsaturated specular reflection area.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310244931.5A CN116309435A (en) | 2023-03-13 | 2023-03-13 | Medical image specular reflection restoration method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310244931.5A CN116309435A (en) | 2023-03-13 | 2023-03-13 | Medical image specular reflection restoration method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116309435A true CN116309435A (en) | 2023-06-23 |
Family
ID=86802667
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310244931.5A Pending CN116309435A (en) | 2023-03-13 | 2023-03-13 | Medical image specular reflection restoration method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116309435A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117314911A (en) * | 2023-11-29 | 2023-12-29 | 美迪信(天津)有限责任公司 | Method, device and storage medium for optimizing eye medical image |
-
2023
- 2023-03-13 CN CN202310244931.5A patent/CN116309435A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117314911A (en) * | 2023-11-29 | 2023-12-29 | 美迪信(天津)有限责任公司 | Method, device and storage medium for optimizing eye medical image |
| CN117314911B (en) * | 2023-11-29 | 2024-03-01 | 美迪信(天津)有限责任公司 | Method, device and storage medium for optimizing eye medical image |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5113841B2 (en) | Computer-aided analysis using video from an endoscope | |
| Lai et al. | Improved local histogram equalization with gradient-based weighting process for edge preservation | |
| CN109801221A (en) | Generate training method, image processing method, device and the storage medium of confrontation network | |
| JPH07299053A (en) | Computer diagnosis support method | |
| CN114240797B (en) | OCT image denoising method, device, equipment and medium | |
| CN114638767B (en) | Laparoscope image smoke removal method based on generation of countermeasure network | |
| CN112991365B (en) | Coronary artery segmentation method, system and storage medium | |
| CN111275686A (en) | Method and device for generating medical image data for artificial neural network training | |
| CN113284061B (en) | Underwater image enhancement method based on gradient network | |
| CN116309435A (en) | Medical image specular reflection restoration method and system | |
| CN112950596A (en) | Tone mapping omnidirectional image quality evaluation method based on multi-region and multi-layer | |
| Colomer et al. | Assessment of sparse-based inpainting for retinal vessel removal | |
| Yue et al. | Perceptual quality assessment of enhanced colonoscopy images: A benchmark dataset and an objective method | |
| Cui et al. | Progressive dual-branch network for low-light image enhancement | |
| CN111476739B (en) | Underwater image enhancement method, system and storage medium | |
| CN114119447A (en) | Method, system, medium and device for constructing lung nodule multi-attribute classification model | |
| CN116630198A (en) | Multi-scale fusion underwater image enhancement method combining self-adaptive gamma correction | |
| Xu et al. | A reference image database approach for NLM filter-regularized CT reconstruction | |
| CN114897757A (en) | Remote sensing image fusion method based on NSST and parameter self-adaptive PCNN | |
| Bhateja et al. | Mammographic image enhancement using double sigmoid transformation function | |
| CN114862762A (en) | Quality evaluation method and device for human body scanning image | |
| CN112102216B (en) | Self-adaptive weight total variation image fusion method | |
| Lee et al. | Image fusion-based tone mapping using gaussian mixture model clustering | |
| CN108876734B (en) | Image denoising method and device, electronic equipment and storage medium | |
| Suresh et al. | Improving the mammogram images by intelligibility mammogram enhancement method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |