CN114529518A - Image pyramid and NLM-based image enhancement method for cryoelectron microscope - Google Patents
Image pyramid and NLM-based image enhancement method for cryoelectron microscope Download PDFInfo
- Publication number
- CN114529518A CN114529518A CN202210072173.9A CN202210072173A CN114529518A CN 114529518 A CN114529518 A CN 114529518A CN 202210072173 A CN202210072173 A CN 202210072173A CN 114529518 A CN114529518 A CN 114529518A
- Authority
- CN
- China
- Prior art keywords
- image
- pyramid
- background
- electron microscope
- formula
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 238000001000 micrograph Methods 0.000 claims abstract description 39
- 238000006243 chemical reaction Methods 0.000 claims abstract description 20
- 238000003705 background correction Methods 0.000 claims abstract description 18
- 238000005070 sampling Methods 0.000 claims abstract description 8
- 230000008569 process Effects 0.000 claims description 15
- 238000004422 calculation algorithm Methods 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 13
- 239000011159 matrix material Substances 0.000 claims description 13
- 238000005516 engineering process Methods 0.000 claims description 7
- 230000009466 transformation Effects 0.000 claims description 7
- 238000001914 filtration Methods 0.000 claims description 6
- 230000004927 fusion Effects 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 3
- 239000002245 particle Substances 0.000 abstract description 29
- 238000012545 processing Methods 0.000 description 12
- 238000013135 deep learning Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 4
- 238000003709 image segmentation Methods 0.000 description 4
- 238000002474 experimental method Methods 0.000 description 3
- 239000000523 sample Substances 0.000 description 3
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 238000002360 preparation method Methods 0.000 description 2
- 238000010187 selection method Methods 0.000 description 2
- 238000003325 tomography Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 230000002146 bilateral effect Effects 0.000 description 1
- 239000012472 biological sample Substances 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 238000012733 comparative method Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000008014 freezing Effects 0.000 description 1
- 238000007710 freezing Methods 0.000 description 1
- 229920002521 macromolecule Polymers 0.000 description 1
- 238000000386 microscopy Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000000746 purification Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10056—Microscopic image
- G06T2207/10061—Microscopic image from scanning electron microscope
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The invention relates to a cryoelectron microscope image enhancement method based on an image pyramid and NLM. Aiming at the condition that the signal-to-noise ratio of a frozen electron microscope image is extremely low and the particle image is difficult to select, the invention circularly uses NLM (non-line local mean) denoising, background correction and down-sampling to form an image pyramid, and each level of image is fused into a final enhanced image after histogram conversion and up-sampling, thereby improving the discrimination of the background and the particles. Compared with other methods, the image enhancement method provided by the invention has the advantages that the image peak signal-to-noise ratio and the structural similarity are obviously improved.
Description
Technical Field
The invention belongs to the technical field of structural biology analysis, and particularly relates to a cryoelectron microscope image enhancement method based on an image pyramid and NLM.
Background
The Cryo-Electron microscope (Cryo-EM) technology is one of the powerful means and mainstream methods for measuring biomacromolecule structure at present. The overall steps of reconstructing the three-dimensional structure of a biomolecule using cryoelectron microscopy techniques can be roughly divided into 4 steps: sample purification and preparation, data acquisition and analysis, two-dimensional particle image selection and classification, and three-dimensional structure reconstruction. Due to the limitation of the preparation technology of the biological sample and an electron microscope hardware system, the signal to noise ratio of the image acquired by the cryoelectron microscope is extremely low, and partial interference of non-sample particles exists, so that the automatic selection of the sample particles in the cryoelectron microscope image is difficult.
The current common particle selection methods comprise a template matching method, an image segmentation method, a deep learning method and the like. The template matching method judges whether the region is particle or background noise by calculating the correlation score between the template and the window to be detected of the image, wherein the template can be manually selected or simulated particle and noise. Image segmentation methods generally use image segmentation techniques to segment background and particles from cryo-electron microscope images that have undergone image enhancement. Due to the excellent performance of deep learning in recent years, a large number of deep learning methods are used for particle sorting, and the deep learning methods achieve particle sorting by training a deep learning network or model to have the capability of distinguishing background from particles.
Due to the problem of extremely low signal-to-noise ratio of a cryoelectron microscope image, image features are difficult to extract, except for preprocessing the image by an image segmentation method, other methods generally have no or only simple preprocessing, and the features of particle areas in the image are not sufficiently enhanced, so that the mainstream particle selection method is very complex in flow, limited in adaptive scene, and not high in overall accuracy of particle selection.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a freezing electron microscope image enhancement method based on an image pyramid and NLM, which is used for enhancing the quality of an electron microscope image, reducing the influence of noise on a particle area and enabling the edge to be clearer, so that the subsequent particle selection process can be simplified, and the overall selection efficiency can be improved.
In order to achieve the above object, the technical scheme provided by the invention is a cryo-electron microscope image enhancement method based on an image pyramid and an NLM, comprising the following steps:
step 2, removing image noise from the frozen electron microscope image by using a non-local mean denoising algorithm;
step 3, a background correction algorithm based on discrete cosine transform is used for the image with the noise removed, and the phenomenon of uneven brightness of the image is reduced;
step 4, setting the minimum size MinSize of the pyramid image, copying the image after background correction to construct an image pyramid;
step 6, using image histogram conversion and up-sampling technology to each layer of the image pyramid to make the size of each level of image of the pyramid the same;
and 7, accumulating and averaging the images with the same size formed by all levels of images of the image pyramid to obtain a fused image.
Moreover, the non-local mean denoising algorithm in the step 2 has the following calculation formula:
where v represents an image containing noise, x is a pixel point in the noise image v, and ΩxIs the search window of pixel point x, w (x, y) is the similarity weight between pixel point x and pixel point y in the neighborhood,
and the pixel value of a pixel point x in the denoised image is obtained.
w (x, y) is calculated as follows:
in the formula (d)2And calculating Euclidean distance of the blocks for the two similarities, wherein sigma is standard deviation of noise, and h is a filter smoothing parameter based on sigma.
d2The calculation method of (c) is as follows:
d2＝||V(x)-V(y)||2 (3)
in the formula, v (x), v (y) are similarity calculation blocks of the pixel point x and the pixel point y, respectively.
Moreover, for the image f (x, y) with the size of M × N in step 3, the two-dimensional discrete cosine transform DCT process is as follows:
in the formula, (x, y), (u, v) are pixel coordinates, and C (u, v) is a DCT coefficient matrix after discrete cosine transform.
And (4) applying a low-pass filter to the coefficient matrix C (u, v), and obtaining the approximate background of the image through an inverse transformation process of discrete cosine transform.
The low pass filter is formulated as follows:
in the formula uth,vthThreshold values in the u, v directions, respectively, CLP(u, v) is low pass filteringA matrix of post-wave DCT coefficients.
The process of obtaining the image background using the inverse IDCT of the two-dimensional discrete cosine transform is as follows:
wherein (x, y), (u, v) are pixel coordinates, fB(x, y) represents an approximate background image, CLP(u, v) is the low-pass filtered DCT coefficient matrix.
According to the original image f (x, y) and the background image fB(x, y) the formula for image rectification is as follows:
fsub(x,y)＝f(x,y)-fB(x,y) (11)
in the formula (f)sub(x, y) is a corrected image obtained by subtracting the background from the original image, fnorm(x, y) is an image obtained by normalizing the corrected image.
Moreover, the histogram transformation formula in step 6 is as follows:
where A is the maximum luminance value after histogram conversion, B is the mean value of background pixel distribution before conversion, k is a coefficient for controlling the gradient of the conversion function image, f (x, y) represents a pyramid image before histogram conversion, andHT(x, y) represents a straight lineAnd (5) a pyramid image after the square map transformation.
In step 7, the final image fusion mode is as follows:
in the formula, upsampling (I)k) Representing the image IkK is the number of layers of the pyramid image.
Compared with the prior art, the invention has the following advantages:
1) by using the non-local mean denoising method, the noise in the image can be more effectively inhibited, and the negative influence of the noise on the subsequent image enhancement step and the particle selection step is reduced;
2) the image pyramid, the image background correction and the image histogram conversion are used, so that the characteristics of particles in the enhanced image are more obvious, the discrimination between the particles and the background is higher, and the extraction and the segmentation of the particle image are more facilitated.
Drawings
FIG. 1 is a technical flow chart of an embodiment of the present invention.
Fig. 2 is a comparison graph of the results of the two image enhancement methods of the present invention, wherein fig. 2(a) is an enhancement graph obtained by simulating an electron microscope image and three image enhancement methods, fig. 2(b) is a binary graph of fig. 2(a), fig. 2(c) is an enhancement graph obtained by a real electron microscope image and three image enhancement methods, and fig. 2(d) is a binary graph of fig. 2 (c).
Fig. 3 is a comparison graph of the results of the binary image PSNR and the SSIM of the noisy image (solid line) and the enhanced image (dotted line), where fig. 3(a) is a comparison graph of the binary image PSNR and fig. 3(b) is a comparison graph of the binary image SSIM.
Detailed Description
The invention provides a cryoelectron microscope image enhancement method based on an image pyramid and an NLM, and the technical scheme of the invention is further explained by combining the attached drawings and the embodiment.
As shown in fig. 1, the process of the embodiment of the present invention includes the following steps:
And 2, removing image noise from the frozen electron microscope image by using a non-local mean (NLM) denoising algorithm.
The non-local mean denoising algorithm has the following calculation formula:
where v represents an image containing noise, x is a pixel point in the noise image v, and ΩxIs the search window of pixel point x, w (x, y) is the similarity weight between pixel point x and pixel point y in the neighborhood,
and the pixel value of a pixel point x in the denoised image is obtained.
w (x, y) is calculated as follows:
in the formula (d)2And calculating Euclidean distance of the blocks for the two similarities, wherein sigma is standard deviation of noise, and h is a filter smoothing parameter based on sigma.
d2The calculation method of (c) is as follows:
d2＝||V(x)-V(y)||2 (3)
in the formula, v (x), v (y) are similarity calculation blocks (similarity blocks) of the pixel point x and the pixel point y, respectively.
In this embodiment, in addition to using the NLM denoising algorithm, conventional denoising methods such as gaussian filtering, median filtering, wiener filtering, guided filtering, bilateral filtering, K-SVD, BM3D, and KNNM may be used, and a deep learning based denoising method such as DnCNN and Noise2Noise may also be used. In consideration of the complexity of an algorithm model, the calculation time of the algorithm, the denoising effect and the like, a better denoising effect can be achieved with higher efficiency by using the NLM denoising algorithm, a large amount of calculation can be reduced and the image processing speed can be improved by selecting a smaller neighborhood in the NLM algorithm, the size of a search window of each pixel point is 21 x 21, and the size of a similar block image is selected to be 7 x 7 pixels.
And 3, reducing the phenomenon of uneven brightness of the image by using a background correction algorithm based on discrete cosine transform on the image without the noise.
For an image f (x, y) of size M × N, the process of two-dimensional Discrete Cosine Transform (DCT) is as follows:
in the formula, (x, y), (u, v) are pixel coordinates, and C (u, v) is a DCT coefficient matrix after discrete cosine transform.
In the coefficient matrix C (u, v) of DCT, the low frequency part is the part where the image energy is concentrated, and the image is relatively flat and is generally the background; the high frequency portion is a region where the image change is severe, and is generally a boundary. Therefore, a low-pass filter is applied to the coefficient matrix C (u, v), and then the approximate background of the image can be obtained through the inverse transform process of the discrete cosine transform.
The low pass filter is formulated as follows:
in the formula uth,vthThreshold values in the u, v directions, respectively, CLP(u, v) is the low-pass filtered DCT coefficient matrix.
Through the analysis of the uneven brightness of the denoised cryoelectron microscope image, the overall background is foundThe uneven brightness areas are distributed in two corner areas on the diagonal line, or the middle area of the image, and the size of the uneven brightness areas occupies about one fourth to one half of the area of the image. In experiments using DCT for background correction, two thresholds u were also foundth,vthWhen all the brightness values are 3, various brightness uneven images can be considered, and the effect of meeting the follow-up processing can be achieved.
The process of obtaining the image background using the inverse two-dimensional discrete cosine transform (IDCT) is as follows:
wherein (x, y), (u, v) are pixel coordinates, fB(x, y) represents an approximate background image, CLP(u, v) is the low-pass filtered DCT coefficient matrix.
According to the original image f (x, y) and the background image fB(x, y) the formula for image rectification is as follows:
fsub(x,y)＝f(x,y)-fB(x,y) (11)
in the formula (f)sub(x, y) is a corrected image obtained by subtracting the background from the original image, fnorm(x, y) is an image obtained by normalizing the corrected image.
In this embodiment, the image background correction method may obtain the image background by using, in addition to the image background obtained based on the DCT transform, the image background by using the image gaussian blur, the top-hat-bottom-hat transform, the fourier transform, and the like, but there are problems of poor correction effect or long calculation time, and the like.
And 4, setting the minimum size MinSize of the pyramid image, copying the image after background correction and constructing an image pyramid.
Through analysis of a cryo-electron microscope image with the original size of 4096 × 4096 pixels, it is found that: for small size particles (80 × 80 pixels and below), most particles are substantially indistinguishable from the background when the image size is reduced to 200 × 200 pixels, and in order to allow for the handling of different sized particles, the minimum size of the pyramid image in this embodiment is set to 200 × 200 pixels.
And 6, using image histogram conversion and up-sampling technology for each layer of the image pyramid to ensure that the sizes of all levels of images of the pyramid are the same.
The histogram transformation formula is as follows:
where A is the maximum luminance value after histogram conversion, B is the mean value of background pixel distribution before conversion, k is a coefficient for controlling the gradient of the conversion function image, f (x, y) represents a pyramid image before histogram conversion, andHT(x, y) represents a pyramid image after histogram conversion.
Analyzing the image of the cryoelectron microscope image after image non-local mean de-noising and background correction, and finding that the background still has the condition that the brightness of a partial region is too high or the discrimination of the partial background region and a particle region is poor. Through a plurality of experimental trials, it is found that when a is set to 230, B is set to 110, and k is set to 20, the maximum brightness of the over-brightness region can be reduced to 230, thereby avoiding the overexposure problem which may occur in the subsequent processing, and the overall background brightness is improved to different degrees, so that the brightness of the particle region is reduced. Comparing the image histograms before and after histogram transformation, it can be seen that the particle area pixels are concentrated below 50 and the background pixels are concentrated above 200 after processing, which increases the discrimination between particles and background.
And 7, accumulating and averaging the images with the same size formed by all levels of images of the image pyramid to obtain a fused image.
Each layer of the pyramid image is up-sampled, the up-sampled result image and the original image have the same size, and the final fusion mode of the images is as follows:
in the formula IiThe ith layer, upsampling (I), representing a pyramid imagei) Indicating that the ith layer of the pyramid image is up-sampled, and k is the total number of layers of the pyramid image.
The effects of the present invention will be further described by comparative experiments.
Firstly, simulation conditions:
the experimental image data includes two types, one is a simulated electron microscope image obtained by using the biomacromolecule three-dimensional structure projection, and the other is a real electron microscope image shot by a cryoelectron microscope. The three-dimensional structure file used herein is a tomographic image of a biological macromolecule, all data files are from an open repository EMDB storing a cryoelectron microscope image and a tomographic image, and four representative tomographic image files used are numbered as EMD-2660, EMD-2824, EMD-3137, and EMD-3645. The used real cryoelectron microscope image data sets are numbered as EMPIAR-10028, EMPIAR-10017, EMPIAR-10033 and EMPIAR-10089, the tomography image file is formed by the real electron mirror image data sets through particle selection and three-dimensional reconstruction, and the real electron microscope number sequence corresponds to the tomography image file number sequence. In the experiment, the language used by all methods was python, and the software for running the two comparative methods (image enhancement method in AutoCryoPicker, supercryeme picker) was MATLAB.
Second, experimental contents and results
And respectively taking the original electron microscope images of the electron microscope image data set as input data of the method and the two comparison methods. In the processing flow of the method, the image noise is removed by using a non-local mean denoising method for the electron microscope image, then the image background is corrected by using discrete cosine transform, then the image is downsampled to obtain the lower-level image, the lower-level image is obtained by using the denoising, the background correction and the downsampling processing again, and the process is circulated until the image size reaches a set value. And forming each level of gradually reduced resolution in the image pyramid by the background corrected image with gradually reduced scale obtained in the circulation process. And performing histogram conversion and upsampling on each level of image of the pyramid to obtain images with the same size, and finally fusing the upsampled result images to obtain a final enhanced image. In the processing flow of the contrast method, an original electron microscope image is used as input data, and a final enhanced image is obtained through a series of image processing steps in the contrast method.
The method comprises the steps of processing a simulated electron microscope image and a real electron microscope image by three image enhancement methods, evaluating the images in subjective and objective aspects, subjectively comparing the enhanced images and binary images obtained by the three methods, and objectively measuring the similarity between an image binary image obtained by image enhancement of the simulated electron microscope image and a clean noiseless image binary image by using peak signal-to-noise ratio (PSNR) and Structural Similarity (SSIM).
The image enhancement result of the invention and the comparison method is shown in FIG. 2, and the experimental data result of PSNR and SSIM calculation on the binary image of the analog image enhancement result is shown in Table 1:
TABLE 1 PSNR and SSIM obtained by three image enhancement methods
As can be seen from Table 1, the image obtained by the method has an average peak signal-to-noise ratio (PSNR) of 13.1687, which is about 4.3dB higher than that obtained by an enhancement method 8.8393 in an AutoCryoPicker, and about 1.7dB higher than that obtained by an enhancement method 11.4388 in a SuperCryoEMPicker. From the viewpoint of similar average structure SSIM, the SSIM of the method is 0.8576, which is improved by about 0.1847 compared with the enhancement method 0.6729 in the AutoCryoPicker and is improved by about 0.0148 compared with the enhancement method 0.8428 in the SuperCryoEMPicker. As can be seen from the comparison of the results, the image enhancement method based on the image pyramid and the NLM can obtain a better result in the image enhancement stage of the cryoelectron microscope. As can be seen from fig. 2, the method of the present invention can obtain better image enhancement effect from subjectively comparing and enhancing the resulting image.
The PSNR and SSIM result pair calculated for the binary image is shown in fig. 3, where the solid line is the result of the image before processing and the dotted line is the result of the image after processing. As can be seen from FIG. 3, when the noise ratio is large (0.5 to 0.9), both the PSNR and SSIM of the processed image are significantly improved. Due to the fact that the image pyramid is used, the image loses part of detailed structures after multiple times of down sampling, and when the noise ratio is small (0.1-0.3), the PSNR and SSIM results of the processed image are lower than or close to those of the processed image. However, the real cryoelectron microscope image is characterized by extremely low signal-to-noise ratio, and the processing of the real cryoelectron microscope image by the method is not influenced by the condition of low noise ratio.
In specific implementation, the above process can adopt computer software technology to realize automatic operation process.
The specific embodiments described herein are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or ambit of the invention as defined in the appended claims.
Claims (5)
1. A cryo-electron microscope image enhancement method based on an image pyramid and NLM is characterized by comprising the following steps:
step 1, inputting cryo-electron microscope image data;
step 2, removing image noise from the frozen electron microscope image by using a non-local mean denoising algorithm;
step 3, a background correction algorithm based on discrete cosine transform is used for the image with the noise removed, and the phenomenon of uneven brightness of the image is reduced;
step 4, setting the minimum size MinSize of the pyramid image, copying the image after background correction to construct an image pyramid;
step 5, judging whether the size of the image after background correction is smaller than a set value MinSize, if so, finishing the construction of an image pyramid, and executing step 6, otherwise, re-executing the steps 2-4 by using a down-sampling technology on the image after background correction;
step 6, using image histogram conversion and upsampling technology to each layer of the image pyramid to enable the size of each level of image of the pyramid to be the same;
and 7, accumulating and averaging the images with the same size formed by all levels of images of the image pyramid to obtain a fusion image.
2. The cryo-electron microscope image enhancement method based on image pyramid and NLM as claimed in claim 1, wherein: the non-local mean denoising algorithm in the step 2 has the following calculation formula:
wherein v represents the image containing noise, x is the pixel point in the noise image v, omegaxIs the search window of pixel point x, w (x, y) is the similarity weight between pixel point x and pixel point y in the neighborhood,
the pixel value of a pixel point x in the denoised image is obtained;
w (x, y) is calculated as follows:
in the formula (d)2Calculating Euclidean distance of the blocks for the two similarities, wherein sigma is standard deviation of noise, and h is a filtering smoothing parameter based on sigma;
d2the calculation method of (c) is as follows:
d2＝||V(x)-V(y)||2 (3)
in the formula, v (x), v (y) are similarity calculation blocks of the pixel point x and the pixel point y, respectively.
3. The cryo-electron microscope image enhancement method based on image pyramid and NLM as claimed in claim 1, wherein: for the image f (x, y) with the size of M × N in step 3, the process of two-dimensional discrete cosine transform DCT is as follows:
wherein, (x, y), (u, v) are pixel coordinates, and C (u, v) is a DCT coefficient matrix after discrete cosine transform;
using a low-pass filter for the coefficient matrix C (u, v), and obtaining an approximate background of the image through an inverse transformation process of discrete cosine transform;
the low pass filter is formulated as follows:
in the formula uth,vthThreshold values in the u, v directions, respectively, CLP(u, v) is a low-pass filtered DCT coefficient matrix;
the process of obtaining the image background using the inverse IDCT of the two-dimensional discrete cosine transform is as follows:
wherein (x, y), (u, v) are pixel coordinates, fB(x, y) represents an approximate background image, CLP(u, v) is a low-pass filtered DCT coefficient matrix;
according to the original image f (x, y) and the background image fB(x, y) the formula for image rectification is as follows:
fsub(x,y)＝f(x,y)-fB(x,y) (11)
in the formula (f)sub(x, y) is a corrected image obtained by subtracting the background from the original image, fnorm(x, y) is an image obtained by normalizing the corrected image.
4. The cryo-electron microscope image enhancement method based on image pyramid and NLM as claimed in claim 1, wherein: the histogram transformation formula in step 6 is as follows:
where A is the maximum luminance value after histogram conversion, B is the mean value of background pixel distribution before conversion, k is a coefficient for controlling the gradient of the conversion function image, f (x, y) represents a pyramid image before histogram conversion, andHT(x, y) represents a pyramid image after histogram conversion.
5. The cryo-electron microscope image enhancement method based on image pyramid and NLM as claimed in claim 1, wherein: in step 7, each layer of the pyramid image is up-sampled, the up-sampled result image has the same size as the original image, and the final fusion mode of the images is as follows:
in the formula IiLayer I, representing a pyramid image, upsampling (I)i) Indicating that the ith layer of the pyramid image is up-sampled, and k is the total number of layers of the pyramid image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210072173.9A CN114529518A (en) | 2022-01-21 | 2022-01-21 | Image pyramid and NLM-based image enhancement method for cryoelectron microscope |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210072173.9A CN114529518A (en) | 2022-01-21 | 2022-01-21 | Image pyramid and NLM-based image enhancement method for cryoelectron microscope |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114529518A true CN114529518A (en) | 2022-05-24 |
Family
ID=81621442
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210072173.9A Pending CN114529518A (en) | 2022-01-21 | 2022-01-21 | Image pyramid and NLM-based image enhancement method for cryoelectron microscope |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114529518A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115359057A (en) * | 2022-10-20 | 2022-11-18 | 中国科学院自动化研究所 | Deep learning-based method and device for selecting particles of cryoelectron microscope and electronic equipment |
-
2022
- 2022-01-21 CN CN202210072173.9A patent/CN114529518A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115359057A (en) * | 2022-10-20 | 2022-11-18 | 中国科学院自动化研究所 | Deep learning-based method and device for selecting particles of cryoelectron microscope and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112200750B (en) | Ultrasonic image denoising model establishing method and ultrasonic image denoising method | |
| CN111209952B (en) | Underwater target detection method based on improved SSD and migration learning | |
| Fu et al. | Removing rain from single images via a deep detail network | |
| CN108765325B (en) | Small unmanned aerial vehicle blurred image restoration method | |
| CN109978848B (en) | Method for detecting hard exudation in fundus image based on multi-light-source color constancy model | |
| CN110503617B (en) | Underwater image enhancement method based on high-frequency and low-frequency information fusion | |
| CN110490814B (en) | Mixed noise removing method and system based on smooth rank constraint and storage medium | |
| CN111340716A (en) | Image deblurring method for improving dual-discrimination countermeasure network model | |
| CN110796616A (en) | Fractional order differential operator based L0Norm constraint and adaptive weighted gradient turbulence degradation image recovery method | |
| Mahmoud et al. | Segmentation of skin cancer images based on gradient vector flow (GVF) snake | |
| CN108564019A (en) | A kind of face identification method and system | |
| CN113837198A (en) | Improved self-adaptive threshold Canny edge detection method based on three-dimensional block matching | |
| Pujar et al. | Medical image segmentation based on vigorous smoothing and edge detection ideology | |
| CN114529518A (en) | Image pyramid and NLM-based image enhancement method for cryoelectron microscope | |
| Zhang et al. | A new image filtering method: Nonlocal image guided averaging | |
| CN115829967A (en) | Industrial metal surface defect image denoising and enhancing method | |
| CN113313655B (en) | Blind image deblurring method based on saliency mapping and gradient cepstrum technology | |
| CN110647843B (en) | Face image processing method | |
| CN114240990A (en) | SAR image point target segmentation method | |
| Thayammal et al. | Performance analysis of image denoising using deep convolutional neural network | |
| Xie et al. | DHD-Net: A novel deep-learning-based dehazing network | |
| Elhefnawy et al. | Effective visibility restoration and enhancement of air polluted images with high information fidelity | |
| CN114677303B (en) | Sub-pixel edge detection method based on integrated edge data enhancement | |
| CN116934776B (en) | FISH microscopic image signal point region extraction method, device, equipment and medium | |
| Wang et al. | Improved cell segmentation with adaptive bi-Gaussian mixture models for image contrast enhancement pre-processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |