CN110992288B - Video image blind denoising method used in mine shaft environment - Google Patents
Video image blind denoising method used in mine shaft environment Download PDFInfo
- Publication number
- CN110992288B CN110992288B CN201911242397.4A CN201911242397A CN110992288B CN 110992288 B CN110992288 B CN 110992288B CN 201911242397 A CN201911242397 A CN 201911242397A CN 110992288 B CN110992288 B CN 110992288B
- Authority
- CN
- China
- Prior art keywords
- image
- block
- frame
- video
- image block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000006073 displacement reaction Methods 0.000 claims abstract description 42
- 230000009466 transformation Effects 0.000 claims abstract description 28
- 239000011159 matrix material Substances 0.000 claims abstract description 17
- 238000001914 filtration Methods 0.000 claims abstract description 10
- 230000000694 effects Effects 0.000 claims description 16
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 7
- 230000033001 locomotion Effects 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- 230000003044 adaptive effect Effects 0.000 claims description 4
- 230000004927 fusion Effects 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 abstract description 3
- 230000004931 aggregating effect Effects 0.000 abstract 1
- 238000002474 experimental method Methods 0.000 description 4
- 230000032683 aging Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/223—Analysis of motion using block-matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/285—Analysis of motion using a sequence of stereo image pairs
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention relates to a blind denoising method for video images in a mine environment, which comprises the following steps: selecting a plurality of image frames adjacent to the front and back of the video image to be denoised, searching an image block group with typical characteristics in each image frame, and determining the accurate displacement between frames by searching a block group with the minimum difference degree with the image block group in the previous frame image; dividing a video image to be denoised into a plurality of image blocks, and quickly constructing a self-similar sequence set of each image block according to the accurate displacement between frames; combining two-dimensional image blocks corresponding to the self-similar sequence set into a three-dimensional matrix, performing three-dimensional transformation, and performing self-adaptive threshold filtering on transformation coefficients; and aggregating the three-dimensional matrix after the three-dimensional inverse transformation to generate a denoising image. Compared with the existing image denoising method, the method improves the algorithm execution efficiency, and the obtained denoising image has better signal-to-noise ratio and visual effect.
Description
Technical Field
The invention relates to the technical field of mine An Tong whole and digital image processing, in particular to a blind denoising method for a video image in a mine shaft environment.
Background
With the development of image processing technology, the collected video images of mine shaft facilities are spliced and identified, so that importance is placed on whether equipment has faults or not rapidly and accurately, however, in a complex lifting shaft environment, more noise is contained in the facility video images collected by means of a camera shooting technology on a cage moving at a high speed, and the automatic analysis and identification precision of the faults of subsequent shaft facilities are affected. Denoising noisy images is a fundamental and critical issue in the fields of digital image processing and computer vision. In particular, noise often appears on an image as isolated pixels or blocks of pixels that cause a strong visual effect, and image denoising refers to the process of reducing noise in a digital image.
In recent years, a great deal of researches are carried out on image denoising algorithms by a plurality of researchers, and currently, the image denoising method mainly comprises two main types of image denoising methods based on deep learning and traditional image denoising methods. With the development of big data technology and computer hardware, the image denoising method based on deep learning becomes a hot spot for researching the image denoising field, but when the method is used for training an image denoising model, a large number of noise images and corresponding clean images are required as sample pairs by a training set, and under the condition that the clean images are difficult to obtain or the training set is lacking, the denoising performance of the image denoising method based on deep learning is greatly affected. The traditional image denoising method only uses the information of the image to denoise, wherein the recognized algorithm with good denoising effect is NL-Means algorithm and BM3D algorithm, and on the basis, a plurality of students improve the algorithm to improve the denoising performance and aging. However, the existing image denoising calculation has a good denoising effect on a noise image with a known noise level, but the denoising effect on a mine shaft video image with an unknown noise level is not ideal. The noise is not removed well when the noise variance is estimated to be too small, and the details of the denoised image are blurred when the noise variance is estimated to be too large.
In addition, image denoising is a relatively time-consuming process from the time-consuming aspect, and the existing traditional denoising method needs to calculate the similarity between each pixel point in an image and a plurality of pixels in a search window, or divide the image into a plurality of image blocks and search the similar image blocks in the search window, and perform weighted average by using the pixels or the image blocks to realize image denoising. The size of the search window directly affects the time complexity of the algorithm, a small search area has small calculation amount but poor denoising performance, and a larger search area has better denoising performance but increased calculation amount. The existing image denoising algorithm has the defects that the complexity of the algorithm is high and the denoising effect is not ideal because redundant information among image frames of the well bore video is not utilized.
Disclosure of Invention
Aiming at the technical problems, the invention aims to provide a blind denoising method for video images in a mine shaft environment, which utilizes redundant information existing among video image frames to effectively and blindly denoise shaft images with unknown noise variance with high signal-to-noise ratio and high efficiency.
The technical scheme of the invention is a blind denoising method for video images in a mine shaft environment, which comprises the following steps:
step 2, calculating the accurate displacement between video frames by using the typical characteristic block group obtained in the step 1;
step 3, constructing a self-similar sequence set for each image block in the noise image frame according to the inter-frame accurate displacement calculated in the step 2;
step 4, estimating the noise level of the noise image, and calculating the self-adaptive threshold value of the three-dimensional transformation coefficient on the basis of the noise level;
and 5, filtering the self-similar sequence set by using three-dimensional transformation and three-dimensional inverse transformation based on the self-adaptive threshold according to the similar sequence set obtained in the step 3 to obtain a corresponding three-dimensional image block matrix, and carrying out weighted fusion on the three-dimensional image block matrix to obtain a final denoising image.
Further, the implementation mode of the step 1 is that,
for image blocks with given size, searching n image blocks with the mean square error larger than a given threshold as typical characteristic blocks, and constructing n typical characteristic block groups according to a predefined block group structure;
let the video image to be denoised at time t be u t The video image of the previous frame is u t-1 Determining video image frame u t And u t-1 Image u at inter-frame displacement of (2) t Each image block of (a) has a size of N 2 ×N 2 Video image frame u t Image block to be distinguished with upper left corner coordinate at x
Normalizing to generate->
Mean>
Sum of mean square error->
The method comprises the following steps:
wherein Ω x is a block of an image
The coordinate set of all pixels in the image block, k 'is the element coordinate in the image block, k' is E.OMEGA.x, a threshold value Ts is given, and the image block to be distinguished is +.>
Satisfy->
Is identified as a typical characteristic image block in the video image u t Stopping when n typical characteristic image blocks are searched;
each typical characteristic block is taken as a center, and the adjacent image blocks which are directly adjacent to the block group and have the same size are taken as characteristic block groups according to the block group structure.
Further, the implementation manner of the step 2 is that,
by video image u in the previous frame t-1 In search and current image frame u t The block group positioning inter-frame accurate displacement with minimum difference degree of typical characteristic block group, and the obtained accurate displacement is recorded as
Set video image frame u t Comprises N 1 ×N 1 Reference image block of individual pixels
And video image frame u at the kth time k Image block->
Degree of difference between->
The method comprises the following steps:
wherein the subscript xR is a reference image block
The upper left pixel is at u t In, the subscript x is the image block +.>
The upper left pixel is at u k Two-dimensional coordinates of (a); />
Is a two-dimensional Euclidean distance; t (T) 2D Is a two-dimensional discrete cosine DCT transform;
then->
And->
Is distinguished as a similar image block, T m Refers to a given degree of variance threshold;
let the upper left pixel coordinate of the 1 st found representative feature image block be x 10 Estimating delta from an initial given inter-frame shift ini Mapped to the previous video image frame u t-1 The coordinate of (a) is x 10 +Δ ini The method comprises the steps of carrying out a first treatment on the surface of the In video frame u t-1 In the form of the coordinate x 10 +Δ ini Is centered and has a width of 2Δ ini Is a square neighborhood of (c)
In, searching the image block group with the smallest difference degree with the n typical characteristic image block groups, and determining the accurate displacement between frames>
The method comprises the following steps:
wherein x is ij The pixel coordinate of the upper left corner of the image block with the sequence number j in the ith characteristic block group; n is the number of typical characteristic image block groups in the image; m is the number of image blocks in each block group; w (w) ij For each block group i, the block distance d of the typical feature image block is the distance weight of the image block with the serial number j in the ith feature block group i1 =0, and the block group is directly adjacent to the top, bottom, left and right image block distance d ij Block distance at four corners =1
Further, the implementation manner of the step 3 is that,
frame u of video image t With step length N step Divided into N 1 ×N 1 Image block with size is used as reference block, and u is used as reference block t Centered video image u of continuous 2l+1 frames t-l ,…,u t ,…,u t+l Sequentially searching the self-similar image blocks of each reference block and constructing a corresponding self-similar sequence set;
assume that the video is collected by the collecting device in a relatively uniform motion from top to bottom through the displacement between adjacent image frames
Fast determination of video image frame u t Each reference picture block of (a)>
Corresponding similar image blocks in adjacent 2l+1 image frames, each reference image block +.>
Self-similar sequence set->
The method comprises the following steps:
wherein x is the upper left corner coordinate of the corresponding similar image block in the image frame;
stored are 2l+1 video image frames and reference image blocks +.>
The upper left element coordinates and the image frame number of the similar image block, +.>
Denoted as->
The number of similar blocks in the self-similar sequence set of (a) is +.>
Further, the implementation manner of the step 4 is that,
calculating a three-dimensional transformation adaptive threshold value, and calculating a video image frame u by using a typical characteristic block group and image frame displacement t And u t-1 The difference degree between the two is used for estimating the noise level of the noise image, and the calculation process is as follows:
inter-frame accurate displacement when 2l+1 frames
After being determined, the video image frame u is obtained t And inter-frame displacement of other adjacent 2l video image frames, and average inter-frame difference of adjacent 2l video image frames: />
By calculating threshold lambda corresponding to video image frames with different noise variances in optimal denoising effect 3D And corresponding average inter-frame disparity dist t By a pair lambda 3D And dist t Fitting to obtain self-adaptive threshold lambda 3D ：
λ 3D ＝min(0.1824×(dist t ) 2 +0.7285×dist t +0.0394,6.386)。
Further, the specific implementation manner of the step 5 is that,
for reference blocks
Self-similar sequence set->
If->
Denoising the image block by adopting a discrete cosine denoising algorithm based on a threshold value, otherwise, carrying out threshold value processing on the transformed coefficient by adopting 3D filtering transformation, so as to weaken noise interference; will->
Corresponding image blocks of the image are combined into a three-dimensional image matrix +.>
Three-dimensional matrix after 3D transformation and inverse transformation>
The method comprises the following steps:
wherein T is 3D For a 3D transformation operator, adopting 3D Fourier transformation;
is a 3D inverse transform operator; gamma is based on a threshold lambda 3D Transform operator of (2) to reduce 3D transform coefficients to less than a threshold lambda 3D Is set to 0 and the 3D transform coefficient is greater than the threshold lambda 3D The coefficients of (2) remain unchanged;
three-dimensional image block matrix
Is composed of->
A 3D filtered two-dimensional image block, each of the filtered two-dimensional image blocks being denoted +.>
The subscript sn denotes the frame number in which the two-dimensional image block is located, while each image block is +.>
The coordinates of all elements of (a) are mapped to and reference picture block +.>
The coordinates of the corresponding pixels are the same, i.e. each image block +.>
The upper left corner pixel coordinate of (2) is xR; every image block->
The weight of a pixel at any coordinate x within is defined as:
wherein the image block
Is->
Corresponding image blocks before filtering, fusing the filtered image blocks, and denoising the pixel gray level estimated value +.>
The definition is as follows:
wherein the method comprises the steps of
The frame number of the stored image block is shown, and X is the coordinate set of the noise image.
The invention has the following advantages and positive effects:
(1) The invention utilizes the information redundancy among the frames of the video, and rapidly determines the accurate displacement among the frames through the typical characteristic block group, thereby solving the problem that the inter-frame displacement is different due to the shake and the non-uniform motion of the image video acquisition device;
(2) The invention only utilizes the accurate displacement between video frames to quickly construct the self-similar sequence set, thereby avoiding the regional search of each image block and further improving the execution efficiency of the algorithm;
(3) According to the inter-frame information redundancy, the method utilizes the average inter-frame difference degree to estimate the noise intensity of the video image to be denoised, and builds an adaptive threshold on the basis, so that the blind denoising problem of the image is solved;
(4) The invention filters the image by using the 3D transformation and the 3D inverse transformation based on the self-adaptive threshold value, and carries out weighted aggregation on the three-dimensional matrix after the 3D inverse transformation to generate the denoising image, thereby leading the denoising image after the algorithm processing to have better visual effect and signal-to-noise ratio.
Drawings
FIG. 1 is a schematic view of a typical feature block group structure according to an embodiment of the present invention, wherein the middle part is a typical feature image block;
FIG. 2 is a process flow diagram of an embodiment of the present invention;
fig. 3 is a processing effect diagram of an embodiment of the present invention, where (a) is a noise image, (B) is an image denoised by NL-Means algorithm, (c) is an image denoised by FNL-Means algorithm, (D) is an image denoised by BM3D algorithm, (e) is an image denoised by BM3D-B algorithm, (f) is an image denoised by the present invention, and an image after an arrow is an enlarged effect diagram.
Detailed Description
The technical scheme of the invention is described in detail below with reference to the accompanying drawings and examples.
The technical scheme of the invention can adopt a computer software technology to realize an automatic operation flow, and the flow of the video image blind denoising method for the mine shaft environment provided by the embodiment sequentially comprises the following steps:
1. searching typical characteristic image blocks on an image to be denoised, and constructing a typical characteristic block group according to a block group structure
The invention utilizes the typical characteristic image blocks to construct the characteristic block group for rapidly positioning the accurate displacement between frames so as to rapidly construct the self-similar sequence set of the image blocks subsequently, thereby improving the denoising time efficiency and the signal-to-noise ratio. However, in practical applications, the jitter and non-uniform motion of the video acquisition device make the inter-frame displacement calculated only by the movement speed of the acquisition device inaccurate, and further accurate determination of the inter-frame displacement is required, so as to avoid that the accuracy of the inter-frame displacement positioning of the video can be improved through typical feature blocks.
The embodiment defines the video image frame at the t-th time as u t The size of the image block to be distinguished is N 2 ×N 2 The experiment shows that N is equal to N 2 A value of 16 gives a better effect. To determine video image frame u t-1 And u t For example, in order to prevent the influence of factors such as illumination, the video image frame u is t Image block to be distinguished
Normalizing to generate->
Mean>
Sum of mean square error->
The method comprises the following steps:
wherein subscript x is an image block
The upper left pixel is in image frame u t In, Ω x is the image block +.>
K 'is the element coordinates in the image block, k' e Ω x.
The embodiment gives a threshold value Ts, and the image block to be distinguished
Satisfy->
Is then identified as a representative feature image block, and in particular implementations, the representative feature block threshold value Ts is set at [0.04,0.08 ]]Better experimental results can be obtained by taking values, and the invention selects Ts=0.06, and selects N from Ts in the text 2 =16. N typical feature image blocks which are not overlapped with each other are searched sequentially from left to right from top to bottom, according to the block group structure comprising 5 image blocks as shown in fig. 1 given in the embodiment, n typical feature image block groups are constructed to be combined together to determine inter-frame shift, and experiments show that n=2 can obtain better denoising effect and higher time efficiency.
2. Calculating video inter-frame accurate displacement using representative feature block group
The present invention uses the representative feature block group searched in the current image frame to determine the exact inter-frame displacement with the previous frame video image, and the implementation example locates the inter-frame exact displacement by searching the previous frame video image for a block group having the smallest degree of difference with the representative feature block group of the current image frame.
The embodiment defines a video image frame u at time t t Comprises N 1 ×N 1 Reference image block of individual pixels
And video image frame u at the kth time k Image block->
Degree of difference between->
The method comprises the following steps:
in which the subscript xR is a reference image block
The upper left pixel is in video image frame u t In, the subscript x is the image block +.>
The upper left pixel is in video image frame u k Two-dimensional coordinates of (a); />
Is a two-dimensional Euclidean distance; t (T) 2D Is a two-dimensional discrete cosine DCT transform. Ideally, when->
At the time, image block->
Reference imageBlock->
Is a similar image block. Given the variance threshold T taking into account the effect of noise m The method comprises the following steps:
then
And->
Is discriminated as a similar image block.
Different N 1 The value affects the effectiveness of the algorithm and the signal-to-noise ratio of the de-dried image. Experimental results indicate that when N 1 When the value is smaller, the denoised image has the blocking effect phenomenon, and when N is 1 When the value is larger, the correlation of pixels in the image block is lower, the denoised image contains more noise, the time efficiency is reduced, and N 1 Selecting 8 achieves better results. In particular, when T m Smaller, the average signal-to-noise ratio of the denoised image follows T m Increase by increasing, and when T m When the average signal-to-noise ratio of the denoised image is reduced when the average signal-to-noise ratio is increased beyond a certain value. T (T) m Excessive values will cause large difference between image blocks in the self-similar sequence set, and the denoised image has blurring phenomenon, T m Too small a value will result in a reduction of image blocks in the set of self-similar sequences, so that the denoised image contains more noise. From the experimental results, T m In section [0.14,0.26 ]]Better denoising results can be obtained by taking the value of the image, and when the noise variance of the image is less than 30, T is removed m When the value is equal to or greater than 30, T is taken m ＝0.20。
The invention sets the upper left corner pixel coordinate of the searched 1 st typical characteristic image block as x 10 Estimating delta from an initial given inter-frame shift ini Mapped toPrevious video image frame u t-1 The coordinate of (a) is x 10 +Δ ini . In video frame u t-1 In the form of the coordinate x 10 +Δ ini Is centered and has a width of 2Δ ini Is a square neighborhood of (c)
In, searching the image block group with the smallest difference degree with the n typical characteristic image block groups, and determining the accurate displacement between frames>
The method comprises the following steps:
wherein x is ij The pixel coordinate of the upper left corner of the image block with the sequence number j in the ith characteristic block group; n is the number of typical characteristic image block groups in the image; m is the number of image blocks in each block group; w (w) ij For each block group i, the block distance d of the typical feature image block is the distance weight of the image block with the serial number j in the ith feature block group i1 =0, and immediately adjacent upper, lower, left and right image block distances d ij Block distance at four corners =1
In the implementation process of the embodiment, experiments show that the number n of the characteristic feature block groups is 2, so that a good operation effect can be obtained.
Embodiments in practice de-desiccate successive multi-frame video image frames, when video image frame u t When the denoising operation of (a) is completed, the video image frame u t+l The previous inter-frame accurate displacements have been obtained for video image frame u t+1 When denoising is performed, only the video image frame u is needed t+l+1 Performing a representative feature block lookup to determine a video image frame u t+l And u t+l+1 Inter-frame displacement between the two, the algorithm will further reduce the amount of denoising calculation for each video image frame.
3. Constructing a set of self-similar sequences for each image block for noisy video image frames
The invention is based on the inter-frame accurate displacement
A set of self-similar sequences is constructed for each image block in the noisy image frame. Frame u of video image t With step length N step Divided into N 1 ×N 1 Image block with size is used as reference block, and u is used as reference block t Centered video image u of continuous 2l+1 frames t-l ,…,u t ,…,u t+l And searching the self-similar image blocks of each reference block in turn and constructing a corresponding self-similar sequence set. Different step sizes N step Influence the ageing of the algorithm, N step The value is too small, the running time of the algorithm is long, and the value is too large, so that the denoising effect is poor. Experiments show that N step And 3, a better operation result can be obtained.
The invention sets that the video is collected by the collecting device in a relative uniform motion from top to bottom, and the displacement between adjacent image frames
Fast determination of video image frame u t Each reference picture block of (a)>
Corresponding similar image blocks in adjacent 2l+1 image frames. Each reference picture block->
Self-similar sequence set->
The method comprises the following steps:
In the invention, the signal-to-noise ratio is highest when 4 is taken, the denoising effect is best, but the algorithm is too long; l takes 1, the running time is shortest, but the denoising effect is slightly poor, and the detail part of the denoising image is kept slightly poor. The present invention chooses l=3, i.e. builds a set of self-similar sequences on consecutive 7 video image frames, combining the signal-to-noise ratio and the run time.
4. Estimating the noise level of the noise image, and calculating an adaptive threshold of the three-dimensional transformation coefficient on the basis of the estimated noise level;
the invention utilizes the typical characteristic block group and the accurate displacement between frames to calculate the video image frame u t And u t-1 The degree of difference between them is:
inter-frame accurate displacement when 7 frames
After being determined, the video image frame u can be found t And the inter-frame displacement of other adjacent 6 video image frames, and the average inter-frame difference of the adjacent 6 video image frames:
dist without noise t With a value of 0, dist in the presence of noise t And the intensity correlation of the noise, can be used to estimate the image noise variance.
The invention calculates the threshold lambda corresponding to the video image frames with different noise variances in the best denoising effect 3D And corresponding average inter-frame disparity dist t By a pair lambda 3D And dist t Fitting to obtain self-adaptive threshold lambda 3D The method comprises the following steps:
λ 3D ＝min(0.1824×(dist t ) 2 +0.7285×dist t +0.0394,6.386) (10)
5. and carrying out 3-dimensional filtering of the self-adaptive threshold value on the self-similar sequence set of each image block, and carrying out weighted fusion on the three-bit image block matrix after filtering to generate a final denoising image.
Embodiments are directed to reference blocks
Self-similar sequence set->
If->
The image block is denoised by a Discrete Cosine (DCT) denoising algorithm based on a threshold value, otherwise, the transformed coefficient is thresholded by 3D filtering transformation, so that noise interference is reduced. Will->
Corresponding image blocks of the image are combined into a three-dimensional image matrix +.>
By passing throughThree-dimensional matrix after 3D conversion and inverse conversion>
The method comprises the following steps:
wherein T is 3D For a 3D transformation operator, the invention adopts 3D Fourier transformation;
is a 3D inverse transform operator; gamma is based on a threshold lambda 3D Transform operator of (2) to reduce 3D transform coefficients to less than a threshold lambda 3D Is set to 0 and the 3D transform coefficient is greater than the threshold lambda 3D The coefficients of (2) remain unchanged.
Three-dimensional image block matrix
Is composed of->
A 3D filtered two-dimensional image block, each of the filtered two-dimensional image blocks being denoted +.>
The subscript sn denotes the frame number in which the two-dimensional image block is located, while each image block is +.>
The coordinates of all elements of (a) are mapped to and reference picture block +.>
The coordinates of the corresponding pixels are the same, i.e. each image block +.>
The upper left pixel coordinate of (c) is xR. Every image block->
The weight of a pixel at any coordinate x within is defined as:
wherein the image block
Is->
Corresponding pre-filtered image block, +.>
And reference image block->
The smaller the degree of difference, the video image frame u at which sn And a video image frame u to be denoised t The more adjacent the block->
The greater the contribution to the final estimate of the denoised video image.
Fusing the filtered image blocks, and denoising each pixel point x in the noise image to obtain a pixel gray level estimated value
The definition is as follows:
wherein the method comprises the steps of
The frame number of the stored image block is shown, and X is the coordinate set of the noise image.
In particular, if successive multi-frame video image frames in a video are denoised, then when video image frame u t When the denoising operation of (a) is completed, the video image frame u t+l The previous inter-frame accurate displacements have been obtained for video image frame u t +1 When denoising is performed, only the video image frame u is needed t+l+1 Performing a representative feature block lookup to determine a video image frame u t+l And u t +l+1 Inter-frame displacement between the two, the algorithm will further reduce the amount of denoising calculation for each video image frame.
Experimental results show that through the technical scheme, effective blind denoising can be performed on the mine shaft video image up to the noise level, and compared with the existing image denoising algorithm, the algorithm has higher denoising signal-to-noise ratio and better aging.
The specific embodiments described herein are offered by way of example only. Those skilled in the art may make various modifications or additions to the described embodiments or substitutions thereof without departing from the spirit of the invention or exceeding the scope of the invention as defined in the accompanying claims.
Claims (5)
1. The blind denoising method for the video image in the mine shaft environment is characterized by comprising the following steps of:
step 1, searching n typical characteristic image blocks on an image to be denoised, and constructing a typical characteristic block group according to a block group structure;
the implementation manner of the step 1 is that,
for image blocks with given size, searching n image blocks with the mean square error larger than a given threshold as typical characteristic blocks, and constructing n typical characteristic block groups according to a predefined block group structure;
let the video image to be denoised at time t be u t The video image of the previous frame is u t-1 Determining video image frame u t And u t-1 Image u at inter-frame displacement of (2) t Each image block of (a) has a size of N 2 ×N 2 Video image frame u t Image block to be distinguished with upper left corner coordinate at x
Normalizing to generate->
Mean>
Sum of mean square error->
The method comprises the following steps:
wherein Ω x is a block of an image
The coordinate set of all pixels in the image block, k 'is the element coordinate in the image block, k' is E.OMEGA.x, a threshold value Ts is given, and the image block to be distinguished is +.>
Satisfy->
Is identified as a typical characteristic image block in the video image u t Middle seekingStopping when n typical characteristic image blocks are found;
taking each typical characteristic block as a center, and taking the neighborhood image blocks which are directly adjacent to the block group and have the same size as the characteristic block group together according to the block group structure;
step 2, calculating the accurate displacement between video frames by using the typical characteristic block group obtained in the step 1;
step 3, constructing a self-similar sequence set for each image block in the noise image frame according to the inter-frame accurate displacement calculated in the step 2;
step 4, estimating the noise level of the noise image, and calculating the self-adaptive threshold value of the three-dimensional transformation coefficient on the basis of the noise level;
and 5, filtering the self-similar sequence set by using three-dimensional transformation and three-dimensional inverse transformation based on the self-adaptive threshold according to the similar sequence set obtained in the step 3 to obtain a corresponding three-dimensional image block matrix, and carrying out weighted fusion on the three-dimensional image block matrix to obtain a final denoising image.
2. The blind denoising method for video images in a mine shaft environment according to claim 1, wherein the method comprises the following steps: the implementation manner of the step 2 is that,
by video image u in the previous frame t-1 In search and current image frame u t The block group positioning inter-frame accurate displacement with minimum difference degree of typical characteristic block group, and the obtained accurate displacement is recorded as
Set video image frame u t Comprises N 1 ×N 1 Reference image block of individual pixels
And video image frame u at the kth time k Image block->
Degree of difference between->
The method comprises the following steps:
wherein the subscript xR is a reference image block
The upper left pixel is at u t In, the subscript x is the image block +.>
The upper left pixel is at u k Two-dimensional coordinates of (a); />
Is a two-dimensional Euclidean distance; t (T) 2D Is a two-dimensional discrete cosine DCT transform; />
Then->
And->
Is distinguished as a similar image block, T m Refers to a given degree of variance threshold;
let the upper left pixel coordinate of the 1 st found representative feature image block be x 10 Estimating delta from an initial given inter-frame shift ini Mapped to the previous video image frame u t-1 The coordinate of (a) is x 10 +Δ ini The method comprises the steps of carrying out a first treatment on the surface of the In video frame u t-1 In the form of the coordinate x 10 +Δ ini Is centered and has a width of 2Δ ini Is a square neighborhood of (c)
In, searching the image block group with the smallest difference degree with the n typical characteristic image block groups, and determining the accurate displacement between frames>
The method comprises the following steps:
wherein x is ij The pixel coordinate of the upper left corner of the image block with the sequence number j in the ith characteristic block group; n is the number of typical characteristic image block groups in the image; m is the number of image blocks in each block group; w (w) ij For each block group i, the block distance d of the typical feature image block is the distance weight of the image block with the serial number j in the ith feature block group i1 =0, and the block group is directly adjacent to the top, bottom, left and right image block distance d ij Block distance at four corners =1
3. The blind denoising method for video images in a mine shaft environment according to claim 2, wherein the method comprises the following steps: the implementation manner of the step 3 is that,
frame u of video image t With step length N step Divided into N 1 ×N 1 Image block with size is used as reference block, and u is used as reference block t Centered video image u of continuous 2l+1 frames t-l ,…,u t ,…,u t+l Sequentially searching the self-similar image blocks of each reference block and constructing a corresponding self-similar sequence set;
suppose that the video is mined byThe collecting device collects relative uniform motion from top to bottom through displacement between adjacent image frames
Fast determination of video image frame u t Each reference picture block of (a)>
Corresponding similar image blocks in adjacent 2l+1 image frames, each reference image block +.>
Self-similar sequence set->
The method comprises the following steps:
wherein x is the upper left corner coordinate of the corresponding similar image block in the image frame;
stored are 2l+1 video image frames and reference image blocks +.>
The upper left element coordinates and the image frame number of the similar image block, +.>
Represented as
The number of similar blocks in the self-similar sequence set of (a) is +.>
4. A method for blind denoising of video images in a mine wellbore environment according to claim 3, wherein: the implementation manner of the step 4 is that,
calculating a three-dimensional transformation adaptive threshold value, and calculating a video image frame u by using a typical characteristic block group and image frame displacement t And u t-1 The difference degree between the two is used for estimating the noise level of the noise image, and the calculation process is as follows:
inter-frame accurate displacement when 2l+1 frames
After being determined, the video image frame u is obtained t And inter-frame displacement of other adjacent 2l video image frames, and average inter-frame difference of adjacent 2l video image frames:
by calculating threshold lambda corresponding to video image frames with different noise variances in optimal denoising effect 3D And corresponding average inter-frame disparity dist t By a pair lambda 3D And dist t Fitting to obtain self-adaptive threshold lambda 3D ：
λ 3D ＝min(0.1824×(dist t ) 2 +0.7285×dist t +0.0394,6.386)。
5. The blind denoising method for video images in a mine shaft environment according to claim 4, wherein the method comprises the following steps: the specific implementation manner of the step 5 is that,
for reference blocks
Self-similar sequence set->
If->
Denoising the image block by adopting a discrete cosine denoising algorithm based on a threshold value, otherwise, carrying out threshold value processing on the transformed coefficient by adopting 3D filtering transformation, so as to weaken noise interference; will->
Corresponding image blocks of the image are combined into a three-dimensional image matrix +.>
Three-dimensional matrix after 3D transformation and inverse transformation>
The method comprises the following steps:
wherein T is 3D For a 3D transformation operator, adopting 3D Fourier transformation;
is a 3D inverse transform operator; gamma is based on a threshold lambda 3D Transform operator of (2) to reduce 3D transform coefficients to less than a threshold lambda 3D Is set to 0 and the 3D transform coefficient is greater than the threshold lambda 3D The coefficients of (2) remain unchanged;
three-dimensional image block matrix
Is composed of->
A 3D filtered two-dimensional image block, each of the filtered two-dimensional image blocks being denoted +.>
The subscript sn denotes the frame number in which the two-dimensional image block is located, while each image block is +.>
The coordinates of all elements of (a) are mapped to and reference picture block +.>
The coordinates of the corresponding pixels are the same, i.e. each image block +.>
The upper left corner pixel coordinate of (2) is xR; every image block->
The weight of a pixel at any coordinate x within is defined as:
wherein the image block
Is->
Corresponding image blocks before filtering, fusing the filtered image blocks, and denoising the pixel gray level estimated value +.>
The definition is as follows:
wherein the method comprises the steps of
The frame number of the stored image block is shown, and X is the coordinate set of the noise image. />
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911242397.4A CN110992288B (en) | 2019-12-06 | 2019-12-06 | Video image blind denoising method used in mine shaft environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911242397.4A CN110992288B (en) | 2019-12-06 | 2019-12-06 | Video image blind denoising method used in mine shaft environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110992288A CN110992288A (en) | 2020-04-10 |
| CN110992288B true CN110992288B (en) | 2023-04-28 |
Family
ID=70090839
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911242397.4A Active CN110992288B (en) | 2019-12-06 | 2019-12-06 | Video image blind denoising method used in mine shaft environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110992288B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111583151B (en) * | 2020-05-09 | 2023-05-12 | 浙江大华技术股份有限公司 | Video noise reduction method and device, and computer readable storage medium |
| CN112055255B (en) * | 2020-09-15 | 2022-07-05 | 深圳创维－Rgb电子有限公司 | Shooting image quality optimization method and device, smart television and readable storage medium |
| CN112529854B (en) * | 2020-11-30 | 2024-04-09 | 华为技术有限公司 | Noise estimation method, device, storage medium and equipment |
| CN113055670B (en) * | 2021-03-08 | 2024-03-19 | 浙江裕瀚科技有限公司 | HEVC/H.265-based video coding method and system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103404148A (en) * | 2011-03-09 | 2013-11-20 | 日本电信电话株式会社 | Image processing method, image processing device, video encoding/decoding method, video encoding/decoding device, and programs therefor |
| CN106934775A (en) * | 2017-03-08 | 2017-07-07 | 中国海洋大学 | A kind of non local image de-noising method recovered based on low-rank |
| CN107172322A (en) * | 2017-06-16 | 2017-09-15 | 北京飞识科技有限公司 | A kind of vedio noise reduction method and apparatus |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9311690B2 (en) * | 2014-03-11 | 2016-04-12 | Adobe Systems Incorporated | Video denoising using optical flow |
-
2019
- 2019-12-06 CN CN201911242397.4A patent/CN110992288B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103404148A (en) * | 2011-03-09 | 2013-11-20 | 日本电信电话株式会社 | Image processing method, image processing device, video encoding/decoding method, video encoding/decoding device, and programs therefor |
| CN106934775A (en) * | 2017-03-08 | 2017-07-07 | 中国海洋大学 | A kind of non local image de-noising method recovered based on low-rank |
| CN107172322A (en) * | 2017-06-16 | 2017-09-15 | 北京飞识科技有限公司 | A kind of vedio noise reduction method and apparatus |
Non-Patent Citations (1)
| Title |
|---|
| Kostadin Dabov等.Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering.IEEE TRANSACTIONS ON IMAGE PROCESSING.2007,第16卷(第8期),摘要，正文II、III部分. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110992288A (en) | 2020-04-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110992288B (en) | Video image blind denoising method used in mine shaft environment | |
| CN111209952B (en) | Underwater target detection method based on improved SSD and migration learning | |
| JP7256902B2 (en) | Video noise removal method, apparatus and computer readable storage medium | |
| CN108257165B (en) | Image stereo matching method and binocular vision equipment | |
| CN109509163B (en) | FGF-based multi-focus image fusion method and system | |
| CN107730536B (en) | High-speed correlation filtering object tracking method based on depth features | |
| CN111612817A (en) | Target tracking method based on depth feature adaptive fusion and context information | |
| CN110189390B (en) | Monocular vision SLAM method and system | |
| CN111753693B (en) | Target detection method under static scene | |
| CN111652790A (en) | Sub-pixel image registration method | |
| CN115131351A (en) | Engine oil radiator detection method based on infrared image | |
| Abd Manap et al. | Disparity refinement based on depth image layers separation for stereo matching algorithms | |
| KR101870700B1 (en) | A fast key frame extraction method for 3D reconstruction from a handheld video | |
| CN108010044A (en) | A kind of method of video boundaries detection | |
| CN113421210B (en) | Surface point Yun Chong construction method based on binocular stereoscopic vision | |
| Zhang et al. | Local stereo matching: An adaptive weighted guided image filtering-based approach | |
| CN107945119B (en) | Method for estimating correlated noise in image based on Bayer pattern | |
| CN107248143B (en) | Depth image restoration method based on image segmentation | |
| CN107993193B (en) | Tunnel lining image splicing method based on illumination equalization and surf algorithm improvement | |
| RU2580466C1 (en) | Device for recovery of depth map of scene | |
| CN111160362B (en) | FAST feature homogenizing extraction and interframe feature mismatching removal method | |
| CN115439771A (en) | Improved DSST infrared laser spot tracking method | |
| CN112801903A (en) | Target tracking method and device based on video noise reduction and computer equipment | |
| RU2716311C1 (en) | Device for reconstructing a depth map with searching for similar blocks based on a neural network | |
| CN110430340B (en) | Method and system for reducing noise of pulse array signal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |