CN115034992A - Long-wave infrared image denoising method - Google Patents

Abstract

The invention discloses a long-wave infrared image denoising method, which comprises the following steps: decomposing a long-wave infrared image with high noise and low contrast acquired by long-wave infrared by using a singular value decomposition algorithm through SVD (singular value decomposition), zeroing the maximum singular value of a characteristic value matrix obtained by decomposition, calculating an average singular value, replacing the rest singular values except the head singular value with the average singular value, giving sequentially descending weight to each position of the singular value according to the position, reconstructing the characteristic value matrix, reconstructing the long-wave infrared image matrix, and finally, further denoising the reconstructed long-wave infrared image by using a median filtering algorithm. In the aspect of image processing, the invention overcomes the defects of noise and low contrast of the long-wave infrared image per se; in the aspect of feature processing, the method overcomes the problems of feature mis-tracking and the like caused by the self defects of the long-wave infrared image, and provides a feasible scheme for the subsequent SLAM method to operate in a low-light environment.

Description

A method for denoising of long-wave infrared images

technical field

The invention belongs to the technical field of image processing, and in particular relates to a long-wave infrared image denoising method.

Background technique

Vision-based Simultaneous Localization and Mapping (SLAM) system refers to using the environmental information obtained by the camera to estimate the current pose information of the carrier, and construct a map model of the explored area at the same time. With the development of mechanical hardware and computer vision, the types of cameras are increasing. Among them, visible light cameras are widely used in SLAM systems due to their low cost and light weight. At present, the research of visible light SLAM has achieved good results. When there is a stable light source, the SLAM system based on the visible light camera performs well, but when there are non-ideal environmental conditions such as illumination changes, no light source, or heavy fog, the positioning and mapping accuracy of the SLAM system based on the visible light camera will be affected. It is seriously affected, so that the task of simultaneous positioning and mapping cannot be completed.

Compared with visible-light cameras, long-wave infrared imaging can sense the infrared information in the environment, and the imaging results reflect the temperature distribution of the surrounding environment. It can image even in an environment without light at night, and is less affected by changes in light and bad weather. Therefore, SLAM based on long-wave infrared can effectively solve the problem of poor system robustness of visible light cameras under conditions such as no illumination and heavy fog. However, affected by the principle of long-wave infrared imaging, the imaging results will have problems such as low signal-to-noise ratio, lack of texture, and low contrast, resulting in feature extraction errors and feature tracking errors in the front-end of SLAM, which directly affects the performance of SLAM systems based on long-wave infrared. .

Therefore, in order to reduce the impact of the above shortcomings of the long-wave infrared on the performance of the SLAM system, the present invention proposes a long-wave infrared image denoising method, which uses the SVD method to decompose the long-wave infrared acquisition image, and processes the decomposed eigenvalue matrix to reduce the impact of noise. , enhance image contrast, improve image feature extraction and tracking accuracy, provide reliable data association for subsequent SLAM systems, and improve positioning accuracy and robustness.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to provide a long-wave infrared image denoising method, which can process high-noise and low-contrast long-wave infrared data in a weak light environment, and provide a data association basis for subsequent synchronous positioning and mapping functions.

The object of the present invention is achieved through the following technical solutions: a long-wave infrared image denoising method, comprising the following steps:

Step (1): In a weak light environment, a long-wave infrared camera C with a calibrated internal parameter is set up, and a long-wave infrared image I ₀ with noise interference is collected at time t ₀ , and the size of the image I ₀ is M×N, M and N are the height and width of the image, respectively;

Step (2): use the SVD method to decompose the long-wave infrared image I ₀ to obtain the corresponding eigenvalue matrix Σ, process the eigenvalue matrix Σ based on the definition of the SVD decomposition, reconstruct the long-wave infrared image I ₀ , and obtain the reconstructed long-wavelength infrared image Infrared image I ₁ ;

Step (3): use the median filtering method to process the reconstructed image I ₁ to obtain a denoised long-wave infrared image I ₂ .

Further, the process of processing the eigenvalue matrix Σ in step (2) is: set the largest singular value of the eigenvalue matrix Σ to zero, calculate the average singular value, and replace the remaining singular values except the first singular value with the average singular value, A new eigenvalue matrix is formed by assigning decreasing weights to each singular value according to its position.

Further, the process of processing the eigenvalue matrix Σ in step (2) is:

Using the SVD method to decompose the long-wave infrared image I ₀ to get:

Among them, the size of matrix U is M×M, which is composed of left singular vectors (Left Singular Vector, LSV) u _i ; the size of matrix V is N×N, which is composed of right singular vectors (Right Singular Vector, RSV) v _i ; matrix Σ The size is M×N, and its diagonal elements represent the singular values s _i of the image, and s _i are arranged in descending order on the diagonal of the eigenvalue matrix Σ from large to small, T represents the transpose operation; from the image matrix From an angle, the singular value _si represents the overall brightness of the image, and the singular vectors _ui and vi _represent the texture features in the image. At the same time, the largest singular value can reflect the feature information with a high proportion in the image, and the remaining singular values can reflect the image. Feature information with a low proportion of the medium;

Process the decomposed eigenvalue matrix Σ to make the largest singular value s ₁ =0, and calculate the average singular value s _avg

Use the average singular value to replace all singular values except the first singular value in the eigenvalue matrix Σ, and assign decreasing weights according to their positions to obtain a new eigenvalue matrix Σ′ composed of singular values s′ _i

Use the new eigenvalue matrix Σ′ to reconstruct the image matrix to obtain the reconstructed long-wave infrared image I ₁

I ₁ =[U][Σ′][V] ^T . (4)

Further, the process of median filtering processing image I ₁ in step (3) is:

Suppose the size of the sliding window is K×K, the window slides on the image I ₁ by column, and the processed pixel value p _i ′ in the window is

为图像I₁原始像素值，Median函数为取集合中位数；该窗口按列在整幅图像上滑动，中值处理窗口内像素值，得到处理后的图像I₂。Among them, the collection

is the original pixel value of the image I ₁ , and the Median function is to take the median of the set; the window slides on the entire image in columns, and the median value processes the pixel values in the window to obtain the processed image I ₂ .

The beneficial effects of the present invention are as follows: the present invention utilizes the singular value decomposition algorithm, decomposes the high-noise and low-contrast long-wave infrared images collected by the long-wave infrared through SVD, sets the maximum singular value of the decomposed eigenvalue matrix to zero, and calculates the average singular value , replace the remaining singular values except the first singular value with the average singular value, give each singular value a weight in descending order according to its position, reconstruct the eigenvalue matrix, and then reconstruct the long-wave infrared image matrix, and finally, in use The value filtering algorithm further denoises the reconstructed LWIR image. In the aspect of image processing, the present invention overcomes the shortcomings of high noise and low contrast of the long-wave infrared image itself; in the aspect of feature processing, the present invention overcomes the problem of feature mistracking caused by the defects of the long-wave infrared image itself, and provides a better solution for the subsequent SLAM method in weak Operation in a light environment provides a feasible solution.

Description of drawings

In order to illustrate the technical solutions of the present invention more clearly, the following drawings will briefly describe the drawings used in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be It is regarded as a limitation on the protection scope of the present invention. In the various figures, similar components are numbered similarly.

1 is a flowchart of a long-wave infrared image denoising method;

Fig. 2 is the specific algorithm flow chart of the SVD reconstruction long-wave infrared image in the process of using a long-wave infrared image denoising method;

3 is a flowchart of a specific algorithm of median filtering during the use of a long-wave infrared image denoising method;

Fig. 4 is the optical flow tracking effect diagram of the original long-wave infrared image;

FIG. 5 is an effect diagram of optical flow tracking of the reconstructed long-wave infrared image according to the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments.

The components of the embodiments of the invention generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Thus, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but is merely representative of selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present invention.

Hereinafter, various embodiments of the present disclosure will be described more fully. The present disclosure is capable of various embodiments, and adaptations and changes may be made therein. It should be understood, however, that there is no intention to limit the various embodiments of the present disclosure to the specific embodiments disclosed herein, but the present disclosure should be construed to cover various embodiments falling within the spirit and scope of the present disclosure. All adjustments, equivalents and/or alternatives.

Hereinafter, the terms "comprising", "having" and their cognates, which may be used in various embodiments of the present invention, are only intended to denote particular features, numbers, steps, operations, elements, components, or combinations of the foregoing, and should not be construed as first excluding the presence of or adding one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing or the possibility of a combination of the foregoing.

Furthermore, the terms "first", "second", "third", etc. are only used to differentiate the description and should not be construed to indicate or imply relative importance.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which various embodiments of this invention belong. The terms (such as those defined in commonly used dictionaries) will be interpreted as having the same meaning as the contextual meaning in the relevant technical field and will not be interpreted as having an idealized or overly formal meaning, unless explicitly defined in the various embodiments of the present invention.

The present invention provides a long-wave infrared image denoising method. In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments. FIG. 1 is a schematic flowchart of a long-wave infrared image denoising method provided in an embodiment, and the method includes:

Step 110 : In a harsh environment such as weak light such as heavy fog, a long-wave infrared camera C with well-calibrated internal parameters is set up, and a long-wave infrared image I ₀ with noise interference is collected at time t ₀ , and the size of the image I ₀ is 240×640 ;

Step 120: Use the SVD method to decompose the long-wave infrared image I ₀ to obtain a corresponding eigenvalue matrix Σ, process the eigenvalue matrix Σ based on the definition of the SVD decomposition, and reconstruct the long-wave infrared image I ₀ to obtain a reconstructed long-wave infrared image. I ₁ ;

Step 130: Use the median filter method to process the reconstructed image I ₁ to obtain a denoised long-wave infrared image I ₂ .

Further, in step 120, the largest singular value of the eigenvalue matrix Σ is set to zero, the average singular value is calculated, the remaining singular values except the first singular value are replaced with the average singular value, and each singular value is assigned according to its location. The weights that decrease in turn form a new eigenvalue matrix, as shown in Figure 2, which includes the following sub-steps:

Step 121, using the SVD method to decompose the long-wave infrared image I ₀ to obtain:

Among them, the size of matrix U is 240×240, which is composed of left singular vectors (Left Singular Vector, LSV) _ui ; the size of matrix V is 320×320, which is composed of right singular vectors (Right Singular Vector, RSV) v _i ; matrix Σ The size is 240×320, and its diagonal elements represent the singular values s _i of the image, and s _i are arranged in descending order on the diagonal of the eigenvalue matrix Σ from large to small. From the perspective of the image matrix, the singular value _si represents the overall brightness of the image, and the singular vectors _ui and vi _represent the texture features in the image. At the same time, the largest singular value can reflect the feature information with a higher proportion in the image, and the remaining singular values It can reflect the feature information with a low proportion in the image.

Step 122: Process the decomposed eigenvalue matrix Σ to make the maximum singular value s ₁ =0.

Step 123, calculate the average singular value s _avg

Step 124: Use the average singular value to replace all singular values except the first singular value in the eigenvalue matrix Σ, and assign weights in descending order according to their positions, to obtain a new eigenvalue matrix Σ′ composed of singular values s _i ′

Step 125, using the new eigenvalue matrix Σ′ to reconstruct the image matrix, the reconstructed long-wave infrared image I ₁

I ₁ =[U][Σ′][V] ^T (4)

Further, in step 130, the process of median filtering processing the image _I1 is shown in FIG. 3, including:

Step 131, create a sliding window with a size of 3×3, the window slides on the image _I1 by column, and traverses the image.

Step 132: Calculate the processed pixel value p′ _i in the window

p′ _i =Median{p ₁ ,...,p ₉ },i=1,...,9 (5)

Among them, the set {p ₁ ,...,p ₉ } is the original pixel value of the image I ₁ in the window, and the Median function is the median of the set.

Step 133 , the window slides on the entire image in columns, and the median value of the pixels in the window is processed, and after the traversal is completed, exits, and the processed image I ₂ is obtained.

In order to further illustrate the effectiveness of the present invention in denoising, the original image I ₀ and the reconstructed image I ₂ are rotated 10° clockwise at the same time, and the LK (Lucas Kanade) optical flow method is used to perform the denoising between the original image and the rotated image. Optical flow tracking, the results are shown in Figure 4 and Figure 5. The root mean square error RMSE is used to calculate the error of the tracking point, and the results are shown in the following table:

image Tracking point error/pixel I₀ 50.1044 I₂ 3.2068

The original long-wave infrared image I ₀ has a large error in feature tracking due to the problems of noise interference and low contrast, and the long-wave infrared image I ₂ processed by the present invention has a smaller error in feature tracking and has high precision and system robustness, The basic data association can be provided for the subsequent automatic driving to realize the function of positioning and mapping, which verifies the effectiveness of the method of the present invention.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included in the scope of the present invention. within the scope of protection.

Claims (4)

1. A long-wave infrared image denoising method is characterized by comprising the following steps:

step (1): under the weak light environment, a long-wave infrared camera C with a calibrated internal reference is arranged, and at t ₀ Constantly acquiring long-wave infrared image I with noise interference ₀ Picture I ₀ The size is M N, M and N are the height and width of the image respectively;

step (2): decomposition of long-wave infrared images I using SVD method ₀ Obtaining a corresponding characteristic value matrix sigma, processing the characteristic value matrix sigma based on the definition of SVD decomposition, and processing the long-wave infrared image I ₀ Reconstructing to obtain a reconstructed long-wave infrared image I ₁ ；

And (3): processing a reconstructed image I using a median filtering method ₁ Obtaining a de-noised long-wave infrared image I ₂ 。

2. The method of claim 1, wherein the processing of the eigenvalue matrix Σ in step (2) is: setting the maximum singular value of the eigenvalue matrix sigma to zero, calculating the average singular value, replacing the singular values except the first singular value with the average singular value, and giving sequentially decreasing weight to each singular value according to the position to form a new eigenvalue matrix.

3. The method of claim 2, wherein the processing of the eigenvalue matrix Σ in step (2) is:

decomposition of long-wave infrared images I using SVD method ₀ Obtaining:

wherein,the matrix U has a size of M × M and is composed of left singular vectors U _i Composition is carried out; the matrix V is NXN in size and is composed of right singular vectors V _i Composition is carried out; the matrix Σ has a size mxn and diagonal elements representing the singular values s of the image _i ，s _i Arranging the characteristic value matrixes sigma diagonal in sequence from large to small according to a descending order, wherein T represents transposition operation; from the perspective of the image matrix, singular values s _i Representing the brightness of the image as a whole, singular vectors u _i 、v _i Representing texture features in the image, wherein the maximum singular value can reflect feature information with high ratio in the image, and the rest singular values can reflect feature information with low ratio in the image;

processing the eigenvalue matrix sigma obtained by decomposition to make the maximum singular value s ₁ 0 and calculating the mean singular value s _avg

Replacing all singular values except the first singular value in the eigenvalue matrix sigma with the average singular value, and giving sequentially decreasing weights according to the positions to obtain singular values s' _i Constructed new eigenvalue matrix sigma'

Using the new characteristic value matrix sigma' to reconstruct an image matrix to obtain a reconstructed long-wave infrared image I ₁

I ₁ ＝[U][Σ′][V] ^T 。 (4)

4. The method of claim 1, wherein the image I is median filtered in step (3) ₁ The process comprises the following steps:

with a sliding window of size K x K, which is arranged in the image I ₁ Up-sliding, in-window processed pixel value p' _i Is composed of

Wherein, aggregate

As an image I ₁ Original pixel value, Median function is to take Median in the set; the window is slid on the whole image according to columns, the pixel values in the window are processed by the median value to obtain a processed image I ₂ 。

Images