CN114757854A - Night vision image quality improving method, device and equipment based on multispectral analysis - Google Patents
Night vision image quality improving method, device and equipment based on multispectral analysis Download PDFInfo
- Publication number
- CN114757854A CN114757854A CN202210674311.0A CN202210674311A CN114757854A CN 114757854 A CN114757854 A CN 114757854A CN 202210674311 A CN202210674311 A CN 202210674311A CN 114757854 A CN114757854 A CN 114757854A
- Authority
- CN
- China
- Prior art keywords
- night vision
- images
- image set
- sparse
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000004297 night vision Effects 0.000 title claims abstract description 235
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000004458 analytical method Methods 0.000 title claims abstract description 36
- 238000012545 processing Methods 0.000 claims abstract description 106
- 238000011084 recovery Methods 0.000 claims abstract description 53
- 230000003595 spectral effect Effects 0.000 claims abstract description 29
- 238000001914 filtration Methods 0.000 claims abstract description 25
- 238000012549 training Methods 0.000 claims abstract description 18
- 230000000903 blocking effect Effects 0.000 claims abstract description 15
- 239000011159 matrix material Substances 0.000 claims description 44
- 230000006870 function Effects 0.000 claims description 26
- 238000005457 optimization Methods 0.000 claims description 26
- 238000004422 calculation algorithm Methods 0.000 claims description 22
- 230000006872 improvement Effects 0.000 claims description 13
- 238000000638 solvent extraction Methods 0.000 claims description 12
- 230000003044 adaptive effect Effects 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 7
- 238000010606 normalization Methods 0.000 claims description 6
- 238000010183 spectrum analysis Methods 0.000 claims description 6
- 238000000354 decomposition reaction Methods 0.000 claims description 4
- 230000004927 fusion Effects 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 4
- 230000017105 transposition Effects 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 10
- 238000013473 artificial intelligence Methods 0.000 abstract description 7
- 230000008569 process Effects 0.000 description 10
- 238000004891 communication Methods 0.000 description 7
- 238000005286 illumination Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 230000009467 reduction Effects 0.000 description 6
- 238000007726 management method Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 239000003086 colorant Substances 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 239000004973 liquid crystal related substance Substances 0.000 description 4
- 238000013507 mapping Methods 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000005070 sampling Methods 0.000 description 4
- 241001270131 Agaricus moelleri Species 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 210000001525 retina Anatomy 0.000 description 2
- 238000002834 transmittance Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004438 eyesight Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention relates to an artificial intelligence technology, and discloses a night vision image quality improving method based on multispectral analysis, which comprises the following steps: the method comprises the steps of blocking images in an original night vision image set, conducting self-adaptive training by using blocked night vision images to obtain a local sparse structure dictionary, conducting sparse de-noising on the images in the original night vision image set by using the local sparse structure dictionary to obtain a night vision de-noising image set, conducting spectral filtering processing and color enhancement processing on the images in the night vision de-noising image set to obtain an enhanced image set, conducting color recovery processing on the images in the enhanced image set to obtain a standard night vision image set, and conducting sub-histogram equalization processing on the images in the standard night vision image set to obtain the enhanced night vision image set. The invention further provides a night vision image quality improving device and equipment based on the multispectral analysis. The invention can improve the night vision image quality.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a night vision image quality improving method and device based on multispectral analysis, electronic equipment and a computer-readable storage medium.
Background
With the advancement of science and technology and the development of society, monitoring systems are applied to various fields and play an increasingly important role. The demand for monitoring systems is increasing from military to prison, from banks to shopping malls, from schools to residential areas and even every street. Meanwhile, with the increase of the night activity time of people, the information embodied in low-illumination and even dim light environment is more and more important. The low-light night vision image is mainly characterized in that: 1. the image has low gray level mean value, low signal-to-noise ratio and low contrast ratio, so that the image quality is low; 2. the gray level distribution of the image is concentrated, the whole frame of image is usually concentrated in the gray level of dozens to hundreds, and the means for improving the image quality is less.
Disclosure of Invention
The invention provides a night vision image quality improving method and device based on multispectral analysis, an electronic device and a readable storage medium, and mainly aims to improve the night vision image quality.
In order to achieve the above object, the present invention provides a night vision image quality improving method based on multispectral analysis, which includes:
acquiring an original night vision image set, blocking images in the original night vision image set, and performing adaptive training by using the blocked night vision images to obtain a local sparse structure dictionary;
performing sparse denoising on the images in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set;
performing spectral filtering processing and color enhancement processing on the images in the night vision denoising image set to obtain an enhanced image set;
performing color recovery processing on the images in the enhanced image set to obtain a standard night vision image set;
and performing sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set.
Optionally, the blocking the images in the original night vision image set, and performing adaptive training by using the blocked night vision images to obtain a local sparse structure dictionary includes:
partitioning the images in the original night vision image set according to a preset partitioning size to obtain a partitioned image set;
calculating a neighborhood reconstruction weight matrix of the block images in the block image set;
and performing iterative optimization on the neighborhood reconstruction weight matrix based on a K-LSPSc algorithm to obtain a sparse coefficient, and constructing a local sparse structure dictionary based on the sparse coefficient.
Optionally, the performing iterative optimization on the neighborhood reconstruction weight matrix based on the K-LSPSc algorithm to obtain a sparse coefficient, and constructing a local sparse structure dictionary based on the sparse coefficient includes:
acquiring a preset sparse dictionary, fixing the sparse dictionary, and performing iterative optimization on the neighborhood reconstruction weight matrix by using a preset alternating optimization formula to obtain a sparse coefficient matrix;
and fixing the sparse coefficient matrix, and updating the sparse dictionary by using a preset dictionary updating formula to obtain a local sparse structure dictionary.
Optionally, the performing sparse denoising on the image in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set includes:
carrying out sparse representation on images in the block image set by using the local sparse structure dictionary to obtain a sparse image set;
carrying out quadratic term solving on the images in the sparse image set by using the following formula to obtain the night vision de-noising image set:
wherein,for the set of night vision de-noised images,is a constant of experience and is,is a matrix of the unit, and is,is a square window operator, and is a square window operator,a set of block images is represented as a set of block images,for the purpose of the local sparse structure dictionary,the transpose is represented by,is a firstSparse coefficients of the individual center block images.
Optionally, the performing spectral filtering and color enhancement processing on the images in the night vision de-noising image set to obtain an enhanced image set includes:
performing spectral filtering processing on the images in the night vision denoising image set to obtain a plurality of three-primary-color images;
and performing band screening on the three primary color images by using a preset band range to obtain a plurality of band images, performing image fusion on the plurality of band images, and summarizing all fused images to obtain the enhanced image set.
Optionally, the performing color recovery processing on the images in the enhanced image set to obtain a standard night vision image set includes:
carrying out color channel decomposition on the images in the enhanced image set to obtain images with different color channels;
carrying out multi-scale enhancement processing on the image of each color channel by using an MSRCR algorithm to obtain a multi-scale enhanced image;
calculating the color recovery coefficient of the multi-scale enhanced image, and performing color recovery on the multi-scale enhanced image of different color channels based on the color recovery coefficient to obtain color recovery images of different color channels;
and fusing the color recovery images of different color channels to obtain the standard night vision image set.
Optionally, the calculating the color recovery coefficient of the multi-scale enhanced image includes:
calculating a color recovery coefficient of the multi-scale enhanced image using the following formula:
wherein,is as followsThe color recovery coefficients of the individual color channels,is a constant value of the gain, and is,is a non-linear coefficient of the linear vibration,is as followsThe image of each of the color channels is,is as followsColor recovery map for individual color channelsLike the image of the eye(s) to be,in order to enhance the image at a multi-scale,is as followsThe weight of each of the scales is determined,is a function of the surrounding of the gaussian,represents convolution operation, and D represents the number of scales.
Optionally, the performing sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set includes:
calculating original histograms of images in the standard night vision image set one by one, and partitioning the original histograms to obtain a plurality of sub-histograms;
carrying out gray dynamic adjustment on the plurality of sub-histograms to obtain a plurality of adjusted sub-histograms;
performing gray level balance enhancement processing on the plurality of adjustment sub-histograms based on a gray level probability density function to obtain a plurality of standard sub-histograms;
and carrying out normalization processing and histogram equalization processing on each standard sub-histogram, and summarizing all the images subjected to histogram equalization processing to obtain the enhanced night vision image set.
In order to solve the above problem, the present invention further provides a night vision image quality improving apparatus based on multispectral analysis, the apparatus comprising:
the local dictionary building module is used for acquiring an original night vision image set, blocking images in the original night vision image set, and performing self-adaptive training by using the blocked night vision images to obtain a local sparse structure dictionary;
the sparse denoising module is used for performing sparse denoising on the images in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set;
the color enhancement module is used for performing spectral filtering processing and color enhancement processing on the images in the night vision de-noising image set to obtain an enhanced image set;
the color recovery module is used for performing color recovery processing on the images in the enhanced image set to obtain a standard night vision image set;
and the night vision image enhancement module is used for carrying out sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set.
In order to solve the above problem, the present invention also provides an electronic device, including:
a memory storing at least one computer program; and
and the processor executes the computer program stored in the memory to realize the night vision image quality improvement method based on the multispectral analysis.
According to the method, the images in the original night vision image set are subjected to self-adaptive training to obtain the local sparse structure dictionary, the images in the original night vision image set are subjected to sparse denoising based on the local sparse structure dictionary, and the noise can be effectively inhibited while the image details are kept by utilizing the noise invariant characteristic of the local sparse structure, so that the denoising effect is improved. Meanwhile, multispectral analysis and color enhancement are carried out on the images in the night vision denoising image set based on spectral filtering processing and color enhancement processing, the contrast ratio of the night vision images is improved, and the authenticity of the night vision images is also improved by carrying out color recovery processing. And finally, the night vision image is distributed more uniformly through sub-histogram equalization processing, the original image details are further kept, and the night vision image quality is improved. Therefore, the night vision image quality improving method and device based on multispectral analysis, the electronic device and the computer readable storage medium can improve the night vision image quality.
Drawings
Fig. 1 is a schematic flowchart of a night vision image quality improving method based on multispectral analysis according to an embodiment of the present invention;
FIG. 2 is a functional block diagram of a night vision image quality improving apparatus based on multi-spectral analysis according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an electronic device implementing the night vision image quality improvement method based on multispectral analysis according to an embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.
The embodiment of the application provides a night vision image quality improving method based on multispectral analysis. The execution subject of the night vision image quality improvement method based on multispectral analysis includes, but is not limited to, at least one of the electronic devices of a server, a terminal, and the like, which can be configured to execute the method provided by the embodiments of the present application. In other words, the night vision image quality improvement method based on multispectral analysis may be executed by software or hardware installed in a terminal device or a server device, and the software may be a block chain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like. The server may be an independent server, or may be a cloud server that provides basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a Content Delivery Network (CDN), a big data and artificial intelligence platform, and the like.
Referring to fig. 1, a flow chart of a night vision image quality improvement method based on multi-spectral analysis according to an embodiment of the invention is shown. In this embodiment, the method for improving the quality of the night vision image based on the multispectral analysis includes:
s1, acquiring an original night vision image set, blocking the images in the original night vision image set, and performing self-adaptive training by using the blocked night vision images to obtain a local sparse structure dictionary.
In the embodiment of the invention, the original night vision image set is an image acquired by low-light-level night vision equipment or a low-light-level night vision system under a low-light-level illumination condition, such as a low-light-level photo shot by a monitoring camera under a night condition.
In detail, the blocking the image in the original night vision image set, and performing adaptive training by using the blocked night vision image to obtain a local sparse structure dictionary includes:
partitioning the images in the original night vision image set according to a preset partitioning size to obtain a partitioned image set;
calculating a neighborhood reconstruction weight matrix of the block images in the block image set;
and performing iterative optimization on the neighborhood reconstruction weight matrix based on a K-LSPSc algorithm to obtain a sparse coefficient, and constructing a local sparse structure dictionary based on the sparse coefficient.
If the image block is smaller, the local structure has no representation, and if the image block is larger, the sparsity of extracted features is weaker. Therefore, in the embodiment of the present invention, in order to achieve both image sparse information and structural features, the size of each image block is set to 8 × 8.
Further, the calculating a neighborhood reconstruction weight matrix of the block images in the block image set includes:
step A: randomly sampling the block image set to obtain a sample image set, and performing center marking and neighborhood marking on the block images in the sample image set to obtain a center block image and a neighborhood block image;
in the embodiment of the invention, in order to reduce the calculation amount, only part of the samples are used for training the dictionary, for example, 25% of the samples are randomly selected from all the block image sets. Due to random sampling, the number of non-overlapping neighboring blocks of each image block may be too small, and in order to ensure calculation of an accurate local structural relationship, the embodiment of the present invention centers on each block image and marks 64 neighboring block images that are nearest to each image block.
And B: calculating neighborhood reconstruction weight by using optimization function from neighborhood block image to center block image of each mark;
In one embodiment of the present invention, the optimization function can be expressed as:
wherein,blocking images for each neighborhood,Is as followsEach central block image, wherein N is the number of block images;the weight matrix is reconstructed for the neighborhood and,represents the first in the neighborhood reconstruction weight matrixNeighborhood reconstruction weights for individual center block images,is shown asCenter block image andand (5) neighborhood reconstruction weights of the neighborhood block images.
Further, the performing iterative optimization on the neighborhood reconstruction weight matrix based on the K-LSPSc algorithm to obtain a sparse coefficient, and constructing a local sparse structure dictionary based on the sparse coefficient includes:
acquiring a preset sparse dictionary, fixing the sparse dictionary, and performing iterative optimization on the neighborhood reconstruction weight matrix by using a preset alternative optimization formula to obtain a sparse coefficient matrix;
and fixing the sparse coefficient matrix, and updating the sparse dictionary by using a preset dictionary updating formula to obtain a local sparse structure dictionary.
In an optional embodiment of the present invention, the preset sparse dictionary is a template dictionary learned by using a template image set. And fixing the sparse dictionary to determine a fixed learning dictionary, and further performing optimization updating on the sparse structure.
In an optional embodiment of the present invention, the preset alternating optimization formula is:
wherein,is as followsSparse reconstruction error terms for individual center block images,is as followsThe sparse coefficients of the individual center block images,is as followsA center block image, B is the preset sparse dictionary,reconstructing a weight matrix for the neighborhoodCenter block image andneighborhood reconstruction weights for individual neighborhood block images,to optimize sparse coefficientsThe remaining sparse coefficients are fixed in time,、in order to optimize the parameters for the fixation,is a mapping function, wherein, in the embodiment of the present invention, the mapping functionMay be a radial basis kernel function.
The invention is optionalIn an embodiment, each is optimized by the alternating optimization formulaWhen the constraint condition is satisfiedAnd meanwhile, summarizing all sparse coefficients to obtain a total sparse coefficient matrix S.
Further, in an optional embodiment of the present invention, the preset dictionary update formula is:
wherein B is the preset sparse dictionary,the transpose is represented by,is a matrix of the units,a matrix of sparse coefficients is represented by a matrix of,representing a set of block images.
In an optional embodiment of the invention, the dictionary is continuously updated by using a preset dictionary updating formula until the constraint condition of the dictionary is metAnd obtaining the local sparse structure dictionary.
S2, performing sparse denoising on the images in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set.
In the embodiment of the invention, a and b are used for representing a pure signal image and a noise-containing image, c is superimposed noise, the noise-containing image can be represented as b ═ a + c, and image denoising is to reconstruct a from b.
In detail, the sparse denoising of the image in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set includes:
carrying out sparse representation on images in the block image set by using the local sparse structure dictionary to obtain a sparse image set;
carrying out quadratic term solving on the images in the sparse image set by using the following formula to obtain a night vision de-noising image set:
wherein,for the set of night vision de-noised images,is a constant of experience and is,is a matrix of the units,is a square window operator, and is characterized in that,a set of block images is represented as a set of block images,for the purpose of the local sparse structure dictionary,the transpose is represented by,is as followsSparse coefficients of the individual center block images.
In the embodiment of the invention, the existing sparse noise reduction method depends on Gaussian noise distribution hypothesis, the problem of noise redundancy removal of complex distribution in low-light-level images cannot be well solved, image details are difficult to effectively retain under strong noise interference, a local sparse structure dictionary trained by a K-LSPSc algorithm can reduce noise interference and improve robustness of sparse representation of image blocks, and meanwhile, the noise is effectively inhibited and the image details are retained by utilizing the noise invariant characteristic of a local sparse structure, so that the noise reduction effect is improved.
S3, performing spectral filtering processing and color enhancement processing on the images in the night vision de-noising image set to obtain an enhanced image set.
In the embodiment of the invention, because the illumination in the night vision image is weak, the color of the image is seriously distorted, the spectral filtering processing is carried out by the optical filters with a plurality of wave bands, and meanwhile, because the images with different wave bands are concentrated in different colors, the colors can be enhanced by fusing the images with different wave bands, and the image reality is improved.
Specifically, the performing spectral filtering processing and color enhancement processing on the image in the night vision denoising image set to obtain an enhanced image set includes:
performing spectral filtering processing on the images in the night vision denoising image set to obtain a plurality of three primary color images;
and performing band screening on the three primary color images by using a preset band range to obtain a plurality of band images, performing image fusion on the plurality of band images, and summarizing all fused images to obtain the enhanced image set.
In an optional embodiment of the invention, the images in the night vision de-noising image set can be decomposed into R, G, B three primary color images by using spectral filters with different spectral transmittances. The image of each primary color has images of different wave bands, and the short-wave image (380-480 nm), the medium-wave image (510-565 nm) and the long-wave image (590-650 nm) are extracted from each primary color image sequence in the embodiment of the invention and are fused to obtain the enhanced image.
And S4, performing color recovery processing on the images in the enhanced image set to obtain a standard night vision image set.
In the embodiment of the invention, because the illumination in the night vision image is weak, and the color in the image after spectral filtering processing and color enhancement processing is still weak, the color of the image needs to be restored, thereby improving the authenticity of the image.
In one embodiment of the present invention, the images in the enhanced image set may be color-restored using the MSRCR algorithm. The MSRCR (Multi-Scale Retinex with Color retrieval) algorithm introduces the proportional relation in the image into three Color channels to adjust the proportion of the MSR (Multi-Scale Retinex, Multi-Scale retina enhancement) result, thereby optimizing the Color of the image, avoiding Color unsaturation and distortion, and enabling the image to have better Color presentation.
In detail, the performing color recovery processing on the images in the enhanced image set to obtain a standard night vision image set includes:
carrying out color channel decomposition on the images in the enhanced image set to obtain images with different color channels;
performing multi-scale enhancement processing on the image of each color channel by using an MSRCR algorithm to obtain a multi-scale enhanced image;
calculating the color recovery coefficient of the multi-scale enhanced image, and performing color recovery on the multi-scale enhanced image of different color channels based on the color recovery coefficient to obtain color recovery images of different color channels;
and fusing the color recovery images of different color channels to obtain the standard night vision image set.
In an alternative embodiment of the present invention, the color recovery coefficient of the multi-scale enhanced image may be calculated by using the following formula:
wherein,is a firstThe color recovery coefficients of the individual color channels,is a constant value of the gain, and is,is a non-linear coefficient, and is,is as followsThe image of each of the color channels is,is as followsThe color of the individual color channels restores the image,in order to enhance the image at a multi-scale,is a firstThe weight of each scale is determined by the weight of each scale,is a function of the surrounding of the gaussian,represents convolution operation, and D represents the number of scales.
In the embodiment of the invention, three scales of large, medium and small are taken, and the weight of each scale is 1/3. When the MSR algorithm is used for processing the RGB color image, R, G, B channels are respectively and independently enhanced, so that the proportion of pixel values of pixel points in each color channel can be changed in the enhancing process, the real color of objects in the whole or specific area of the image is faded, and the accuracy of restoring the real color of the image is improved by increasing the color restoring coefficient.
And S5, performing sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set.
In the embodiment of the invention, the image processed by the histogram equalization method can well improve the contrast and definition, but the method for integrally improving the contrast cannot well process very bright and very dark places and is difficult to integrally control, so that the embodiment of the invention provides an improved histogram equalization algorithm, namely sub-histogram equalization, to perform image enhancement processing aiming at a narrow range.
In detail, the performing sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set includes:
calculating original histograms of images in the standard night vision image set one by one, and partitioning the original histograms to obtain a plurality of sub-histograms;
carrying out gray dynamic adjustment on the plurality of sub-histograms to obtain a plurality of adjusted sub-histograms;
performing gray level balance enhancement processing on the plurality of adjustment sub-histograms based on a gray level probability density function to obtain a plurality of standard sub-histograms;
and carrying out normalization processing and histogram equalization processing on each standard sub-histogram, and summarizing all the images subjected to histogram equalization processing to obtain the enhanced night vision image set.
In the embodiment of the invention, the original histogram of the image in the standard night vision image set can be divided into four sub-histograms, and the gray level intervals of the four sub-histograms are [ m0, m1 ] in sequence],[m1,m2],[m2,m3],[m3,m4]And using the following formula:and dynamically adjusting the gray scales of the plurality of sub-histograms. Wherein in the formulaRepresenting the gray level dynamic range of the sub-histogram,representing the gray level dynamic range of the adjusted sub-histogram,which is indicative of the adjustment factor(s),is the total number of gray levels of the image,indicating the number of sub-histograms.
According to the embodiment of the invention, the new adjusted histogram is obtained by adjusting the sub-histograms according to the set dynamic range, each sub-interval is dynamically stretched, and the possibility of gray level combination is reduced.
Further, in an optional embodiment of the present invention, the gray level equalization enhancement processing is performed on the plurality of adjustment sub-histograms through the following gray level probability density function:
wherein,for the gray level frequencies of the enhanced standard sub-histogram,is the median of the grey scale frequencies and,in order to control the factors, the control unit is provided with a control unit,is the minimum value of the gray-level frequency,is the maximum value of the frequency of the grey scale,the gray level frequencies of the sub-histograms are adjusted for the current unprocessed tone.
In an optional embodiment of the present invention, the gray level frequency of the standard sub-Histogram is further normalized, and then HE (Histogram Equalization) processing is performed, so as to enhance the night vision image. The normalization process and the HE process are well known in the art and are not described herein again.
In the embodiment, the images in the original night vision image set are subjected to self-adaptive training to obtain the local sparse structure dictionary, the images in the original night vision image set are subjected to sparse de-noising based on the local sparse structure dictionary, and the noise is effectively suppressed while the image details are kept by utilizing the noise invariant characteristic of the local sparse structure, so that the noise reduction effect is improved. Meanwhile, multispectral analysis and color enhancement are carried out on the images in the night vision de-noising image set based on spectral filtering processing and color enhancement processing, the contrast of the night vision image is improved, and the authenticity of the night vision image is also improved by carrying out color recovery processing. And finally, the night vision image is distributed more uniformly through sub-histogram equalization processing, the original image details are further kept, and the night vision image quality is improved. Therefore, the night vision image quality improving method based on multispectral analysis can improve the night vision image quality.
Fig. 2 is a functional block diagram of a night vision image quality improving apparatus based on multi-spectral analysis according to an embodiment of the present invention.
The night vision image quality improving apparatus 100 based on multispectral analysis of the present invention can be installed in an electronic device. According to the implemented functions, the night vision image quality improving device 100 based on multispectral analysis may include a local dictionary construction module 101, a sparse denoising module 102, a color enhancement module 103, a color recovery module 104, and a night vision image enhancement module 105. The module of the present invention, which may also be referred to as a unit, refers to a series of computer program segments that can be executed by a processor of an electronic device and that can perform a fixed function, and that are stored in a memory of the electronic device.
In the present embodiment, the functions of the respective modules/units are as follows:
the local dictionary building module 101 is configured to obtain an original night vision image set, block images in the original night vision image set, and perform adaptive training by using the blocked night vision images to obtain a local sparse structure dictionary;
the sparse denoising module 102 is configured to perform sparse denoising on the images in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set;
the color enhancement module 103 is configured to perform spectral filtering processing and color enhancement processing on the images in the night vision denoising image set to obtain an enhanced image set;
the color recovery module 104 is configured to perform color recovery processing on the images in the enhanced image set to obtain a standard night vision image set;
the night vision image enhancement module 105 is configured to perform sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set.
In detail, the implementation of the modules of the night vision image quality improving apparatus 100 based on multispectral analysis is as follows:
the method comprises the steps of firstly, obtaining an original night vision image set, blocking images in the original night vision image set, and carrying out self-adaptive training by using the blocked night vision images to obtain a local sparse structure dictionary.
In the embodiment of the invention, the original night vision image set is an image acquired by low-light-level night vision equipment or a low-light-level night vision system under a low-light-level illumination condition, such as a low-light-level photo shot by a monitoring camera under a night condition.
In detail, the blocking the image in the original night vision image set, and performing adaptive training by using the blocked night vision image to obtain a local sparse structure dictionary includes:
partitioning the images in the original night vision image set according to a preset partitioning size to obtain a partitioned image set;
calculating a neighborhood reconstruction weight matrix of the block images in the block image set;
and performing iterative optimization on the neighborhood reconstruction weight matrix based on a K-LSPSc algorithm to obtain a sparse coefficient, and constructing a local sparse structure dictionary based on the sparse coefficient.
If the image block is smaller, the local structure has no representation, and if the image block is larger, the sparsity of extracted features is weaker. Therefore, in the embodiment of the present invention, in order to make consideration of both image sparse information and structural features, the size of each image block is set to 8 × 8.
Further, the calculating a neighborhood reconstruction weight matrix of the block images in the block image set includes:
step A: randomly sampling the block image set to obtain a sample image set, and performing center marking and neighborhood marking on the block images in the sample image set to obtain a center block image and a neighborhood block image;
in the embodiment of the invention, in order to reduce the amount of calculation, the dictionary is trained by only using part of samples, for example, 25% of the samples are randomly selected from all the block image sets. Due to random sampling, the number of non-overlapping neighbor image blocks of each image block may be too small, and in order to ensure accurate calculation of the local structural relationship, the embodiment of the present invention centers on each block image and marks 64 neighbor block images of the nearest neighbor of each image block.
And B, step B: calculating neighborhood reconstruction weight by using optimization function from neighborhood block image to center block image of each mark;
In one embodiment of the present invention, the optimization function can be expressed as:
wherein,blocking images for each neighborhood,Is as followsEach central block image, wherein N is the number of block images;the weight matrix is reconstructed for the neighborhood and,represents the first in the neighborhood reconstruction weight matrixNeighborhood reconstruction weights for individual center block images,is shown asCenter block image andand (5) neighborhood reconstruction weights of the neighborhood block images.
Further, the performing iterative optimization on the neighborhood reconstruction weight matrix based on the K-LSPSc algorithm to obtain a sparse coefficient, and constructing a local sparse structure dictionary based on the sparse coefficient includes:
acquiring a preset sparse dictionary, fixing the sparse dictionary, and performing iterative optimization on the neighborhood reconstruction weight matrix by using a preset alternating optimization formula to obtain a sparse coefficient matrix;
and fixing the sparse coefficient matrix, and updating the sparse dictionary by using a preset dictionary updating formula to obtain a local sparse structure dictionary.
In an optional embodiment of the present invention, the preset sparse dictionary is a template dictionary learned by using a template image set. And fixing the sparse dictionary to determine a fixed learning dictionary, and further performing optimization updating on the sparse structure.
In an optional embodiment of the present invention, the preset alternating optimization formula is:
wherein,is as followsSparse reconstruction error terms for the individual center block images,is as followsThe sparse coefficients of the individual center block images,is as followsA center block image, B is the preset sparse dictionary,reconstructing a weight matrix for the neighborhoodCenter block image andneighborhood reconstruction weights for individual neighborhood block images,to optimize sparse coefficientsThe remaining sparse coefficients are fixed in time,、in order to optimize the parameters for the fixation,is a mapping function, wherein, in the embodiment of the present invention, the mapping functionMay be a radial basis kernel function.
In an alternative embodiment of the invention, each is optimized by said alternating optimization formulaWhen the constraint condition is satisfiedAnd meanwhile, summarizing all sparse coefficients to obtain a total sparse coefficient matrix S.
Further, in an optional embodiment of the present invention, the preset dictionary update formula is:
wherein B is the preset sparse dictionary,the transpose is represented by,is a matrix of the units,a matrix of sparse coefficients is represented by a matrix of,representing a set of block images.
In an optional embodiment of the present invention, the word is continuously updated using a predetermined dictionary update formulaDictionary until the dictionary constraint condition is satisfiedAnd obtaining the local sparse structure dictionary.
And secondly, performing sparse denoising on the images in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set.
In the embodiment of the invention, a and b are used for representing a pure signal image and a noise-containing image, c is superimposed noise, the noise-containing image can be represented as b ═ a + c, and image denoising is to reconstruct a from b.
In detail, the sparse denoising of the image in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set comprises:
carrying out sparse representation on images in the block image set by using the local sparse structure dictionary to obtain a sparse image set;
carrying out quadratic term solving on the images in the sparse image set by using the following formula to obtain a night vision de-noising image set:
wherein,for the set of night vision de-noised images,is a constant of experience and is,is a matrix of the units,is a square window operator, and is a square window operator,a set of block images is represented as a set of block images,for the purpose of the local sparse structure dictionary,the transpose is represented by,is as followsSparse coefficients of the individual center block images.
In the embodiment of the invention, the existing sparse noise reduction method depends on Gaussian noise distribution hypothesis, the problem of noise redundancy removal of complex distribution in low-light-level images cannot be well solved, image details are difficult to effectively retain under strong noise interference, a local sparse structure dictionary trained by a K-LSPSc algorithm can reduce noise interference and improve robustness of sparse representation of image blocks, and meanwhile, the noise is effectively inhibited and the image details are retained by utilizing the noise invariant characteristic of a local sparse structure, so that the noise reduction effect is improved.
And thirdly, performing spectral filtering processing and color enhancement processing on the images in the night vision denoising image set to obtain an enhanced image set.
In the embodiment of the invention, because the illumination in the night vision image is weak, the color of the image is seriously distorted, the spectral filtering processing is carried out by the optical filters with a plurality of wave bands, and meanwhile, because the images with different wave bands are concentrated in different colors, the colors can be enhanced by fusing the images with different wave bands, and the image reality is improved.
Specifically, the performing spectral filtering processing and color enhancement processing on the image in the night vision denoising image set to obtain an enhanced image set includes:
performing spectral filtering processing on the images in the night vision denoising image set to obtain a plurality of three primary color images;
and performing band screening on the three primary color images by using a preset band range to obtain a plurality of band images, performing image fusion on the plurality of band images, and summarizing all fused images to obtain the enhanced image set.
In an optional embodiment of the present invention, the images in the night vision de-noising image set may be decomposed into R, G, B three primary color images by using spectral filters with different spectral transmittances. The image of each primary color has images of different wave bands, and the short-wave image (380-480 nm), the medium-wave image (510-565 nm) and the long-wave image (590-650 nm) are extracted from each primary color image sequence in the embodiment of the invention and are fused to obtain the enhanced image.
And fourthly, performing color recovery processing on the images in the enhanced image set to obtain a standard night vision image set.
In the embodiment of the invention, because the illumination in the night vision image is weak, and the color in the image after the spectral filtering processing and the color enhancement processing is still weak, the color recovery needs to be carried out on the image, thereby improving the authenticity of the image.
In one embodiment of the present invention, the images in the enhanced image set may be color-restored using the MSRCR algorithm. The MSRCR (Multi-Scale Retinex with Color retrieval) algorithm introduces the proportional relation in the image into three Color channels to adjust the proportion of the MSR (Multi-Scale Retinex, Multi-Scale retina enhancement) result, thereby optimizing the Color of the image, avoiding Color unsaturation and distortion, and enabling the image to have better Color presentation.
In detail, the performing color recovery processing on the images in the enhanced image set to obtain a standard night vision image set includes:
carrying out color channel decomposition on the images in the enhanced image set to obtain images with different color channels;
carrying out multi-scale enhancement processing on the image of each color channel by using an MSRCR algorithm to obtain a multi-scale enhanced image;
calculating the color recovery coefficient of the multi-scale enhanced image, and performing color recovery on the multi-scale enhanced image of different color channels based on the color recovery coefficient to obtain color recovery images of different color channels;
and fusing the color recovery images of different color channels to obtain the standard night vision image set.
In an alternative embodiment of the present invention, the color recovery coefficient of the multi-scale enhanced image may be calculated by using the following formula:
wherein,is as followsThe color recovery coefficients of the individual color channels,is a constant value of the gain, and is,is a non-linear coefficient, and is,is as followsThe image of each of the color channels is,is a firstThe color of each color channel is restored to the image,in order to enhance the image at a multi-scale,is as followsThe weight of each scale is determined by the weight of each scale,is a function of the surrounding of the gaussian,denotes convolution operation, and D denotes scale degree.
In the embodiment of the invention, three scales of large, medium and small are taken, and the weight of each scale is 1/3. When the MSR algorithm is used for processing the RGB color image, R, G, B channels are respectively and independently enhanced, so that the proportion of pixel values of pixel points in each color channel can be changed in the enhancing process, the real color of objects in the whole or specific area of the image is faded, and the accuracy of restoring the real color of the image is improved by increasing the color restoring coefficient.
And fifthly, performing sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set.
In the embodiment of the invention, the contrast and the definition of the image processed by the histogram equalization method can be well improved, but the method for integrally improving the contrast cannot well process very bright and very dark places and is difficult to integrally control, so that the embodiment of the invention provides an improved histogram equalization algorithm, namely sub-histogram equalization, so as to perform image enhancement processing aiming at a narrow range.
In detail, the performing sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set includes:
calculating original histograms of images in the standard night vision image set one by one, and partitioning the original histograms to obtain a plurality of sub-histograms;
carrying out gray dynamic adjustment on the plurality of sub-histograms to obtain a plurality of adjusted sub-histograms;
performing gray level balance enhancement processing on the plurality of adjustment sub-histograms based on a gray level probability density function to obtain a plurality of standard sub-histograms;
and carrying out normalization processing and histogram equalization processing on each standard sub-histogram, and summarizing all the images subjected to histogram equalization processing to obtain the enhanced night vision image set.
In the embodiment of the invention, the original histogram of the image in the standard night vision image set can be divided into four sub-histograms, and the gray level intervals of the four sub-histograms are [ m0, m1 ] in sequence],[m1,m2],[m2,m3],[m3,m4]And using the following formula:and dynamically adjusting the gray scales of the plurality of sub-histograms. Wherein in the formulaRepresenting the gray level dynamic range of the sub-histogram,representing the gray level dynamic range of the adjusted sub-histogram,which is indicative of the adjustment factor(s),is the total number of gray levels of the image,indicating the number of sub-histograms.
According to the embodiment of the invention, the new adjusted histogram is obtained by adjusting the sub-histograms according to the set dynamic range, each sub-interval is dynamically stretched, and the possibility of gray level combination is reduced.
Further, in an optional embodiment of the present invention, the gray scale equalization enhancement processing is performed on the plurality of adjustment sub-histograms by using the following gray scale probability density function:
wherein,for the gray level frequencies of the enhanced standard sub-histogram,is the median value of the grey scale frequencies,in order to control the factors, the control unit is provided with a control unit,is the minimum value of the frequency of the grey level,is the maximum value of the frequency of the grey scale,the gray level frequencies of the sub-histograms are adjusted for the current unprocessed tone.
In an optional embodiment of the present invention, the gray scale frequency of the standard sub-Histogram is further normalized, and then HE (Histogram Equalization) processing is performed, so as to enhance the night vision image. The normalization process and the HE process are well known in the art and are not described herein again.
In the embodiment, the images in the original night vision image set are subjected to self-adaptive training to obtain the local sparse structure dictionary, the images in the original night vision image set are subjected to sparse de-noising based on the local sparse structure dictionary, and the noise is effectively suppressed while the image details are kept by utilizing the noise invariant characteristic of the local sparse structure, so that the noise reduction effect is improved. Meanwhile, multispectral analysis and color enhancement are carried out on the images in the night vision de-noising image set based on spectral filtering processing and color enhancement processing, the contrast of the night vision image is improved, and the authenticity of the night vision image is also improved by carrying out color recovery processing. And finally, the night vision image is distributed more uniformly through sub-histogram equalization processing, the original image details are further kept, and the night vision image quality is improved. Therefore, the night vision image quality improving device based on multispectral analysis can improve the night vision image quality.
Fig. 3 is a schematic structural diagram of an electronic device for implementing the night vision image quality improvement method based on multispectral analysis according to an embodiment of the present invention.
The electronic device may include a processor 10, a memory 11, a communication interface 12, and a bus 13, and may further include a computer program stored in the memory 11 and executable on the processor 10, such as a night vision image quality enhancement program based on multi-spectral analysis.
The memory 11 includes at least one type of readable storage medium, which includes flash memory, removable hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device, for example a removable hard disk of the electronic device. The memory 11 may also be an external storage device of the electronic device in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the electronic device. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device. The memory 11 may be used not only to store application software installed in the electronic device and various types of data, such as codes of a night vision image quality improvement program based on multispectral analysis, etc., but also to temporarily store data that has been output or will be output.
The processor 10 may be formed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be formed of a plurality of integrated circuits packaged with the same function or different functions, including one or more Central Processing Units (CPUs), microprocessors, digital Processing chips, graphics processors, and combinations of various control chips. The processor 10 is a Control Unit (Control Unit) of the electronic device, connects various components of the electronic device by using various interfaces and lines, and executes various functions and processes of the electronic device by running or executing programs or modules (e.g., a night vision image quality improvement program based on multispectral analysis, etc.) stored in the memory 11 and calling data stored in the memory 11.
The communication interface 12 is used for communication between the electronic device and other devices, and includes a network interface and a user interface. Optionally, the network interface may include a wired interface and/or a wireless interface (e.g., WI-FI interface, bluetooth interface, etc.), which are typically used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a Display (Display), an input unit, such as a Keyboard (Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable, among other things, for displaying information processed in the electronic device and for displaying a visualized user interface.
The bus 13 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus 13 may be divided into an address bus, a data bus, a control bus, etc. The bus 13 is arranged to enable connection communication between the memory 11 and at least one processor 10 or the like.
Fig. 3 shows only an electronic device having components, and those skilled in the art will appreciate that the structure shown in fig. 3 does not constitute a limitation of the electronic device, and may include fewer or more components than those shown, or some components may be combined, or a different arrangement of components.
For example, although not shown, the electronic device may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 10 through a power management device, so that functions such as charge management, discharge management, and power consumption management are implemented through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The electronic device may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.
Further, the electronic device may further include a network interface, and optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a bluetooth interface, etc.), which are generally used to establish a communication connection between the electronic device and other electronic devices.
Optionally, the electronic device may further comprise a user interface, which may be a Display (Display), an input unit (such as a Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable, among other things, for displaying information processed in the electronic device and for displaying a visualized user interface.
It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.
The memory 11 in the electronic device stores a multi-spectral analysis based night vision image quality improvement program that is a combination of instructions that, when executed in the processor 10, may implement:
acquiring an original night vision image set, blocking images in the original night vision image set, and performing adaptive training by using the blocked night vision images to obtain a local sparse structure dictionary;
performing sparse denoising on the images in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set;
performing spectral filtering processing and color enhancement processing on the images in the night vision denoising image set to obtain an enhanced image set;
performing color recovery processing on the images in the enhanced image set to obtain a standard night vision image set;
and performing sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set.
Specifically, the specific implementation method of the instruction by the processor 10 may refer to the description of the relevant steps in the embodiment corresponding to the drawings, which is not described herein again.
Further, the electronic device integrated module/unit, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in a computer readable storage medium. The computer readable storage medium may be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).
The present invention also provides a computer-readable storage medium, storing a computer program which, when executed by a processor of an electronic device, may implement:
acquiring an original night vision image set, blocking images in the original night vision image set, and performing adaptive training by using the blocked night vision images to obtain a local sparse structure dictionary;
performing sparse denoising on the images in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set;
performing spectral filtering processing and color enhancement processing on the images in the night vision denoising image set to obtain an enhanced image set;
performing color recovery processing on the images in the enhanced image set to obtain a standard night vision image set;
and performing sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set.
In the several embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
The embodiment of the application can acquire and process related data based on an artificial intelligence technology. Among them, Artificial Intelligence (AI) is a theory, method, technique and application system that simulates, extends and expands human Intelligence using a digital computer or a machine controlled by a digital computer, senses the environment, acquires knowledge and uses the knowledge to obtain the best result.
The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. A night vision image quality improvement method based on multispectral analysis is characterized by comprising the following steps:
acquiring an original night vision image set, blocking images in the original night vision image set, and performing adaptive training by using the blocked night vision images to obtain a local sparse structure dictionary;
performing sparse denoising on the images in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set;
performing spectral filtering processing and color enhancement processing on the images in the night vision denoising image set to obtain an enhanced image set;
performing color recovery processing on the images in the enhanced image set to obtain a standard night vision image set;
and performing sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set.
2. The method for improving night vision image quality based on multispectral analysis as claimed in claim 1, wherein the step of blocking the images in the original night vision image set and performing adaptive training using the blocked night vision images to obtain a local sparse structure dictionary comprises:
partitioning the images in the original night vision image set according to a preset partitioning size to obtain a partitioned image set;
calculating a neighborhood reconstruction weight matrix of the block images in the block image set;
and performing iterative optimization on the neighborhood reconstruction weight matrix based on a K-LSPSc algorithm to obtain a sparse coefficient, and constructing a local sparse structure dictionary based on the sparse coefficient.
3. The night vision image quality improvement method based on multispectral analysis as claimed in claim 2, wherein the performing iterative optimization on the neighborhood reconstruction weight matrix based on the K-LSPSc algorithm to obtain sparse coefficients, and constructing a local sparse structure dictionary based on the sparse coefficients comprises:
acquiring a preset sparse dictionary, fixing the sparse dictionary, and performing iterative optimization on the neighborhood reconstruction weight matrix by using a preset alternative optimization formula to obtain a sparse coefficient matrix;
and fixing the sparse coefficient matrix, and updating the sparse dictionary by using a preset dictionary updating formula to obtain a local sparse structure dictionary.
4. The method for improving night vision image quality based on multispectral analysis as claimed in claim 2, wherein the sparse denoising of the images in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set comprises:
carrying out sparse representation on images in the block image set by using the local sparse structure dictionary to obtain a sparse image set;
carrying out quadratic term solving on the images in the sparse image set by using the following formula to obtain the night vision de-noising image set:
wherein,for the set of night vision de-noised images,is a constant of experience and is,is a matrix of the units,is a square window operator, and is characterized in that,a set of block images is represented as a set of block images,for the purpose of the local sparse structure dictionary,which represents a transposition of the image,is as followsSparse coefficients of the individual center block images.
5. The method for improving night vision image quality based on multispectral analysis as claimed in claim 1, wherein the performing spectral filtering and color enhancement on the image in the night vision de-noised image set to obtain an enhanced image set includes:
performing spectral filtering processing on the images in the night vision denoising image set to obtain a plurality of three-primary-color images;
and performing band screening on the three primary color images by using a preset band range to obtain a plurality of band images, performing image fusion on the plurality of band images, and summarizing all fused images to obtain the enhanced image set.
6. The method for improving night vision image quality based on multispectral analysis as claimed in claim 1, wherein the performing a color recovery process on the images in the enhanced image set to obtain a standard night vision image set comprises:
carrying out color channel decomposition on the images in the enhanced image set to obtain images with different color channels;
carrying out multi-scale enhancement processing on the image of each color channel by using an MSRCR algorithm to obtain a multi-scale enhanced image;
calculating the color recovery coefficient of the multi-scale enhanced image, and performing color recovery on the multi-scale enhanced image of different color channels based on the color recovery coefficient to obtain color recovery images of different color channels;
and fusing the color recovery images of different color channels to obtain the standard night vision image set.
7. The method for night vision image quality improvement based on multispectral analysis of claim 6, wherein the calculating the color recovery coefficient of the multiscale enhanced image comprises:
calculating a color recovery coefficient of the multi-scale enhanced image using the following formula:
wherein,is a firstThe color recovery coefficients of the individual color channels,is a constant value of the gain, and is,is a non-linear coefficient, and is,is as followsThe image of each of the color channels is,is a firstThe color of the individual color channels restores the image,in order to enhance the image at a multi-scale,is a firstThe weight of each of the scales is determined,is a function of the surrounding of the gaussian,denotes convolution operation, and D denotes scale degree.
8. The method for improving night vision image quality based on multispectral analysis as claimed in claim 1, wherein the performing sub-histogram equalization on the images in the standard night vision image set to obtain an enhanced night vision image set includes:
calculating original histograms of the images in the standard night vision image set one by one, and partitioning the original histograms to obtain a plurality of sub-histograms;
carrying out gray dynamic adjustment on the plurality of sub-histograms to obtain a plurality of adjusted sub-histograms;
performing gray level balance enhancement processing on the plurality of adjustment sub-histograms based on a gray level probability density function to obtain a plurality of standard sub-histograms;
and carrying out normalization processing and histogram equalization processing on each standard sub-histogram, and summarizing all the images subjected to histogram equalization processing to obtain the enhanced night vision image set.
9. A night vision image quality improvement device based on multispectral analysis, the device comprising:
the local dictionary building module is used for acquiring an original night vision image set, blocking images in the original night vision image set, and performing self-adaptive training by using the blocked night vision images to obtain a local sparse structure dictionary;
the sparse denoising module is used for performing sparse denoising on the images in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set;
the color enhancement module is used for performing spectral filtering processing and color enhancement processing on the images in the night vision de-noising image set to obtain an enhanced image set;
the color recovery module is used for performing color recovery processing on the images in the enhanced image set to obtain a standard night vision image set;
and the night vision image enhancement module is used for carrying out sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set.
10. An electronic device, characterized in that the electronic device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the method for improving night vision image quality based on multi-spectral analysis of any one of claims 1 to 8.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210674311.0A CN114757854B (en) | 2022-06-15 | 2022-06-15 | Night vision image quality improving method, device and equipment based on multispectral analysis |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210674311.0A CN114757854B (en) | 2022-06-15 | 2022-06-15 | Night vision image quality improving method, device and equipment based on multispectral analysis |
Publications (2)
Publication Number | Publication Date |
---|---|
CN114757854A true CN114757854A (en) | 2022-07-15 |
CN114757854B CN114757854B (en) | 2022-09-02 |
Family
ID=82336159
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210674311.0A Active CN114757854B (en) | 2022-06-15 | 2022-06-15 | Night vision image quality improving method, device and equipment based on multispectral analysis |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114757854B (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115205297A (en) * | 2022-09-19 | 2022-10-18 | 汶上县金振机械制造有限公司 | Abnormal state detection method for pneumatic winch |
CN116366990A (en) * | 2023-02-20 | 2023-06-30 | 王一川 | Algorithm system applied to night vision device |
Citations (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101907772A (en) * | 2010-07-12 | 2010-12-08 | 南京理工大学 | Infrared and glimmer multispectral fusion front end optical-mechanic structure |
CN102499635A (en) * | 2011-10-26 | 2012-06-20 | 中国科学院光电技术研究所 | Line scanning-based fundus retina multispectral imaging system and method |
CN103390281A (en) * | 2013-07-29 | 2013-11-13 | 西安科技大学 | Double-spectrum night vision instrument vehicle-mounted system and double-spectrum fusion design method |
CN103390272A (en) * | 2013-07-16 | 2013-11-13 | 西安应用光学研究所 | Method for achieving registration and fusion of multi-spectral pseudo color images |
CN103700066A (en) * | 2013-12-05 | 2014-04-02 | 莱阳市科盾通信设备有限责任公司 | Method for processing video image of portable night vision instrument |
CN107241575A (en) * | 2017-06-05 | 2017-10-10 | 西安理工大学 | The wearable video monitoring system and fusion method of multi-source image fusion display |
CN107527332A (en) * | 2017-10-12 | 2017-12-29 | 长春理工大学 | Enhancement Method is kept based on the low-light (level) image color for improving Retinex |
CN107705258A (en) * | 2017-09-19 | 2018-02-16 | 东华大学 | A kind of underwater picture Enhancement Method of three primary colours joint preequalization and deblurring |
CN108416745A (en) * | 2018-02-02 | 2018-08-17 | 中国科学院西安光学精密机械研究所 | Image self-adaptive defogging enhancement method with color constancy |
CN108491868A (en) * | 2018-03-12 | 2018-09-04 | 广东工业大学 | A kind of image processing method and device based on k-means cluster and dictionary learning |
CN110274556A (en) * | 2019-06-14 | 2019-09-24 | 南京林业大学 | A kind of plant phenotype information acquisition system and extracting method |
AU2020104021A4 (en) * | 2020-12-11 | 2021-02-18 | Sichuan University of Science and Engineering | Improved Retinex Image Enhancement Algorithm for Explicit Expression of Multi-scale and Histogram Truncation |
CN112634150A (en) * | 2021-03-08 | 2021-04-09 | 哈尔滨理工大学 | Infrared night vision image enhancement method based on FPGA |
CN113379861A (en) * | 2021-05-24 | 2021-09-10 | 南京理工大学 | Color low-light-level image reconstruction method based on color recovery block |
WO2021211297A1 (en) * | 2020-04-17 | 2021-10-21 | Caterpillar Inc. | System and method for removing haziness in digital images |
CN113556526A (en) * | 2021-07-18 | 2021-10-26 | 北京理工大学 | RGBW filter array-based color enhancement method for color night vision equipment |
CN113609942A (en) * | 2021-07-27 | 2021-11-05 | 数量级(上海)信息技术有限公司 | Road intelligent monitoring system based on multi-view and multi-spectral fusion |
CN114018834A (en) * | 2021-08-06 | 2022-02-08 | 中科联芯(广州)科技有限公司 | Intelligent target identification method and detection device for silicon-based multispectral signals |
CN114529475A (en) * | 2022-02-23 | 2022-05-24 | 安徽大学 | Image enhancement method and system based on two-dimensional gamma correction and tone mapping |
-
2022
- 2022-06-15 CN CN202210674311.0A patent/CN114757854B/en active Active
Patent Citations (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101907772A (en) * | 2010-07-12 | 2010-12-08 | 南京理工大学 | Infrared and glimmer multispectral fusion front end optical-mechanic structure |
CN102499635A (en) * | 2011-10-26 | 2012-06-20 | 中国科学院光电技术研究所 | Line scanning-based fundus retina multispectral imaging system and method |
CN103390272A (en) * | 2013-07-16 | 2013-11-13 | 西安应用光学研究所 | Method for achieving registration and fusion of multi-spectral pseudo color images |
CN103390281A (en) * | 2013-07-29 | 2013-11-13 | 西安科技大学 | Double-spectrum night vision instrument vehicle-mounted system and double-spectrum fusion design method |
CN103700066A (en) * | 2013-12-05 | 2014-04-02 | 莱阳市科盾通信设备有限责任公司 | Method for processing video image of portable night vision instrument |
CN107241575A (en) * | 2017-06-05 | 2017-10-10 | 西安理工大学 | The wearable video monitoring system and fusion method of multi-source image fusion display |
CN107705258A (en) * | 2017-09-19 | 2018-02-16 | 东华大学 | A kind of underwater picture Enhancement Method of three primary colours joint preequalization and deblurring |
CN107527332A (en) * | 2017-10-12 | 2017-12-29 | 长春理工大学 | Enhancement Method is kept based on the low-light (level) image color for improving Retinex |
CN108416745A (en) * | 2018-02-02 | 2018-08-17 | 中国科学院西安光学精密机械研究所 | Image self-adaptive defogging enhancement method with color constancy |
CN108491868A (en) * | 2018-03-12 | 2018-09-04 | 广东工业大学 | A kind of image processing method and device based on k-means cluster and dictionary learning |
CN110274556A (en) * | 2019-06-14 | 2019-09-24 | 南京林业大学 | A kind of plant phenotype information acquisition system and extracting method |
WO2021211297A1 (en) * | 2020-04-17 | 2021-10-21 | Caterpillar Inc. | System and method for removing haziness in digital images |
AU2020104021A4 (en) * | 2020-12-11 | 2021-02-18 | Sichuan University of Science and Engineering | Improved Retinex Image Enhancement Algorithm for Explicit Expression of Multi-scale and Histogram Truncation |
CN112634150A (en) * | 2021-03-08 | 2021-04-09 | 哈尔滨理工大学 | Infrared night vision image enhancement method based on FPGA |
CN113379861A (en) * | 2021-05-24 | 2021-09-10 | 南京理工大学 | Color low-light-level image reconstruction method based on color recovery block |
CN113556526A (en) * | 2021-07-18 | 2021-10-26 | 北京理工大学 | RGBW filter array-based color enhancement method for color night vision equipment |
CN113609942A (en) * | 2021-07-27 | 2021-11-05 | 数量级(上海)信息技术有限公司 | Road intelligent monitoring system based on multi-view and multi-spectral fusion |
CN114018834A (en) * | 2021-08-06 | 2022-02-08 | 中科联芯(广州)科技有限公司 | Intelligent target identification method and detection device for silicon-based multispectral signals |
CN114529475A (en) * | 2022-02-23 | 2022-05-24 | 安徽大学 | Image enhancement method and system based on two-dimensional gamma correction and tone mapping |
Non-Patent Citations (10)
Title |
---|
HAN JING 等: ""Local Sparse Structure Denoising for Low-Light-Level Image"", 《 IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY》 * |
KIN GWNLORE等: ""A Deep Autoencoder Approach to Natural Low-light Image Enhancement"", 《PATTERN RECOGNITION》 * |
余瑾: ""低照度图像增强与去噪研究"", 《中国优秀硕士学位论文全文数据库 (信息科技辑)》 * |
刘晓燕: "基于Python的夜间图像增强方法研究", 《湖北民族学院学报(自然科学版)》 * |
刘远仲 等: ""一种改进的低照度图像增强算法研究"", 《四川轻化工大学学报(自然科学版)》 * |
姜雪松等: "夜晚图像增强方法综述", 《智能计算机与应用》 * |
蒋云峰 等: ""基于负滤光片的真彩色夜视系统设计"", 《光学学报》 * |
陈一超 等: ""单通道三波段彩色夜视系统设计及实验研究"", 《红外技术》 * |
陈晓玲: ""低曝光静脉图像增强方法研究"", 《中国优秀硕士学位论文全文数据库 (信息科技辑)》 * |
马龙 等: ""低光照图像增强算法综述"", 《中国图象图形学报》 * |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115205297A (en) * | 2022-09-19 | 2022-10-18 | 汶上县金振机械制造有限公司 | Abnormal state detection method for pneumatic winch |
CN116366990A (en) * | 2023-02-20 | 2023-06-30 | 王一川 | Algorithm system applied to night vision device |
CN116366990B (en) * | 2023-02-20 | 2024-02-23 | 王一川 | Algorithm system applied to night vision device |
Also Published As
Publication number | Publication date |
---|---|
CN114757854B (en) | 2022-09-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Zhang et al. | Single image defogging based on multi-channel convolutional MSRCR | |
CN114757854B (en) | Night vision image quality improving method, device and equipment based on multispectral analysis | |
CN108520504B (en) | End-to-end blurred image blind restoration method based on generation countermeasure network | |
CN113159147B (en) | Image recognition method and device based on neural network and electronic equipment | |
CN110675336A (en) | Low-illumination image enhancement method and device | |
WO2022247005A1 (en) | Method and apparatus for identifying target object in image, electronic device and storage medium | |
WO2022100510A1 (en) | Image distortion evaluation method and apparatus, and computer device | |
CN109978854B (en) | Screen content image quality evaluation method based on edge and structural features | |
CN113012068B (en) | Image denoising method, image denoising device, electronic equipment and computer-readable storage medium | |
CN112818737B (en) | Video identification method, device, storage medium and terminal | |
Tiwari et al. | Brightness preserving contrast enhancement of medical images using adaptive gamma correction and homomorphic filtering | |
CN111738954B (en) | Single-frame turbulence degradation image distortion removal method based on double-layer cavity U-Net model | |
CN111639704A (en) | Target identification method, device and computer readable storage medium | |
CN111339902A (en) | Liquid crystal display number identification method and device of digital display instrument | |
CN113065609B (en) | Image classification method, device, electronic equipment and readable storage medium | |
CN112183212A (en) | Weed identification method and device, terminal equipment and readable storage medium | |
Srinivas et al. | Remote sensing image segmentation using OTSU algorithm | |
Si et al. | A novel method for single nighttime image haze removal based on gray space | |
Zhou et al. | An improved algorithm using weighted guided coefficient and union self‐adaptive image enhancement for single image haze removal | |
CN110858304A (en) | Method and equipment for identifying identity card image | |
CN117689550A (en) | Low-light image enhancement method and device based on progressive generation countermeasure network | |
CN117152182A (en) | Ultralow-illumination network camera image processing method and device and electronic equipment | |
CN115601267B (en) | Global tone mapping method with local detail compensation capability | |
CN110489584B (en) | Image classification method and system based on dense connection MobileNet model | |
CN115908175A (en) | Low-illumination image multi-level enhancement method and device, electronic equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |