CN114757854A

CN114757854A - Night vision image quality improving method, device and equipment based on multispectral analysis

Info

Publication number: CN114757854A
Application number: CN202210674311.0A
Authority: CN
Inventors: 黄文斌
Original assignee: Shenzhen Anxing Digital Systems Co ltd
Current assignee: Shenzhen Anxing Digital Systems Co ltd
Priority date: 2022-06-15
Filing date: 2022-06-15
Publication date: 2022-07-15
Anticipated expiration: 2042-06-15
Also published as: CN114757854B

Abstract

The invention relates to an artificial intelligence technology, and discloses a night vision image quality improving method based on multispectral analysis, which comprises the following steps: the method comprises the steps of blocking images in an original night vision image set, conducting self-adaptive training by using blocked night vision images to obtain a local sparse structure dictionary, conducting sparse de-noising on the images in the original night vision image set by using the local sparse structure dictionary to obtain a night vision de-noising image set, conducting spectral filtering processing and color enhancement processing on the images in the night vision de-noising image set to obtain an enhanced image set, conducting color recovery processing on the images in the enhanced image set to obtain a standard night vision image set, and conducting sub-histogram equalization processing on the images in the standard night vision image set to obtain the enhanced night vision image set. The invention further provides a night vision image quality improving device and equipment based on the multispectral analysis. The invention can improve the night vision image quality.

Description

Night vision image quality improving method, device and equipment based on multispectral analysis

Technical Field

The invention relates to the technical field of artificial intelligence, in particular to a night vision image quality improving method and device based on multispectral analysis, electronic equipment and a computer-readable storage medium.

Background

With the advancement of science and technology and the development of society, monitoring systems are applied to various fields and play an increasingly important role. The demand for monitoring systems is increasing from military to prison, from banks to shopping malls, from schools to residential areas and even every street. Meanwhile, with the increase of the night activity time of people, the information embodied in low-illumination and even dim light environment is more and more important. The low-light night vision image is mainly characterized in that: 1. the image has low gray level mean value, low signal-to-noise ratio and low contrast ratio, so that the image quality is low; 2. the gray level distribution of the image is concentrated, the whole frame of image is usually concentrated in the gray level of dozens to hundreds, and the means for improving the image quality is less.

Disclosure of Invention

The invention provides a night vision image quality improving method and device based on multispectral analysis, an electronic device and a readable storage medium, and mainly aims to improve the night vision image quality.

In order to achieve the above object, the present invention provides a night vision image quality improving method based on multispectral analysis, which includes:

acquiring an original night vision image set, blocking images in the original night vision image set, and performing adaptive training by using the blocked night vision images to obtain a local sparse structure dictionary;

performing sparse denoising on the images in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set;

performing spectral filtering processing and color enhancement processing on the images in the night vision denoising image set to obtain an enhanced image set;

performing color recovery processing on the images in the enhanced image set to obtain a standard night vision image set;

and performing sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set.

Optionally, the blocking the images in the original night vision image set, and performing adaptive training by using the blocked night vision images to obtain a local sparse structure dictionary includes:

partitioning the images in the original night vision image set according to a preset partitioning size to obtain a partitioned image set;

calculating a neighborhood reconstruction weight matrix of the block images in the block image set;

and performing iterative optimization on the neighborhood reconstruction weight matrix based on a K-LSPSc algorithm to obtain a sparse coefficient, and constructing a local sparse structure dictionary based on the sparse coefficient.

Optionally, the performing iterative optimization on the neighborhood reconstruction weight matrix based on the K-LSPSc algorithm to obtain a sparse coefficient, and constructing a local sparse structure dictionary based on the sparse coefficient includes:

acquiring a preset sparse dictionary, fixing the sparse dictionary, and performing iterative optimization on the neighborhood reconstruction weight matrix by using a preset alternating optimization formula to obtain a sparse coefficient matrix;

and fixing the sparse coefficient matrix, and updating the sparse dictionary by using a preset dictionary updating formula to obtain a local sparse structure dictionary.

Optionally, the performing sparse denoising on the image in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set includes:

carrying out sparse representation on images in the block image set by using the local sparse structure dictionary to obtain a sparse image set;

carrying out quadratic term solving on the images in the sparse image set by using the following formula to obtain the night vision de-noising image set:

wherein,

for the set of night vision de-noised images,

is a constant of experience and is,

is a matrix of the unit, and is,

is a square window operator, and is a square window operator,

a set of block images is represented as a set of block images,

for the purpose of the local sparse structure dictionary,

the transpose is represented by,

is a first

Sparse coefficients of the individual center block images.

Optionally, the performing spectral filtering and color enhancement processing on the images in the night vision de-noising image set to obtain an enhanced image set includes:

performing spectral filtering processing on the images in the night vision denoising image set to obtain a plurality of three-primary-color images;

and performing band screening on the three primary color images by using a preset band range to obtain a plurality of band images, performing image fusion on the plurality of band images, and summarizing all fused images to obtain the enhanced image set.

Optionally, the performing color recovery processing on the images in the enhanced image set to obtain a standard night vision image set includes:

carrying out color channel decomposition on the images in the enhanced image set to obtain images with different color channels;

carrying out multi-scale enhancement processing on the image of each color channel by using an MSRCR algorithm to obtain a multi-scale enhanced image;

calculating the color recovery coefficient of the multi-scale enhanced image, and performing color recovery on the multi-scale enhanced image of different color channels based on the color recovery coefficient to obtain color recovery images of different color channels;

and fusing the color recovery images of different color channels to obtain the standard night vision image set.

Optionally, the calculating the color recovery coefficient of the multi-scale enhanced image includes:

calculating a color recovery coefficient of the multi-scale enhanced image using the following formula:

wherein,

is as follows

The color recovery coefficients of the individual color channels,

is a constant value of the gain, and is,

is a non-linear coefficient of the linear vibration,

is as follows

The image of each of the color channels is,

is as follows

Color recovery map for individual color channelsLike the image of the eye(s) to be,

in order to enhance the image at a multi-scale,

is as follows

The weight of each of the scales is determined,

is a function of the surrounding of the gaussian,

represents convolution operation, and D represents the number of scales.

Optionally, the performing sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set includes:

calculating original histograms of images in the standard night vision image set one by one, and partitioning the original histograms to obtain a plurality of sub-histograms;

carrying out gray dynamic adjustment on the plurality of sub-histograms to obtain a plurality of adjusted sub-histograms;

performing gray level balance enhancement processing on the plurality of adjustment sub-histograms based on a gray level probability density function to obtain a plurality of standard sub-histograms;

and carrying out normalization processing and histogram equalization processing on each standard sub-histogram, and summarizing all the images subjected to histogram equalization processing to obtain the enhanced night vision image set.

In order to solve the above problem, the present invention further provides a night vision image quality improving apparatus based on multispectral analysis, the apparatus comprising:

the local dictionary building module is used for acquiring an original night vision image set, blocking images in the original night vision image set, and performing self-adaptive training by using the blocked night vision images to obtain a local sparse structure dictionary;

the sparse denoising module is used for performing sparse denoising on the images in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set;

the color enhancement module is used for performing spectral filtering processing and color enhancement processing on the images in the night vision de-noising image set to obtain an enhanced image set;

the color recovery module is used for performing color recovery processing on the images in the enhanced image set to obtain a standard night vision image set;

and the night vision image enhancement module is used for carrying out sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set.

In order to solve the above problem, the present invention also provides an electronic device, including:

a memory storing at least one computer program; and

and the processor executes the computer program stored in the memory to realize the night vision image quality improvement method based on the multispectral analysis.

According to the method, the images in the original night vision image set are subjected to self-adaptive training to obtain the local sparse structure dictionary, the images in the original night vision image set are subjected to sparse denoising based on the local sparse structure dictionary, and the noise can be effectively inhibited while the image details are kept by utilizing the noise invariant characteristic of the local sparse structure, so that the denoising effect is improved. Meanwhile, multispectral analysis and color enhancement are carried out on the images in the night vision denoising image set based on spectral filtering processing and color enhancement processing, the contrast ratio of the night vision images is improved, and the authenticity of the night vision images is also improved by carrying out color recovery processing. And finally, the night vision image is distributed more uniformly through sub-histogram equalization processing, the original image details are further kept, and the night vision image quality is improved. Therefore, the night vision image quality improving method and device based on multispectral analysis, the electronic device and the computer readable storage medium can improve the night vision image quality.

Drawings

Fig. 1 is a schematic flowchart of a night vision image quality improving method based on multispectral analysis according to an embodiment of the present invention;

FIG. 2 is a functional block diagram of a night vision image quality improving apparatus based on multi-spectral analysis according to an embodiment of the present invention;

fig. 3 is a schematic structural diagram of an electronic device implementing the night vision image quality improvement method based on multispectral analysis according to an embodiment of the present invention.

The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.

The embodiment of the application provides a night vision image quality improving method based on multispectral analysis. The execution subject of the night vision image quality improvement method based on multispectral analysis includes, but is not limited to, at least one of the electronic devices of a server, a terminal, and the like, which can be configured to execute the method provided by the embodiments of the present application. In other words, the night vision image quality improvement method based on multispectral analysis may be executed by software or hardware installed in a terminal device or a server device, and the software may be a block chain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like. The server may be an independent server, or may be a cloud server that provides basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a Content Delivery Network (CDN), a big data and artificial intelligence platform, and the like.

Referring to fig. 1, a flow chart of a night vision image quality improvement method based on multi-spectral analysis according to an embodiment of the invention is shown. In this embodiment, the method for improving the quality of the night vision image based on the multispectral analysis includes:

s1, acquiring an original night vision image set, blocking the images in the original night vision image set, and performing self-adaptive training by using the blocked night vision images to obtain a local sparse structure dictionary.

In the embodiment of the invention, the original night vision image set is an image acquired by low-light-level night vision equipment or a low-light-level night vision system under a low-light-level illumination condition, such as a low-light-level photo shot by a monitoring camera under a night condition.

In detail, the blocking the image in the original night vision image set, and performing adaptive training by using the blocked night vision image to obtain a local sparse structure dictionary includes:

If the image block is smaller, the local structure has no representation, and if the image block is larger, the sparsity of extracted features is weaker. Therefore, in the embodiment of the present invention, in order to achieve both image sparse information and structural features, the size of each image block is set to 8 × 8.

Further, the calculating a neighborhood reconstruction weight matrix of the block images in the block image set includes:

step A: randomly sampling the block image set to obtain a sample image set, and performing center marking and neighborhood marking on the block images in the sample image set to obtain a center block image and a neighborhood block image;

in the embodiment of the invention, in order to reduce the calculation amount, only part of the samples are used for training the dictionary, for example, 25% of the samples are randomly selected from all the block image sets. Due to random sampling, the number of non-overlapping neighboring blocks of each image block may be too small, and in order to ensure calculation of an accurate local structural relationship, the embodiment of the present invention centers on each block image and marks 64 neighboring block images that are nearest to each image block.

And B: calculating neighborhood reconstruction weight by using optimization function from neighborhood block image to center block image of each mark

；

In one embodiment of the present invention, the optimization function can be expressed as:

wherein,

blocking images for each neighborhood

，

Is as follows

Each central block image, wherein N is the number of block images;

the weight matrix is reconstructed for the neighborhood and,

represents the first in the neighborhood reconstruction weight matrix

Neighborhood reconstruction weights for individual center block images,

is shown as

Center block image and

and (5) neighborhood reconstruction weights of the neighborhood block images.

Further, the performing iterative optimization on the neighborhood reconstruction weight matrix based on the K-LSPSc algorithm to obtain a sparse coefficient, and constructing a local sparse structure dictionary based on the sparse coefficient includes:

acquiring a preset sparse dictionary, fixing the sparse dictionary, and performing iterative optimization on the neighborhood reconstruction weight matrix by using a preset alternative optimization formula to obtain a sparse coefficient matrix;

In an optional embodiment of the present invention, the preset sparse dictionary is a template dictionary learned by using a template image set. And fixing the sparse dictionary to determine a fixed learning dictionary, and further performing optimization updating on the sparse structure.

In an optional embodiment of the present invention, the preset alternating optimization formula is:

wherein,

is as follows

Sparse reconstruction error terms for individual center block images,

is as follows

The sparse coefficients of the individual center block images,

is as follows

A center block image, B is the preset sparse dictionary,

reconstructing a weight matrix for the neighborhood

Center block image and

neighborhood reconstruction weights for individual neighborhood block images,

to optimize sparse coefficients

The remaining sparse coefficients are fixed in time,

、

in order to optimize the parameters for the fixation,

is a mapping function, wherein, in the embodiment of the present invention, the mapping function

May be a radial basis kernel function.

The invention is optionalIn an embodiment, each is optimized by the alternating optimization formula

When the constraint condition is satisfied

And meanwhile, summarizing all sparse coefficients to obtain a total sparse coefficient matrix S.

Further, in an optional embodiment of the present invention, the preset dictionary update formula is:

wherein B is the preset sparse dictionary,

the transpose is represented by,

is a matrix of the units,

a matrix of sparse coefficients is represented by a matrix of,

representing a set of block images.

In an optional embodiment of the invention, the dictionary is continuously updated by using a preset dictionary updating formula until the constraint condition of the dictionary is met

And obtaining the local sparse structure dictionary.

S2, performing sparse denoising on the images in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set.

In the embodiment of the invention, a and b are used for representing a pure signal image and a noise-containing image, c is superimposed noise, the noise-containing image can be represented as b ═ a + c, and image denoising is to reconstruct a from b.

In detail, the sparse denoising of the image in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set includes:

carrying out quadratic term solving on the images in the sparse image set by using the following formula to obtain a night vision de-noising image set:

wherein,

for the set of night vision de-noised images,

is a constant of experience and is,

is a matrix of the units,

is a square window operator, and is characterized in that,

a set of block images is represented as a set of block images,

for the purpose of the local sparse structure dictionary,

the transpose is represented by,

is as follows

Sparse coefficients of the individual center block images.

In the embodiment of the invention, the existing sparse noise reduction method depends on Gaussian noise distribution hypothesis, the problem of noise redundancy removal of complex distribution in low-light-level images cannot be well solved, image details are difficult to effectively retain under strong noise interference, a local sparse structure dictionary trained by a K-LSPSc algorithm can reduce noise interference and improve robustness of sparse representation of image blocks, and meanwhile, the noise is effectively inhibited and the image details are retained by utilizing the noise invariant characteristic of a local sparse structure, so that the noise reduction effect is improved.

S3, performing spectral filtering processing and color enhancement processing on the images in the night vision de-noising image set to obtain an enhanced image set.

In the embodiment of the invention, because the illumination in the night vision image is weak, the color of the image is seriously distorted, the spectral filtering processing is carried out by the optical filters with a plurality of wave bands, and meanwhile, because the images with different wave bands are concentrated in different colors, the colors can be enhanced by fusing the images with different wave bands, and the image reality is improved.

Specifically, the performing spectral filtering processing and color enhancement processing on the image in the night vision denoising image set to obtain an enhanced image set includes:

performing spectral filtering processing on the images in the night vision denoising image set to obtain a plurality of three primary color images;

In an optional embodiment of the invention, the images in the night vision de-noising image set can be decomposed into R, G, B three primary color images by using spectral filters with different spectral transmittances. The image of each primary color has images of different wave bands, and the short-wave image (380-480 nm), the medium-wave image (510-565 nm) and the long-wave image (590-650 nm) are extracted from each primary color image sequence in the embodiment of the invention and are fused to obtain the enhanced image.

And S4, performing color recovery processing on the images in the enhanced image set to obtain a standard night vision image set.

In the embodiment of the invention, because the illumination in the night vision image is weak, and the color in the image after spectral filtering processing and color enhancement processing is still weak, the color of the image needs to be restored, thereby improving the authenticity of the image.

In one embodiment of the present invention, the images in the enhanced image set may be color-restored using the MSRCR algorithm. The MSRCR (Multi-Scale Retinex with Color retrieval) algorithm introduces the proportional relation in the image into three Color channels to adjust the proportion of the MSR (Multi-Scale Retinex, Multi-Scale retina enhancement) result, thereby optimizing the Color of the image, avoiding Color unsaturation and distortion, and enabling the image to have better Color presentation.

In detail, the performing color recovery processing on the images in the enhanced image set to obtain a standard night vision image set includes:

performing multi-scale enhancement processing on the image of each color channel by using an MSRCR algorithm to obtain a multi-scale enhanced image;

In an alternative embodiment of the present invention, the color recovery coefficient of the multi-scale enhanced image may be calculated by using the following formula:

wherein,

is a first

The color recovery coefficients of the individual color channels,

is a constant value of the gain, and is,

is a non-linear coefficient, and is,

is as follows

The image of each of the color channels is,

is as follows

The color of the individual color channels restores the image,

in order to enhance the image at a multi-scale,

is a first

The weight of each scale is determined by the weight of each scale,

is a function of the surrounding of the gaussian,

represents convolution operation, and D represents the number of scales.

In the embodiment of the invention, three scales of large, medium and small are taken, and the weight of each scale is 1/3. When the MSR algorithm is used for processing the RGB color image, R, G, B channels are respectively and independently enhanced, so that the proportion of pixel values of pixel points in each color channel can be changed in the enhancing process, the real color of objects in the whole or specific area of the image is faded, and the accuracy of restoring the real color of the image is improved by increasing the color restoring coefficient.

And S5, performing sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set.

In the embodiment of the invention, the image processed by the histogram equalization method can well improve the contrast and definition, but the method for integrally improving the contrast cannot well process very bright and very dark places and is difficult to integrally control, so that the embodiment of the invention provides an improved histogram equalization algorithm, namely sub-histogram equalization, to perform image enhancement processing aiming at a narrow range.

In detail, the performing sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set includes:

In the embodiment of the invention, the original histogram of the image in the standard night vision image set can be divided into four sub-histograms, and the gray level intervals of the four sub-histograms are [ m0, m1 ] in sequence]，[m1，m2]，[m2，m3]，[m3，m4]And using the following formula:

and dynamically adjusting the gray scales of the plurality of sub-histograms. Wherein in the formula

Representing the gray level dynamic range of the sub-histogram,

representing the gray level dynamic range of the adjusted sub-histogram,

which is indicative of the adjustment factor(s),

is the total number of gray levels of the image,

indicating the number of sub-histograms.

According to the embodiment of the invention, the new adjusted histogram is obtained by adjusting the sub-histograms according to the set dynamic range, each sub-interval is dynamically stretched, and the possibility of gray level combination is reduced.

Further, in an optional embodiment of the present invention, the gray level equalization enhancement processing is performed on the plurality of adjustment sub-histograms through the following gray level probability density function:

wherein,

for the gray level frequencies of the enhanced standard sub-histogram,

is the median of the grey scale frequencies and,

in order to control the factors, the control unit is provided with a control unit,

is the minimum value of the gray-level frequency,

is the maximum value of the frequency of the grey scale,

the gray level frequencies of the sub-histograms are adjusted for the current unprocessed tone.

In an optional embodiment of the present invention, the gray level frequency of the standard sub-Histogram is further normalized, and then HE (Histogram Equalization) processing is performed, so as to enhance the night vision image. The normalization process and the HE process are well known in the art and are not described herein again.

In the embodiment, the images in the original night vision image set are subjected to self-adaptive training to obtain the local sparse structure dictionary, the images in the original night vision image set are subjected to sparse de-noising based on the local sparse structure dictionary, and the noise is effectively suppressed while the image details are kept by utilizing the noise invariant characteristic of the local sparse structure, so that the noise reduction effect is improved. Meanwhile, multispectral analysis and color enhancement are carried out on the images in the night vision de-noising image set based on spectral filtering processing and color enhancement processing, the contrast of the night vision image is improved, and the authenticity of the night vision image is also improved by carrying out color recovery processing. And finally, the night vision image is distributed more uniformly through sub-histogram equalization processing, the original image details are further kept, and the night vision image quality is improved. Therefore, the night vision image quality improving method based on multispectral analysis can improve the night vision image quality.

Fig. 2 is a functional block diagram of a night vision image quality improving apparatus based on multi-spectral analysis according to an embodiment of the present invention.

The night vision image quality improving apparatus 100 based on multispectral analysis of the present invention can be installed in an electronic device. According to the implemented functions, the night vision image quality improving device 100 based on multispectral analysis may include a local dictionary construction module 101, a sparse denoising module 102, a color enhancement module 103, a color recovery module 104, and a night vision image enhancement module 105. The module of the present invention, which may also be referred to as a unit, refers to a series of computer program segments that can be executed by a processor of an electronic device and that can perform a fixed function, and that are stored in a memory of the electronic device.

In the present embodiment, the functions of the respective modules/units are as follows:

the local dictionary building module 101 is configured to obtain an original night vision image set, block images in the original night vision image set, and perform adaptive training by using the blocked night vision images to obtain a local sparse structure dictionary;

the sparse denoising module 102 is configured to perform sparse denoising on the images in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set;

the color enhancement module 103 is configured to perform spectral filtering processing and color enhancement processing on the images in the night vision denoising image set to obtain an enhanced image set;

the color recovery module 104 is configured to perform color recovery processing on the images in the enhanced image set to obtain a standard night vision image set;

the night vision image enhancement module 105 is configured to perform sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set.

In detail, the implementation of the modules of the night vision image quality improving apparatus 100 based on multispectral analysis is as follows:

the method comprises the steps of firstly, obtaining an original night vision image set, blocking images in the original night vision image set, and carrying out self-adaptive training by using the blocked night vision images to obtain a local sparse structure dictionary.

If the image block is smaller, the local structure has no representation, and if the image block is larger, the sparsity of extracted features is weaker. Therefore, in the embodiment of the present invention, in order to make consideration of both image sparse information and structural features, the size of each image block is set to 8 × 8.

in the embodiment of the invention, in order to reduce the amount of calculation, the dictionary is trained by only using part of samples, for example, 25% of the samples are randomly selected from all the block image sets. Due to random sampling, the number of non-overlapping neighbor image blocks of each image block may be too small, and in order to ensure accurate calculation of the local structural relationship, the embodiment of the present invention centers on each block image and marks 64 neighbor block images of the nearest neighbor of each image block.

And B, step B: calculating neighborhood reconstruction weight by using optimization function from neighborhood block image to center block image of each mark

；

wherein,

blocking images for each neighborhood

，

Is as follows

Each central block image, wherein N is the number of block images;

the weight matrix is reconstructed for the neighborhood and,

represents the first in the neighborhood reconstruction weight matrix

Neighborhood reconstruction weights for individual center block images,

is shown as

Center block image and

and (5) neighborhood reconstruction weights of the neighborhood block images.

wherein,

is as follows

Sparse reconstruction error terms for the individual center block images,

is as follows

The sparse coefficients of the individual center block images,

is as follows

A center block image, B is the preset sparse dictionary,

reconstructing a weight matrix for the neighborhood

Center block image and

neighborhood reconstruction weights for individual neighborhood block images,

to optimize sparse coefficients

The remaining sparse coefficients are fixed in time,

、

in order to optimize the parameters for the fixation,

May be a radial basis kernel function.

In an alternative embodiment of the invention, each is optimized by said alternating optimization formula

When the constraint condition is satisfied

wherein B is the preset sparse dictionary,

the transpose is represented by,

is a matrix of the units,

a matrix of sparse coefficients is represented by a matrix of,

representing a set of block images.

In an optional embodiment of the present invention, the word is continuously updated using a predetermined dictionary update formulaDictionary until the dictionary constraint condition is satisfied

And obtaining the local sparse structure dictionary.

And secondly, performing sparse denoising on the images in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set.

In detail, the sparse denoising of the image in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set comprises:

wherein,

for the set of night vision de-noised images,

is a constant of experience and is,

is a matrix of the units,

is a square window operator, and is a square window operator,

a set of block images is represented as a set of block images,

for the purpose of the local sparse structure dictionary,

the transpose is represented by,

is as follows

Sparse coefficients of the individual center block images.

And thirdly, performing spectral filtering processing and color enhancement processing on the images in the night vision denoising image set to obtain an enhanced image set.

In an optional embodiment of the present invention, the images in the night vision de-noising image set may be decomposed into R, G, B three primary color images by using spectral filters with different spectral transmittances. The image of each primary color has images of different wave bands, and the short-wave image (380-480 nm), the medium-wave image (510-565 nm) and the long-wave image (590-650 nm) are extracted from each primary color image sequence in the embodiment of the invention and are fused to obtain the enhanced image.

And fourthly, performing color recovery processing on the images in the enhanced image set to obtain a standard night vision image set.

In the embodiment of the invention, because the illumination in the night vision image is weak, and the color in the image after the spectral filtering processing and the color enhancement processing is still weak, the color recovery needs to be carried out on the image, thereby improving the authenticity of the image.

wherein,

is as follows

The color recovery coefficients of the individual color channels,

is a constant value of the gain, and is,

is a non-linear coefficient, and is,

is as follows

The image of each of the color channels is,

is a first

The color of each color channel is restored to the image,

in order to enhance the image at a multi-scale,

is as follows

The weight of each scale is determined by the weight of each scale,

is a function of the surrounding of the gaussian,

denotes convolution operation, and D denotes scale degree.

And fifthly, performing sub-histogram equalization processing on the images in the standard night vision image set to obtain an enhanced night vision image set.

In the embodiment of the invention, the contrast and the definition of the image processed by the histogram equalization method can be well improved, but the method for integrally improving the contrast cannot well process very bright and very dark places and is difficult to integrally control, so that the embodiment of the invention provides an improved histogram equalization algorithm, namely sub-histogram equalization, so as to perform image enhancement processing aiming at a narrow range.

Representing the gray level dynamic range of the sub-histogram,

representing the gray level dynamic range of the adjusted sub-histogram,

which is indicative of the adjustment factor(s),

is the total number of gray levels of the image,

indicating the number of sub-histograms.

Further, in an optional embodiment of the present invention, the gray scale equalization enhancement processing is performed on the plurality of adjustment sub-histograms by using the following gray scale probability density function:

wherein,

for the gray level frequencies of the enhanced standard sub-histogram,

is the median value of the grey scale frequencies,

is the minimum value of the frequency of the grey level,

is the maximum value of the frequency of the grey scale,

In an optional embodiment of the present invention, the gray scale frequency of the standard sub-Histogram is further normalized, and then HE (Histogram Equalization) processing is performed, so as to enhance the night vision image. The normalization process and the HE process are well known in the art and are not described herein again.

In the embodiment, the images in the original night vision image set are subjected to self-adaptive training to obtain the local sparse structure dictionary, the images in the original night vision image set are subjected to sparse de-noising based on the local sparse structure dictionary, and the noise is effectively suppressed while the image details are kept by utilizing the noise invariant characteristic of the local sparse structure, so that the noise reduction effect is improved. Meanwhile, multispectral analysis and color enhancement are carried out on the images in the night vision de-noising image set based on spectral filtering processing and color enhancement processing, the contrast of the night vision image is improved, and the authenticity of the night vision image is also improved by carrying out color recovery processing. And finally, the night vision image is distributed more uniformly through sub-histogram equalization processing, the original image details are further kept, and the night vision image quality is improved. Therefore, the night vision image quality improving device based on multispectral analysis can improve the night vision image quality.

Fig. 3 is a schematic structural diagram of an electronic device for implementing the night vision image quality improvement method based on multispectral analysis according to an embodiment of the present invention.

The electronic device may include a processor 10, a memory 11, a communication interface 12, and a bus 13, and may further include a computer program stored in the memory 11 and executable on the processor 10, such as a night vision image quality enhancement program based on multi-spectral analysis.

The memory 11 includes at least one type of readable storage medium, which includes flash memory, removable hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device, for example a removable hard disk of the electronic device. The memory 11 may also be an external storage device of the electronic device in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the electronic device. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device. The memory 11 may be used not only to store application software installed in the electronic device and various types of data, such as codes of a night vision image quality improvement program based on multispectral analysis, etc., but also to temporarily store data that has been output or will be output.

The processor 10 may be formed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be formed of a plurality of integrated circuits packaged with the same function or different functions, including one or more Central Processing Units (CPUs), microprocessors, digital Processing chips, graphics processors, and combinations of various control chips. The processor 10 is a Control Unit (Control Unit) of the electronic device, connects various components of the electronic device by using various interfaces and lines, and executes various functions and processes of the electronic device by running or executing programs or modules (e.g., a night vision image quality improvement program based on multispectral analysis, etc.) stored in the memory 11 and calling data stored in the memory 11.

The communication interface 12 is used for communication between the electronic device and other devices, and includes a network interface and a user interface. Optionally, the network interface may include a wired interface and/or a wireless interface (e.g., WI-FI interface, bluetooth interface, etc.), which are typically used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a Display (Display), an input unit, such as a Keyboard (Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable, among other things, for displaying information processed in the electronic device and for displaying a visualized user interface.

The bus 13 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus 13 may be divided into an address bus, a data bus, a control bus, etc. The bus 13 is arranged to enable connection communication between the memory 11 and at least one processor 10 or the like.

Fig. 3 shows only an electronic device having components, and those skilled in the art will appreciate that the structure shown in fig. 3 does not constitute a limitation of the electronic device, and may include fewer or more components than those shown, or some components may be combined, or a different arrangement of components.

For example, although not shown, the electronic device may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 10 through a power management device, so that functions such as charge management, discharge management, and power consumption management are implemented through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The electronic device may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.

Further, the electronic device may further include a network interface, and optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a bluetooth interface, etc.), which are generally used to establish a communication connection between the electronic device and other electronic devices.

Optionally, the electronic device may further comprise a user interface, which may be a Display (Display), an input unit (such as a Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable, among other things, for displaying information processed in the electronic device and for displaying a visualized user interface.

It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.

The memory 11 in the electronic device stores a multi-spectral analysis based night vision image quality improvement program that is a combination of instructions that, when executed in the processor 10, may implement:

Specifically, the specific implementation method of the instruction by the processor 10 may refer to the description of the relevant steps in the embodiment corresponding to the drawings, which is not described herein again.

Further, the electronic device integrated module/unit, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in a computer readable storage medium. The computer readable storage medium may be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).

The present invention also provides a computer-readable storage medium, storing a computer program which, when executed by a processor of an electronic device, may implement:

In the several embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.

The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.

It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.

The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.

The embodiment of the application can acquire and process related data based on an artificial intelligence technology. Among them, Artificial Intelligence (AI) is a theory, method, technique and application system that simulates, extends and expands human Intelligence using a digital computer or a machine controlled by a digital computer, senses the environment, acquires knowledge and uses the knowledge to obtain the best result.

The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not any particular order.

Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A night vision image quality improvement method based on multispectral analysis is characterized by comprising the following steps:

2. The method for improving night vision image quality based on multispectral analysis as claimed in claim 1, wherein the step of blocking the images in the original night vision image set and performing adaptive training using the blocked night vision images to obtain a local sparse structure dictionary comprises:

3. The night vision image quality improvement method based on multispectral analysis as claimed in claim 2, wherein the performing iterative optimization on the neighborhood reconstruction weight matrix based on the K-LSPSc algorithm to obtain sparse coefficients, and constructing a local sparse structure dictionary based on the sparse coefficients comprises:

4. The method for improving night vision image quality based on multispectral analysis as claimed in claim 2, wherein the sparse denoising of the images in the original night vision image set by using the local sparse structure dictionary to obtain a night vision denoising image set comprises:

wherein,

for the set of night vision de-noised images,

is a constant of experience and is,

is a matrix of the units,

is a square window operator, and is characterized in that,

a set of block images is represented as a set of block images,

for the purpose of the local sparse structure dictionary,

which represents a transposition of the image,

is as follows

Sparse coefficients of the individual center block images.

5. The method for improving night vision image quality based on multispectral analysis as claimed in claim 1, wherein the performing spectral filtering and color enhancement on the image in the night vision de-noised image set to obtain an enhanced image set includes:

6. The method for improving night vision image quality based on multispectral analysis as claimed in claim 1, wherein the performing a color recovery process on the images in the enhanced image set to obtain a standard night vision image set comprises:

7. The method for night vision image quality improvement based on multispectral analysis of claim 6, wherein the calculating the color recovery coefficient of the multiscale enhanced image comprises:

wherein,

is a first

The color recovery coefficients of the individual color channels,

is a constant value of the gain, and is,

is a non-linear coefficient, and is,

is as follows

The image of each of the color channels is,

is a first

The color of the individual color channels restores the image,

in order to enhance the image at a multi-scale,

is a first

The weight of each of the scales is determined,

is a function of the surrounding of the gaussian,

denotes convolution operation, and D denotes scale degree.

8. The method for improving night vision image quality based on multispectral analysis as claimed in claim 1, wherein the performing sub-histogram equalization on the images in the standard night vision image set to obtain an enhanced night vision image set includes:

calculating original histograms of the images in the standard night vision image set one by one, and partitioning the original histograms to obtain a plurality of sub-histograms;

9. A night vision image quality improvement device based on multispectral analysis, the device comprising:

10. An electronic device, characterized in that the electronic device comprises:

at least one processor; and the number of the first and second groups,

a memory communicatively coupled to the at least one processor; wherein,

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the method for improving night vision image quality based on multi-spectral analysis of any one of claims 1 to 8.