CN113160096A - Low-light image enhancement method based on retina model - Google Patents

Abstract

The present invention is a low-light image enhancement method based on retinal model, belonging to the technical field of image processing, comprising: step S1: obtaining a similar pixel group; step S2: performing Haar transform on the similar pixel group, using pixel-level The non-local Haar transform obtains the illumination component and the reflection component respectively in the three channels of R, G and B; Step S3: find the final reflection component; Step S4: find the enhanced illumination component; Step S5: take the minimum component of the enhanced illumination component as final illumination component; step S6: applying the final reflection component and the final illumination component to the retinal model to obtain an enhanced image. The low-light image enhancement method of the present invention is fast and effective, the color of the image processed by the method will not be too bright, the information in the original image can be well preserved, and the problem of uneven illumination after the enhancement can be well solved, and no False signals will be introduced, and the fidelity of image edge information is very high.

Description

A low-light image enhancement method based on retinal model

technical field

The invention belongs to the technical field of image processing, and particularly relates to a low-light image enhancement method based on a retinal model.

Background technique

Low-light image enhancement is to enhance some low-brightness images captured in low-light environments, in order to obtain images with better lighting effects. For example, photos taken at night or in a closed environment often have the problem that the brightness is too low and the specific content of the picture cannot be effectively identified. Therefore, low-light image enhancement has always been one of the hot research directions in the field of computer vision.

Most of the current low-light image enhancement methods are purely based on retinal models, such as MSR, MSRCR, SIRE, and RRM. Such methods often have defects such as insignificant enhancement effects, color distortion, and artifacts, and the existing low-light Image enhancement methods take a lot of time to process images.

SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome at least one disadvantage of the prior art, and to provide a low-light image enhancement method based on a retinal model, which can effectively solve the problem of overexposure caused by enhancement in very bright places, and the problem caused by enhancement in very dark places. distortion problems, resulting in better low-light image enhancement.

The invention discloses a low-light image enhancement method based on a retinal model, comprising:

Step S1: obtaining similar pixel groups;

Step S2: performing Haar transform on similar pixel groups, and using the low-frequency coefficients and high-frequency coefficients of pixel-level non-local Haar transforms to obtain illumination components and reflection components in three channels of R, G, and B, respectively;

Step S3: find the minimum component of the reflection components of the three channels R, G, and B as the final reflection component;

Step S4: Find the maximum component of the three channel illumination components of R, G, and B, and use the combination of exponential and logarithmic methods to enhance to obtain enhanced illumination components;

Step S5: take the minimum component of the enhanced illumination component as the final illumination component;

Step S6: Apply the final reflection component and the final illumination component to a retinal model to obtain an enhanced image.

Preferably, the step S1 specifically includes:

参考图像块B_r，在以B_r左上角坐标为中心的一个给定大小的邻域内执行块匹配获得与B_r最相似的N2-1个图像块，从而获得连同B_r在内N2个相似图像块；Step S1a: Perform block matching and line matching respectively in the three channels of R, G, and B in the RGB color space, and select a size among them according to a certain sliding step.

Referring to the image block B _r , perform block matching in a neighborhood of a given size centered on the upper-left corner coordinate of Br to obtain _N2-1 image blocks most similar to B _r , thereby obtaining N2 similar image blocks together with B _r image block;

的图像块拉伸成一个列向量，标记为

将所有V_l按列拼接为一个

行N2列的矩阵M_b；Step S1b: Convert each dimension to

The image blocks are stretched into a column vector, labeled as

Concatenate all V _l by column into one

matrix M _b with row N2 columns;

Step S1c: Select one row R _r of the matrix M _b as a reference row, calculate the Euclidean distance between _{R r} and all other rows to find the most similar N ₃ -1 row, and construct a size N ₃ × N ₂ similar pixel matrix M _s .

Preferably, the step S1c is specifically:

Taking the i-th row in the matrix M _b as the reference row, the Euclidean distance between the i-th row and all other rows is calculated as:

The N ₃ -1 row with the smallest distance from the ith row is selected, and together with the ith row, a similar pixel matrix M _s of size N ₃ ×N ₂ is finally obtained.

Preferably, the Haar transform in the step S2 specifically includes:

The vertical and horizontal separable lifting Haar transforms are performed on the similar pixel matrix M _s , namely:

C _h =H _l *M _S *H _r

Among them, C _h is the spectral matrix after Haar transformation, and H _l and H _r are Haar matrices.

Preferably, the method for obtaining the illumination component in the step S2 specifically includes:

C _h (1,1) is defined as a low-frequency coefficient, and the illumination component I _l is obtained by reconstructing the image after performing inverse Haar transform with C _h (1,1).

Preferably, the method for obtaining the reflection component in the step S2 specifically includes:

Define C _h (1,1) as the low-frequency coefficient, and use the N ₃ ×N ₂ -1 transformation coefficients of C _h -C _h (1,1) to perform inverse Haar transform and reconstruct the image to obtain the reflection component I _r .

Preferably, the step S4 specifically includes:

和

对

和

进行比较，获得光照分量的最大分量；Step S4a: For the illumination components, obtain 3 illumination components through the R, G, and B channels respectively

and

right

and

Make a comparison to obtain the maximum component of the illumination component;

Step S4b _: Perform the enhancement step with different exponents _γ1 and γ2,

where _γ1 is calculated by the following method:

where _γ2 is calculated by the following method:

if

but

otherwise

Step S4c: obtaining the first enhanced illumination component

Step S4cd: obtaining the second enhanced illumination component

Preferably, the step S5 specifically includes:

Step S5a: Obtain the final illumination component

为最终光照分量，

为第一增强光照分量，

为第二增强光照分量；in,

is the final illumination component,

is the first enhanced illumination component,

is the second enhanced illumination component;

Step S5b: normalize the image to gray value [0,1];

Step S5c: compress the grayscale range of the image.

Preferably, the step S6 specifically includes:

Step S6a: apply the final enhanced illumination component and the final reflection component to the retina model, that is,

where I ^e is the final enhanced image, ⊙ is the dot product operation;

Step S6b: Denote I ^e as Y', replace the V channel in the HSV color space with Y', and then convert back to the RGB color space to obtain a final enhanced color image.

Compared with the prior art, the beneficial effects of the present invention are as follows: a low-light image enhancement method based on a retinal model of the present invention is fast and effective, the color of the image processed by the method will not be too bright, and the original image can be well preserved. It can solve the problem of uneven illumination after enhancement, and will not introduce false signals, and the fidelity of image edge information is very high.

Description of drawings

1 is a flowchart of a retinal model-based low-light image enhancement method of the present invention;

FIG. 2 is a first comparison diagram of image effects processed by the low-light image enhancement method of the present invention and some existing low-light enhancement methods;

FIG. 3 is a second comparison diagram of the image effects processed by the low-light image enhancement method of the present invention and some existing low-light enhancement methods.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings. This description is only used to explain the specific embodiments of the present invention, and should not be construed as limiting the present invention in any form. The specific embodiments are as follows:

As shown in Fig. 1, the present invention proposes a low-light image enhancement method based on the combination of retinal models, including the following steps:

Step S1: obtaining similar pixel groups;

Step S2: performing Haar transform on similar pixel groups, and using the low-frequency coefficients and high-frequency coefficients of pixel-level non-local Haar transforms to obtain illumination components and reflection components in three channels of R, G, and B, respectively;

Step S3: find the minimum component of the reflection components of the three channels R, G, and B as the final reflection component;

Step S4: Find the maximum component of the three channel illumination components of R, G, and B, and use the combination of exponential and logarithmic methods to enhance to obtain enhanced illumination components;

Step S5: take the minimum component of the enhanced illumination component as the final illumination component;

Step S6: Apply the final reflection component and the final illumination component to a retinal model to obtain an enhanced image.

Specifically include:

Get groups of similar pixels.

In a low-light color image I∈R ^h×w×c in RGB color space, I is synchronously converted from RGB color space to HSV color space.

参考图像块B_r，在以B_r左上角坐标为中心的一个给定大小的邻域内执行块匹配获得与B_r最相似的N2-1个图像块，从而获得连同B_r在内N2个相似图像块。将每一个尺寸为

的图像块拉伸成一个列向量，标记为

将所有V_l按列拼接为一个

行N2列的矩阵M_b。Perform block matching and line matching in the three channels of R, G, and B in the RGB color space, and select a size according to a certain sliding step.

Referring to the image block B _r , perform block matching in a neighborhood of a given size centered on the upper-left corner coordinate of Br to obtain _N2-1 image blocks most similar to B _r , thereby obtaining N2 similar image blocks together with B _r image block. convert each dimension to

The image blocks are stretched into a column vector, labeled as

Concatenate all V _l by column into one

Matrix M _b with rows and N2 columns.

To better mine self-similarity in images, we further implement row matching on M _b .

One row R _r is selected as a reference row to calculate its Euclidean distance with all other rows to find the most similar N ₃ -1 row, and together with R _r , a similar pixel matrix M _s of size N ₃ ×N ₂ is constructed.

Specifically, for the i-th row as the reference row, the Euclidean distance between the i-th row and all the remaining rows is calculated as:

Then, the N ₃ -1 row with the smallest distance from the i-th row is selected, and together with the i-th row, a similar pixel matrix M _s of size N ₃ ×N ₂ is finally obtained.

A separable Haar transform is performed on groups of similar pixels M _s .

Perform longitudinal and transverse separable lifting Haar transforms on M _s respectively, namely:

C _h =H _l *M _S *H _r

Among them, C _h is the spectral matrix after Haar transformation, and H _l and H _r are Haar matrices.

Due to the characteristics of separable lifting Haar transform, _{C h} (1, 1) is the weighted average of all pixels of MS, we define it as the low frequency coefficient, after performing the inverse Haar transform with only C _h (1, 1) The ideal illumination component I _l can be obtained by reconstructing the image; on the contrary, the inverse Haar transform is performed by using the N ₃ ×N ₂ -1 transform coefficients (ie medium and high frequency coefficients) of C _h -C _h (1,1). After reconstructing the image, the ideal reflection component I _r is obtained. This method can effectively and quickly separate the illumination and reflection components of the image, and is an important step in low-light image enhancement.

Perform an enhancement operation on the reflected component:

和

对

和

进行比较，获得反射分量的最小分量，然后选取最小分量作为最终反射分量l_r。For the reflection component, 3 reflection components are obtained through the R, G, and B channels respectively

and

right

and

For comparison, the minimum component of the reflection component is obtained, and then the minimum component is selected as the final reflection component l _r .

Perform an augmentation operation on the light component:

和

对

和

进行比较，获得光照分量的最大分量，即得到最亮的光照分量。用不同的指数γ₁和γ₂来执行增强步骤，γ₁和γ₂分别可以通过如下的方法计算：For the illumination components, 3 illumination components are obtained through the R, G, and B channels respectively.

and

right

and

For comparison, the maximum component of the illumination component is obtained, that is, the brightest illumination component is obtained. _The enhancement step is performed with different exponents _γ1 and _γ2 , respectively, which can be calculated as follows _:

γ ₂ is divided into the following two cases:

if

but

otherwise

可由如下方法获得：first enhanced illumination component

It can be obtained by the following methods:

可由如下方法获得：Second Enhanced Illumination Component

It can be obtained by the following methods:

In practice, it is found that if only the exponential transformation is used to enhance the illumination component of the low-brightness part of the image, the brightness value increases too fast, which will lead to the problem of insufficient brightness; if only the logarithmic transformation is used to enhance the illumination of the high-brightness part of the image. Component, the brightness value will also increase too fast, which will also cause uneven brightness.

Therefore, in order to solve the above problems, the present invention obtains the final illumination component I _l in the following manner:

Normalize the image to gray value [0,1], and then compress the grayscale range of the image. The dark area in the original image will have a large degree of bright area, while the bright area changes very little, achieving low The effect of light image enhancement ensures the adaptability of the enhancement results in different lighting areas.

Apply the final illumination component and the final reflection component to the retina model, i.e.

where I ^e is the final enhanced image, and ⊙ is the dot product operation.

Denote I ^e as Y', replace the V channel in the HSV color space with Y' and convert it back to the RGB color space to obtain the final enhanced color image.

The present invention conducts enhancement experiments on a dataset composed of 200 low-light images randomly selected from the CVPR2021UG2+challenge data set and 35 low-light image datasets with MATLAB software, executes the algorithm of the present invention, and obtains an enhanced result image, which is combined with The classical methods in the prior art such as HE, MSRCR, CVC, NPE, SIRE, MF, WVM, CRM, BIMEF, LIME, Jiep and STAR are compared, and the image effects are shown in Figures 2 and 3. The CVPR2021UG2+challenge dataset address is: ( http://cvpr2021.ug2challenge.org/dataset21_t1.html )

It can be seen from Fig. 2 and Fig. 3 that the color of the image enhanced by the method of the present invention will not be too bright, the information in the original image can be well preserved, and the problem of uneven illumination after enhancement can be well solved. False signals will be introduced, and the fidelity of image edge information is very high.

The NIQE, LOE, TMQI and FSIM values of the image enhanced by the method of the present invention and the enhanced image of the prior art are compared as shown in the following table:

method NIQE LOE TMQI FSIM HE 3.62 740.30 0.9220 0.7174 MSRCR 3.17 702.85 0.8506 0.6969 CVC 3.11 654.82 0.8715 0.8578 NPE 3.22 710.21 0.8891 0.8193 SIRE 3.01 637.70 0.8680 0.8991 MF 3.38 776.41 0.8997 0.8233 WVM 2.99 633.40 0.8674 0.8999 CRM 3.13 744.61 0.8964 0.8123 BIMEF 3.04 703.16 0.9017 0.8898 LIME 3.39 779.73 0.8791 0.7131 JieP 2.99 724.52 0.8766 0.8749 STAR 2.93 677.43 0.8784 0.9047 method of the invention 2.76 546.63 0.8616 0.9250

It should be noted that: the lower NIQE, LOE, and TMQI are, the higher the image quality is, and the higher the FSIM is, the higher the image quality is.

It can be seen from the data in the table that the processing effects of the above four index values of the image processed by the low-light image enhancement method of the present invention are better than the results of the low-light image enhancement method in the prior art.

By now, those skilled in the art will appreciate that, although exemplary embodiments of the present invention have been illustrated and described in detail herein, it is still possible to directly follow the present disclosure without departing from the spirit and scope of the present invention. Numerous other variations or modifications can be identified or derived consistent with the principles of the present invention. Accordingly, the scope of the present invention should be understood and deemed to cover all such other variations or modifications.

Claims (9)

1. a low-light image enhancement method based on retinal model, is characterized in that, comprises: Step S1: obtaining similar pixel groups; Step S2: performing Haar transform on similar pixel groups, using the low-frequency coefficients and high-frequency coefficients of pixel-level non-local Haar transforms to obtain illumination components and reflection components in three channels of R, G, and B, respectively; Step S3: find the minimum component of the reflection components of the three channels R, G, and B as the final reflection component; Step S4: Find the maximum component of the three channel illumination components of R, G, and B, and use the combination of exponential and logarithmic methods to enhance to obtain enhanced illumination components; Step S5: take the minimum component of the enhanced illumination component as the final illumination component; Step S6: Apply the final reflection component and the final illumination component to a retinal model to obtain an enhanced image. 2. The low-light image enhancement method according to claim 1, wherein the step S1 specifically comprises: Step S1a: Perform block matching and line matching respectively in the three channels of R, G, and B in the RGB color space, and select a size among them according to a certain sliding step.

Referring to the image block B _r , perform block matching in a neighborhood of a given size centered on the upper-left corner coordinate of Br to obtain _N2-1 image blocks most similar to B _r , thereby obtaining N2 similar image blocks together with B _r image block; Step S1b: Convert each dimension to

The image blocks are stretched into a column vector, labeled as

Concatenate all V _l by column into one

matrix M _b with row N2 columns; Step S1c: Select one row R _r of the matrix M _b as a reference row, calculate the Euclidean distance between _{R r} and all other rows to find the most similar N ₃ -1 row, and construct a size N ₃ × N ₂ similar pixel matrix M _s . 3. The low-light image enhancement method according to claim 2, wherein the step S1c is specifically: Taking the i-th row in the matrix M _b as the reference row, the Euclidean distance between the i-th row and all other rows is calculated as:

The N ₃ -1 row with the smallest distance from the ith row is selected, and together with the ith row, a similar pixel matrix M _s of size N ₃ ×N ₂ is finally obtained. 4. The low-light image enhancement method according to claim 3, wherein the Haar transform in the step S2 specifically comprises: The vertical and horizontal separable lifting Haar transforms are performed on the similar pixel matrix M _s , namely: C _h =H _l *M _s *H _r Among them, C _h is the spectral matrix after Haar transformation, and H _l and H _r are Haar matrices. 5. The low-light image enhancement method according to claim 4, wherein the method for obtaining the illumination component in the step S2 specifically comprises: C _h (1,1) is defined as a low-frequency coefficient, and the illumination component I _l is obtained by reconstructing the image after performing inverse Haar transform with C _h (1,1). 6. The low-light image enhancement method according to claim 4, wherein the method for obtaining the reflection component in the step S2 specifically comprises: Define C _h (1,1) as the low-frequency coefficient, and use the N ₃ ×N ₂ -1 transformation coefficients of C _h -C _h (1,1) to perform inverse Haar transform and reconstruct the image to obtain the reflection component I _r . 7. The low-light image enhancement method according to claim 1, wherein the step S4 specifically comprises: Step S4a: For the illumination components, obtain 3 illumination components through the R, G, and B channels respectively

and

right

and

Make a comparison to obtain the maximum component of the illumination component; Step S4b _: Perform the enhancement step with different exponents _γ1 and γ2, where _γ1 is calculated by the following method:

where _γ2 is calculated by the following method: if

but

otherwise

Step S4c: obtaining the first enhanced illumination component

Step S4cd: obtaining the second enhanced illumination component

8. The low-light image enhancement method according to claim 7, wherein the step S5 specifically comprises: Step S5a: Obtain the final illumination component

in,

is the final illumination component,

is the first enhanced illumination component,

is the second enhanced illumination component; Step S5b: normalize the image to gray value [0,1]; Step S5c: compress the grayscale range of the image. 9. The low-light image enhancement method according to claim 8, wherein the step S6 specifically comprises: Step S6a: apply the final enhanced illumination component and the final reflection component to the retina model, that is,

where I ^e is the final enhanced image, ⊙ is the dot product operation; Step S6b: Denote I ^e as Y', replace the V channel in the HSV color space with Y', and then convert back to the RGB color space to obtain a final enhanced color image.

Images