CN111738941B - Underwater Image Optimization Method Fused with Light Field and Polarization Information - Google Patents

Abstract

The invention discloses an underwater image optimization method for fusing light field and polarization information. The polarization map groups are respectively restored to obtain polarization restoration images at different positions; the target image is determined in the polarization restoration images; the target image is optimized according to the polarization images at all different positions; light field imaging technology and polarization imaging technology are combined Combined, the multi-depth information of the scene is obtained in one acquisition process, and the information dimension obtained by a single imaging is increased. The proposed polarization restoration algorithm is used to initially restore each sub-depth image, and finally the light field correlation algorithm is used for restoration and fusion to improve Underwater image quality.

Description

Underwater Image Optimization Method Fused with Light Field and Polarization Information

technical field

The invention relates to underwater image imaging, in particular to an underwater image optimization method for fusing light field and polarization information.

Background technique

When imaging underwater, the image quality is poor due to the absorption of light by water and the scattering of suspended particles. At present, the underwater image imaging method based on the atmospheric imaging model is not suitable for the low-illumination and multi-suspended particle underwater environment; the existing underwater image processing based on polarization information can improve the imaging quality, but it is difficult to distinguish the underwater environment caused by suspended particles. The forward and backward scattered light leads to severe distortion of the restored image.

Contents of the invention

The present invention provides an underwater image optimization method that combines light field and polarization information to overcome the above technical problems.

An underwater image optimization method for fusing light field and polarization information of the present invention includes:

The target scene is collected with polarized images at different positions from the target scene, and multiple polarized images form a polarized image group;

The polarization image groups at different positions are respectively restored to obtain the polarization restoration images at different positions;

determining a target image in said polarization-recovered image;

The target image is optimized based on the polarization images at all different positions.

Further, the collecting polarization images of the target scene at different positions away from the target scene includes: at the first position away from the target scene, adjusting the angle of the polarizer to collect multiple polarization images for the target scene; and so on , at the Nth position from the target scene, adjust the angle of the polarizer to collect multiple polarization images for the target scene.

Further, the polarization restoration images obtained at different positions are respectively restored to the polarization map groups at different positions, including:

Using Stokes vectors to label the set of polarization maps as:

Obtain the total light intensity image of the image scene at the current position, the intensity difference image in the horizontal and vertical directions, and the intensity difference image in the 45° and -45° directions, where S _N0 represents the total light intensity of the image scene at the current position, and S _N1 Indicates the intensity difference between the horizontal direction and the vertical direction of the current position, and S _N2 indicates the intensity difference between the 45° and -45° directions of the current position;

Calculate the polarization degree of the image scene at the current position according to the total light intensity image of the image scene at the current position, the intensity difference image in the horizontal direction and the vertical direction, and the intensity difference image in the 45° and -45° directions;

Calculate the dark channel image of the total light intensity image according to the total light intensity image of the image scene at the current position;

Adopt quadtree decomposition to divide the dark channel image into four sub-images equally, and calculate the mean value and variance of the four sub-images;

Determine the region of the dark channel image containing only the scattering effect according to the difference between the mean value and the variance of the sub-image, and determine the position of the brightest pixel in the region containing only the scattering effect;

Determine the total light intensity image of the image scene at the current position, the intensity difference images in the horizontal and vertical directions, and the intensity difference images in the 45° and -45° directions according to the region of the dark channel image that only contains the scattering effect. Areas with scattering effects, and determine the parameters of backscattered light at infinity according to the position of the brightest pixel in the dark channel image that only contains scattering effects;

Calculate the polarization degree of backscattered light according to the total light intensity image of the image scene at the current position, the intensity difference image in the horizontal direction and the vertical direction, and the intensity difference image in the 45° and -45° directions only containing the scattering effect. and polarization angle;

Calculate the backscattered light parameter according to the degree of polarization and polarization angle of the backscattered light, the degree of polarization of the current position image scene;

Restoring the image scene at the current position according to the backscattered light parameter at infinity and the backscattered light parameter.

Further, the determination of the region of the dark channel image containing only the scattering effect according to the difference between the mean value and the variance of the sub-image includes:

use the formula

Determining regions of the dark channel image containing only scattering effects, wherein the For the region in the final selected dark channel image that only contains scattering effects, the /> is the τ ^* th sub-image block in the dark channel image, and the τ ^* is the image block serial number τ ^* , /> is the τth sub-image block in the dark channel image, the /> is the mean value of the τth sub-image block in the dark channel image, the /> is the variance of the τth sub-image block in the dark channel image, and τ is the sequence number τ of the image block.

Further, the backscattered light parameters at infinity are determined according to the position of the brightest pixel point in the dark channel image that only contains the scattering effect area, including:

use the formula

Determine the backscattered light parameters at infinity, wherein, the B _N∞ (λ) is the backscattered light at infinity, and the S _N0 (i ^* , j ^* , λ) is the position (i ^* , j ^* ) pixel value in the image S _N0 , the (i ^* , j ^* ) is the position of the brightest pixel in the dark channel image that only contains the scattering effect area, and the for position (i,j) in the image /> Medium pixel value.

The present invention combines the light field imaging technology with the polarization imaging technology, obtains multiple depth-of-field information of the scene in one acquisition process, increases the information dimension obtained by a single imaging, and uses the proposed polarization restoration algorithm to initially restore each sub-depth-of-field image, Finally, the light field correlation algorithm is used for restoration and fusion to improve the underwater imaging quality.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained according to these drawings without any creative effort.

Fig. 1 is the underwater polarized optical imaging flow chart of the underwater image optimization method of fusion light field and polarization information of the present invention;

Fig. 2 is a schematic diagram of the underwater polarized optical imaging of the underwater image optimization method for fusing light field and polarization information of the present invention.

Fig. 3 is the processing process of polarization image in the underwater image optimization method of fusion light field and polarization information of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

As shown in Figure 1, it is a flow chart of the underwater image optimization method for fusing light field and polarization information of the present invention, including the following processing steps:

The target scene is collected with polarized images at different positions from the target scene, and multiple polarized images form a polarized image group;

The polarization image groups at different positions are respectively restored to obtain the polarization restoration images at different positions;

determining a target image in said polarization-recovered image;

The target image is optimized based on the polarization images at all different positions.

Further, the collecting polarization images of the target scene at different positions away from the target scene includes: at the first position away from the target scene, adjusting the angle of the polarizer to collect multiple polarization images for the target scene; and so on , at the Nth position from the target scene, adjust the angle of the polarizer to collect multiple polarization images for the target scene.

In underwater imaging, the image quality is poor due to the absorption of light by water and the scattering of suspended particles, and as the imaging distance increases, these two effects become more and more obvious, which makes underwater image recovery difficult. As shown in Figure 2, the camera receives light intensities from two aspects, one is the signal light from the target reflection and reaches the camera after attenuation, namely:

D(i,j,λ,d)=J(i,j,λ) T(i,j,λ) (1)

The other is the backscattered light that enters the camera after being scattered by suspended particles during light propagation, namely:

B(i,j,λ)=(1-T(i,j,λ))·B _∞ (λ) (2)

The final camera imaging process can be expressed as

I(i,j,λ)=D(i,j,λ)+B(i,j,λ)

=J(i,j,λ)·T(i,j,λ)+(1-T(i,j,λ))·B _∞ (λ) (3)

Among them, (i, j) is a certain point in the image, i is the abscissa of the pixel in the image, j is the ordinate of the pixel in the image, d is the depth of field, λ is the wavelength of light, λ∈{red ,green,blue} correspond to the three color channels of the RGB image, B(i,j,λ) is the backscattered light, I(i,j,λ) is the image acquired by the camera, T(i,j,λ) )=e ^-s(λ)d(i,j) is the transmittance map, B _∞ (λ) is the backscattered light at infinity, D(i,j,λ) is the target reflection and reaches the camera after attenuation The signal light of , J(i,j,λ) is the signal light without any attenuation. Combining formulas (1)-(3), we can get,

Therefore, it can be seen from formula (4) that to restore the image, B _∞ (λ) and B(i,j,λ) are two key parameters, which play a decisive role in the restoration quality of the image.

Further, as shown in Figure 3, the polarization image is processed by the following formula; since the traditional underwater image quality restoration based on polarization information all uses fixed depth of field to capture polarization images, it is difficult to obtain accurate model parameter estimates. In addition, human participation is required in parameter estimation, which has certain limitations in practical application.

The present invention uses different positions from the target scene to shoot the initial polarization images at different angles, that is, to obtain a multi-position polarization image group, so as to obtain the entire larger light field information as follows:

(I ₀ (0), I ₀ (45), I ₀ (90)), (I ₁ (0), I ₁ (45), I ₁ (90)), ......, (I _N (0), I _N (45), I _N (90)),

Initially restore the polarization image group at each sub-position, and take the Nth position distance polarization image group ( _IN (0), _IN (45), _IN (90)) as an example to illustrate the restoration operation:

The restoration of the polarization map groups at different positions to obtain the polarization restoration images at different positions includes: using the Stokes vector to mark the polarization map groups as:

Among them, S _N0 represents the total light intensity of the scene at the current position, S _N1 represents the intensity difference between the horizontal direction and the vertical direction of the current position, and S _N2 represents the intensity difference between the 45° and -45° directions of the current position.

Obtain the total light intensity image of the image scene at the current position, the intensity difference image in the horizontal and vertical directions, and the intensity difference image in the 45° and -45° directions, where S _N0 represents the total light intensity of the image scene at the current position, and S _N1 Indicates the intensity difference between the horizontal direction and the vertical direction of the current position, and S _N2 indicates the intensity difference between the 45° and -45° directions of the current position;

Calculate the polarization degree of the image scene at the current position according to the total light intensity image of the image scene at the current position, the intensity difference image in the horizontal direction and the vertical direction, and the intensity difference image in the 45° and -45° directions;

Calculate the dark channel image of the total light intensity image according to the total light intensity image of the image scene at the current position;

Adopt quadtree decomposition to divide the dark channel image into four sub-images equally, and calculate the mean value and variance of the four sub-images;

Determine the region of the dark channel image containing only the scattering effect according to the difference between the mean value and the variance of the sub-image, and determine the position of the brightest pixel in the region containing only the scattering effect;

Determine the total light intensity image of the image scene at the current position, the intensity difference images in the horizontal and vertical directions, and the intensity difference images in the 45° and -45° directions according to the region of the dark channel image that only contains the scattering effect. Areas with scattering effects, and determine the parameters of backscattered light at infinity according to the position of the brightest pixel in the dark channel image that only contains scattering effects;

Calculate the polarization degree of backscattered light according to the total light intensity image of the image scene at the current position, the intensity difference image in the horizontal direction and the vertical direction, and the intensity difference image in the 45° and -45° directions only containing the scattering effect. and polarization angle;

Calculate the backscattered light parameter according to the degree of polarization and polarization angle of the backscattered light, the degree of polarization of the current position image scene;

Restoring the image scene at the current position according to the backscattered light parameter at infinity and the backscattered light parameter.

Specifically, as the depth of field increases, the scattering effect increases and the image becomes more blurred (that is, the contrast becomes lower and lower). Then, the mean value of image brightness and the variance of image contrast can be used to determine the region at infinity. In order to avoid the influence of bright objects in the target scene, first calculate the dark channel image for the total light intensity image S _N0 :

Then for the dark channel image Carry out quadtree decomposition, calculate the mean value M and variance S of each area after decomposition, and subtract the variance from the mean value, select the area with the largest difference according to formula (7) and repeat this step until the last image block area /> The size is smaller than the set threshold, and the region is determined as an infinite region containing only scattering effects, and the region containing only scattering effects of the dark channel image is determined according to the difference between the mean and variance of the sub-images, Including the use of the following formula:

Determining regions of the dark channel image containing only scattering effects, wherein the For the region in the final selected dark channel image that only contains scattering effects, the /> is the τ ^* th sub-image block in the dark channel image, and the τ ^* is the image block serial number τ ^* , /> is the τth sub-image block in the dark channel image, the /> is the mean value of the τth sub-image block in the dark channel image, the /> is the variance of the τth sub-image block in the dark channel image, and τ is the sequence number τ of the image block.

Find out in the Stokes vector with The area corresponding to the position is denoted as Δ. Thus, the polarization degree and polarization angle of the backscattered light can be obtained from formulas (8)-(9), and |Δ| represents the total number of pixels in the area.

When estimating the key parameters B _N∞ (λ) and B _N (i, j, λ), most of the existing methods estimate by manually selecting an area without a target. However, due to differences in human operation, the restoration quality of the image will be unstable. Because both B _N∞ (λ) and B _N (i,j,λ) are parameters related only to backscattered light, it is necessary to determine the area on the image that only contains the scattering effect. As the distance increases, different wavelengths will be completely absorbed at a certain position, and the absorption effect disappears, so only the scattering effect is contained in the infinitely far region of the image.

Further, the backscattered light parameters at infinity are determined according to the position of the brightest pixel point in the dark channel image that only contains the scattering effect area, including:

use the formula

Determine the backscattered light parameters at infinity, wherein the B _N∞ (λ) is the backscattered light at infinity, and the S _N0 (i ^* , j ^* , λ) is the position (i ^* , j ^* ) Pixel value in the image S _N0 , the (i ^* , j ^* ) is the position of the brightest pixel in the dark channel image that only contains the scattering effect area, the for position (i,j) in the image /> Medium pixel value.

According to the polarization degree and polarization angle of the backscattered light, that is, the formula (8)-(9), the backscattered light parameter B _N (i, j, λ) is estimated

in,

Accordingly, the estimated parameters B _N∞ (λ), B _N (i, j, λ) can be brought into formula (4) to restore the degraded image J _N (i, j, λ ).

The degraded images at different positions are restored by the above method, denoted as J ₀ (i,j,λ), J ₁ (i,j,λ),..., J _N (i,j,λ), the target image J(i, j, λ) is used as a reference image, and the pixels corresponding to the reference image are found in other multi-position images; due to different shooting positions, the intensity, color and other information reflected by the same object point are different. The pixels of the same object point on the restored image at different distances are accumulated and averaged as the optimized pixel of the object point in the target image, thereby improving the quality of the target image.

The present invention combines the light field imaging technology with the polarization imaging technology, obtains multiple depth-of-field information of the scene in one acquisition process, increases the information dimension obtained by a single imaging, and uses the proposed polarization restoration algorithm to initially restore each sub-depth-of-field image, Finally, the light field correlation algorithm is used for restoration and fusion to improve the underwater imaging quality.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present invention. scope.

Claims (4)

1. An underwater image optimization method integrating light field and polarization information is characterized by comprising the following steps:

collecting polarization diagrams of a target scene at different positions away from the target scene, wherein a plurality of polarization diagrams form a polarization diagram group;

respectively restoring the polarization graphs at different positions to obtain polarization restored images at different positions;

the restoring the polarization graphs at different positions respectively to obtain polarization restored images at different positions comprises the following steps:

the polarization map group is labeled as follows using stokes vectors:

obtaining a total light intensity image, a horizontal and vertical direction intensity difference image and a 45 DEG and-45 DEG direction intensity difference image of an image scene at a current position, wherein S _N0 Representing the total light intensity of the current position image scene S _N1 Representing the intensity difference between the horizontal direction and the vertical direction of the current position, S _N2 Representing the intensity difference in the 45 DEG and-45 DEG directions of the current position;

calculating the polarization degree of the image scene at the current position according to the total light intensity image, the intensity difference images in the horizontal direction and the vertical direction and the intensity difference images in the 45-degree directions of the image scene at the current position;

calculating a dark channel image of the total light intensity image according to the total light intensity image of the image scene at the current position;

dividing the dark channel image into four sub-images uniformly by adopting quadtree decomposition, and calculating the mean value and variance of the four sub-images;

determining a region only containing scattering effect of the dark channel image according to the difference value of the mean value and the variance of the sub-image, and determining the position of the brightest pixel point in the region only containing scattering effect;

determining a total light intensity image, a horizontal-direction and vertical-direction intensity difference image and a 45-degree and-45-degree-direction intensity difference image of the image scene at the current position according to the area only containing the scattering effect of the dark channel image, and determining back scattering light parameters at infinity according to the position of the brightest pixel point in the area only containing the scattering effect in the dark channel image;

calculating the polarization degree and the polarization angle of the backward scattered light according to the total light intensity image, the horizontal and vertical intensity difference images and the areas only containing scattering effect of the 45 DEG and-45 DEG intensity difference images of the image scene of the current position;

calculating a back scattering light parameter according to the polarization degree and the polarization angle of the back scattering light and the polarization degree of the image scene at the current position;

restoring the image scene at the current position according to the back scattering light parameter at infinity and the back scattering light parameter;

determining a target image in the polarization recovery image;

the target image is optimized from all the polarization images at different positions.

2. The method of claim 1, wherein the performing polarimetric image acquisition of the target scene at different locations from the target scene comprises: at a first position away from a target scene, adjusting the angle of a polaroid to acquire a plurality of polaroids for the target scene; and by analogy, at the N position away from the target scene, adjusting the angle of the polaroid to acquire a plurality of polaroids for the target scene.

3. The method of claim 1, wherein said determining the region of the dark channel image that contains only scattering effects based on the difference between the mean and variance of the sub-images comprises:

using the formula

Determining a region of the dark channel image containing only scattering effects, wherein theFor regions of the finally selected dark channel image which only contain scattering effects, the +.>Is the τ in the dark channel image ^* Sub-image block, τ ^* For image block sequence number tau ^* ，/>For the τ sub-image block in the dark channel image, said +.>Is the mean value of the tau sub-image block in the dark channel image, said +.>Is the variance of the tau sub-image block in the dark channel image, and tau is the image block serial number tau.

4. The method of claim 1, wherein determining the infinity back-scattered light parameter from the location of the brightest pixel point in the dark channel image including only the scattering effect area comprises:

using the formula

Determining a back-scattered light parameter at infinity, wherein B is _N∞ (lambda) is back-scattered light at infinity, said S _N0 (i ^* ,j ^* Lambda) is the position (i ^* ,j ^* ) In image S _N0 A middle pixel value, said (i) ^* ,j ^* ) For the position of the brightest pixel point in the area only containing scattering effect in the dark channel image, theFor position (i, j) in the image +.>A middle pixel value.