WO2020029033A1

WO2020029033A1 - Haze image clearing method and system, and storable medium

Info

Publication number: WO2020029033A1
Application number: PCT/CN2018/099031
Authority: WO
Inventors: 储颖; 游为麟
Original assignee: 深圳大学
Priority date: 2018-08-06
Filing date: 2018-08-06
Publication date: 2020-02-13

Abstract

A haze image clearing method and system, and a storable medium. The haze image clearing method comprises: randomly selecting a first depth network parameter to remove haze from a haze image so as to acquire a first haze-removed image and acquire a first image quality score; selecting a first high-quality depth network parameter according to the first image quality score; performing a fast bacterial group swimming optimization operation on the first high-quality depth network parameter to acquire several second depth network parameters; acquiring a second haze-removed image through the second depth network parameters and acquiring a second image quality score; selecting a second high-quality depth network parameter and determining whether the second high-quality depth network parameter is optimal; if so, outputting the second haze-removed image corresponding to the second high-quality depth network parameter, and ending the flow (S6); and if not, defining the second high-quality depth network parameter as a new first high-quality depth network parameter (S6-1). The method has the advantages of a fast processing process, and accurate and objective results.

Description

Haze image clearing method, system and storable medium

Technical field

The present invention relates to the technical field of image processing, and more particularly, to a method and system for clearing haze images and a storable medium.

Background technique

In recent years, the smog phenomenon has been more serious. The images taken in the haze weather, because various pollutants such as inhalable particles and fine particles are mixed in the air, have a serious impact on the absorption, refraction and scattering of light, resulting in blurred images and inconvenience for the human eye to view. The visual effect of the image is not good, which not only affects the sharpness of the image, but also causes trouble to determine the target.

The poor quality of the images taken in the haze environment greatly increases the difficulty of subsequent image processing. In addition, the presence of haze images will also have a greater impact on highway traffic monitoring and satellite remote sensing monitoring. It can be seen that the use of image processing technology to reduce the impact of adverse weather such as haze on the imaging effect, effectively improve the image quality, has broad application prospects. The smog problem is not only related to everyone's health, but also to urban safety. From the perspective of image processing, research on how to clear the haze image has become a type of problem that urgently needs to be solved in urban development.

The image dehazing technology aims to remove the interference of unfavorable factors such as fog and haze in the image, restore the effective information and features of the image, and obtain an image with good visual effects. At present, haze image sharpening processing technologies are mainly divided into two categories: image enhancement methods based on image processing, and image restoration methods based on physical models. among them:

The image enhancement method based on image processing is to enhance the image contrast, highlight or weaken some effective information and image features to reduce the interference of haze on the image, and thereby achieve the sharpness of the image. The principle of this method is simple and easy to implement, but it has the problems that the processed image is prone to color distortion and loss of image information. In other words, this type of method only reduces the interference of haze on the image, but does not remove the haze in essence. Image enhancement methods based on image processing are currently more commonly used methods based on histogram equalization, curvelet transform, homomorphic filtering, and Retinex theory.

The physical model-based image restoration method is to model the effect of atmospheric scattering from the perspective of the cause of haze formation, and then to analyze and process the haze removal. Although the image information and features are relatively complete during the processing, the lack of image information is rare. However, the process is complicated and it is greatly affected by the haze morphology. The commonly used methods are based on atmospheric physical models, based on image differences, based on dark channel priority, and based on fusion.

Summary of the invention

The technical problem to be solved by the present invention is to address the problems that the image is prone to color distortion and image information loss during the above-mentioned haze image sharpening process in the prior art, or that the haze processing process is complicated and the result is unstable and defective. Haze image sharpening method, system and storable medium.

The technical solution adopted by the present invention to solve its technical problem is to construct a method for clearing haze images, including the following steps:

S1. Randomly select several first deep network parameters to perform deep learning and haze removal on a haze image to obtain a plurality of first haze images, and perform blind image quality evaluation on the plurality of first haze images to obtain Several first image quality scores;

S2. Select a first high-quality deep network parameter from the plurality of first deep network parameters according to the first image quality score;

S3. Perform a fast bacterial swarm optimization operation on the first high-quality deep network parameters to obtain several second deep network parameters.

S4. Perform deep learning and haze removal on the haze image through the plurality of second deep network parameters to obtain a plurality of second haze images, and perform a blind image quality evaluation on the plurality of second haze images to Obtaining several second image quality scores;

S5. Select a second high-quality deep network parameter from the plurality of second deep network parameters according to the second image quality score, and determine the second high-quality based on a second image quality score corresponding to the second high-quality deep network parameter. Whether the deep network parameters are optimal, if yes, go to step S6; if not, go to step S6-1;

S6. Output a second haze image corresponding to the second high-quality deep network parameter and end the process.

S6-1. Define the second high-quality deep network parameter as a new first high-quality deep network parameter and execute step S3.

Preferably, in step S5, determining whether the second high-quality deep network parameter is optimal according to a second image quality score corresponding to the second high-quality deep network parameter includes:

Calculating a difference between a second image quality score corresponding to the second high-quality deep network parameter and a first image quality score corresponding to the first high-quality deep network parameter, and determining whether the difference satisfies a first preset condition, and if yes, It is determined that the second high-quality deep network parameter is an optimal deep network parameter.

Calculate whether a second image quality score corresponding to the second high-quality deep network parameter satisfies a second preset condition, and if so, determine that the second high-quality deep network parameter is an optimal deep network parameter.

Preferably, in the method, the first deep network parameter and the second deep network parameter are column vector parameters, and each of the column vector parameters includes m parameter values, where m is greater than or equal to 1.

Preferably, in the method, the first deep network parameter and the second deep network parameter are respectively row vector parameters, and each of the row vector parameters includes n parameter values, where n is greater than or equal to 1.

Preferably, in the step S1, performing blind image quality evaluation on the plurality of first dehaze images to obtain a plurality of first image quality scores includes:

Using the DIIVINE algorithm to perform blind image quality evaluation on the plurality of first haze images to obtain several first image quality scores;

In step S4, performing blind image quality evaluation on the plurality of second haze images to obtain a plurality of second image quality scores includes:

The DIIVINE algorithm is used to perform blind image quality evaluation on the plurality of second haze images to obtain a plurality of second image quality scores.

Preferably, in the step S3, the fast bacterial swarm optimization operation is performed on the first high-quality deep network parameter to obtain several second deep network parameters; including: constraints of the fast bacterial swarm optimization operation Conditions met:

When J ⁱ (j + 1, k, l) ＞ J _min (j, k, l),

among them:

J _min (j, k, l) is a first image quality score corresponding to the first high-quality network parameter,

Is the change of the second deep network parameter relative to the first high-quality network parameter,

θ ⁱ (j + 1, k, l) is the second deep network parameter,

C _cc is an attraction factor, and is used to indicate a change factor when the second deep network parameter changes relative to the first high-quality network parameter,

θ ^b (j, k, l) is the first high-quality network parameter.

Preferably, the C _cc includes a dynamic step size C (k, l), and the constraint condition of the dynamic step size is:

C (k, l) = L _red / n ^{k + l-1}

among them:

L _red is the initial chemotaxis step size,

n is the step gradient.

The invention also constructs a haze image sharpening system, which includes: a processor, a memory,

The memory is used to store program instructions,

The processor is configured to execute the steps of any one of the methods according to the program instructions stored in the memory.

The present invention also constructs a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method according to any one of the above are implemented.

The implementation of the haze image sharpening method, system and storable medium of the present invention has the following beneficial effects: it can achieve self-adaptation to haze weather, and objectively realize haze image clear processing based on the current haze weather. The process is fast and the results are accurate and objective.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be further described below with reference to the accompanying drawings and embodiments. In the drawings:

FIG. 1 is a program flowchart of an embodiment of a haze image sharpening method according to the present invention;

FIG. 2 is a schematic diagram showing a comparison of effects of the haze image sharpening method according to the present invention.

detailed description

In order to have a clearer understanding of the technical features, objects, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

As shown in FIG. 1, in the first embodiment of the haze image sharpening method of the present invention, the following steps are included:

S1. Randomly select a number of first deep network parameters to perform deep learning and haze removal on a haze image to obtain a number of first haze images, and perform blind image quality evaluation on the plurality of first haze images to obtain a number of first haze images. An image quality score; specifically, the haze image is input to the haze-removing deep learning network to perform the initial haze-removal of the haze image through the deep learning of the haze-removing deep learning network. During the initial haze-removal process, The initial deep network parameters of the haze deep learning network can also be understood as that the first deep network parameters are randomly selected. This can generate a set of first deep network parameters in a randomly generated manner. Here, a set of first deep network parameters can include several quantities. The first deep network parameter of the haze-removing haze deep learning network performs deep learning on the haze image based on each first deep network parameter to obtain the first haze image corresponding to the first deep network parameter, and then passes the blind The image quality evaluation method performs image quality evaluation on each first haze-free image to obtain each first The image quality score haze image. It can be understood here that a number of the first deep network parameters can be irrelevant or relatively low-level deep network parameters, so that the coverage of the deep network parameters can be expanded and the processing process and processing resources of the entire process can be reduced. Among them, deep learning networks include Convolutional Neural Networks, Autoencoders, Recurrent Neural Networks, and so on. A convolutional neural network can be used here, which uses the local correlation of the input data to greatly reduce the number of parameters of the fully connected network, and has better performance in the image recognition process.

S2. Select the first high-quality deep network parameter from a plurality of first deep network parameters according to the first image quality score. Specifically, compare the image quality scores of the first several haze images obtained above, and select the best image quality score. The first deep network parameter corresponding to the first haze image is used as the first high-quality deep network parameter.

S3. Perform a fast bacterial swarm optimization operation on the first high-quality deep network parameters to obtain several second deep network parameters. Specifically, by performing a fast bacterial swarm optimization operation on the obtained first high-quality deep network parameters, use the first The high-quality deep network parameters move closer to the historical merits of the group, and other surrounding deep network parameters that are better than the first deep network parameters are searched for, and are defined here as the second deep network parameters.

S4. Perform deep learning and haze removal on the haze image through several second deep network parameters to obtain a plurality of second haze images, and perform blind image quality evaluation on the plurality of second haze images to obtain a plurality of second image qualities. Scoring; specifically, the haze-removing deep learning network performs deep learning on the haze images separately based on the selected several second deep network parameters, obtains several corresponding second haze-removing images, and then uses blind image quality evaluation methods for each A second dehaze image is evaluated for image quality, and an image quality score is obtained for each second dehaze image.

S5. Select a second high-quality deep network parameter from a plurality of second deep network parameters according to the second image quality score, and determine whether the second high-quality deep network parameter is optimal according to the second image quality score corresponding to the second high-quality deep network parameter. If yes, go to step S6; if no, go to step S6-1; specifically, compare the image quality scores of the second dehaze images obtained above, and select the second dehaze image with the best image quality score And obtain the corresponding second deep network parameter as the second high-quality deep network parameter. Here, the quality score of the second haze image with the best second score, that is, the second image quality score corresponding to the second high-quality deep network parameter, can be used to determine whether the second high-quality deep network parameter is optimal. If the second image quality score satisfies the optimal score condition, the second high-quality deep network parameter can be used as the optimal deep network parameter during the haze de-haze process, and then the operation of step S6 can be performed. When the second image quality score does not satisfy the condition of the optimal score, it means that the second high-quality deep network parameter is not yet the optimal deep network parameter in the haze de-haze process. Then proceed to step S6-1.

S6. The second dehaze image corresponding to the second high-quality deep network parameter is output and the process ends; specifically, according to the above judgment result, when the second high-quality deep network parameter can be determined as the haze image dehaze When the optimal deep network parameters in the process, then it can be determined that the second haze image corresponding to the second high-quality deep network parameter is the optimal haze image of the haze image. In this way, the second haze image corresponding to the second high-quality deep network parameter is used as the final sharpened image output during the sharpening process of the haze image.

S6-1. Define the second high-quality deep network parameter as the new first high-quality deep network parameter and execute step S3. Specifically, according to the above judgment result, when it is determined that the second high-quality deep network parameter is not yet the optimal deep network parameter in the haze image dehaze process, the second high-quality deep network parameter is used as the new first High-quality deep network parameters. Repeat step S3 and subsequent steps to find a new second high-quality deep network parameter and make a judgment to obtain the final second high-quality deep network parameter that can optimally remove the haze image and output the corresponding. The final sharpened image output during the sharpening process of the haze image.

Here, by comparing the method of manually observing the haze degree of the image and then manually selecting the network parameter when determining network parameters in the existing method, real-time haze image sharpening processing can be performed. In addition, by using an objective blind image quality evaluation algorithm to evaluate the quality of the haze-removed image and obtain the optimal haze-removed image, it has higher accuracy than the subjective evaluation method of the existing methods. In addition, a fast bacterial swarm optimization algorithm is used to intelligently select network parameters, which can avoid the inefficiency caused by traditional exhaustive methods and quickly determine the optimal deep network parameters.

Further, in step S5, determining whether the second high-quality deep network parameter is optimal according to the second image quality score corresponding to the second high-quality deep network parameter includes: calculating a second image corresponding to the second high-quality deep network parameter. The difference between the quality score and the first quality deep network parameter corresponding to the first image quality score determines whether the difference satisfies the first preset condition, and if so, determines that the second quality deep network parameter is the optimal depth network parameter. Specifically, to determine whether the second high-quality deep network parameter is the optimal deep network parameter, the second image quality score corresponding to the second high-quality deep network parameter and the first image quality score corresponding to the first high-quality deep network parameter may be determined. For comparison, when the difference between the two is small, or when the second image quality score and the first image quality score are very close, then the optimal effect of the haze image after deep learning to remove haze has stabilized. It finds the optimal deep network parameters again, which has no effect on the effect of removing haze. Then, at this time, the second high-quality deep network parameters can be used as the optimal deep network parameters in the haze image dehazing process. The second haze image is the final clear image output.

Further, in step S5, determining whether the second high-quality deep network parameter is optimal according to the second image quality score corresponding to the second high-quality deep network parameter includes: calculating a second image corresponding to the second high-quality deep network parameter. Whether the quality score satisfies the second preset condition, and if so, determines that the second high-quality deep network parameter is an optimal deep network parameter. Specifically, in the process of removing haze from some haze images, the best output result of the haze image can also be set, that is, the best clear image that can be output and the corresponding best image can be defined. Quality score, when the second image quality score corresponding to the second high-quality deep network parameter satisfies the best quality score, then we can confirm that the output second haze image meets the best output result of its defined target , Then it can be determined that the second high-quality deep network parameter is the optimal deep network parameter, and the corresponding second haze image is the final clear image output. It can be understood here that different methods of blind image quality evaluation will have different ranges of image quality scores. The image quality scores of different algorithms meet the monotonicity principle. Among the minimum or maximum values, there must be a corresponding Good situation. Therefore, in the image sharpening process, it is only necessary to determine whether the target image quality score of the image sharpening process is the minimum or maximum value according to the type of the blind image quality evaluation algorithm.

Further, in some embodiments, in the haze image sharpening method of the present invention, the first deep network parameter and the second deep network parameter are column vector parameters, and each column vector parameter includes m parameter values, where m Greater than or equal to 1. In other embodiments, in the haze image sharpening method of the present invention, the first deep network parameter and the second deep network parameter are row vector parameters, respectively, and each of the row vector parameters includes n parameter values, where n is greater than or Is equal to 1. Specifically, different haze-removing deep learning networks may have different numbers and physical meaning parameters due to their different structures, such as column vector parameters, row vector parameters, and matrix parameters. For example, the following row vector parameters,

among them:

b _i ——the i-th deep network parameter vector;

-The n-th parameter value of the i-th deep network parameter vector.

Specifically, in step S1, performing blind image quality evaluation on several first haze-free images to obtain several first image quality scores includes: using a DIIVINE algorithm to perform blind image quality evaluation on several first haze-free images to obtain A number of first image quality scores; in step S4, blind image quality evaluation is performed on a plurality of second dehaze images to obtain a plurality of second image quality scores, including: using the DIIVINE algorithm to perform blind images on a plurality of second dehaze images Quality evaluation to obtain several second image quality scores. Specifically, in some embodiments, the blind image quality evaluation of the haze image may use the DIIVINE algorithm. Of course, in other embodiments, other methods may also be used for the first haze image and the second haze image, respectively. Image for blind image quality evaluation. It can be understood here that when a blind image quality evaluation method is selected in the entire process to perform a blind image quality evaluation on the haze-removed image, a blind image quality evaluation method must always be used to complete the entire process. Process. It is not suitable to replace the blind image quality evaluation scheme in the process to perform blind image quality evaluation on the haze-removed images at different stages.

Further, in step S3, a fast bacterial swarm optimization operation is performed on the first high-quality deep network parameters to obtain several second deep network parameters; including: the constraints of the fast bacterial swarm optimization operation satisfy:

When J ⁱ (j + 1, k, l) ＞ J _min (j, k, l),

among them:

J _min (j, k, l) is a first image quality score corresponding to a first high-quality network parameter,

Is the second deep network parameter,

θ ⁱ (j + 1, k, l) is the first deep network parameter,

θ ^b (j, k, l) is the first high-quality network parameter.

Specifically, using the new quorum sensing mechanism described above will allow the first deep network parameters to use the experience of the surrounding first high-quality network parameters to guide their changes, which can greatly reduce the algorithm's search time in the solution space. At the same time, the new mechanism helps the first deep network parameters to jump out of the local optimal solution, which can effectively reduce the possibility of the first high-quality network parameters escaping from the global best advantage.

Further, C _cc includes a dynamic step size C (k, l), and the constraints of the dynamic step size are:

C (k, l) = L _red / n ^{k + l-1}

among them:

L _red is the initial chemotaxis step size,

n is the step gradient.

Specifically, C (k, l) shows a downward trend as the number of replication and elimination-dispelling events increases. When k + l is small, C (k, l) is large, which can avoid excessive search time in a local area; when k + l is gradually increased, C (k, l) is shortened, which can enhance the first high-quality network parameter The local search ability near the global best advantage ensures that the algorithm eventually approaches the global best advantage.

In addition, a haze image sharpening system of the present invention includes: a processor, a memory, and a memory, for storing program instructions, and a processor for performing the steps of any one of the foregoing methods according to the program instructions stored in the memory. Specifically, the above-mentioned haze image sharpening method may be performed by a haze image sharpening system to output a final sharpened image.

In addition, a computer-readable storage medium of the present invention stores a computer program thereon. When the computer program is executed by a processor, the steps of any one of the methods above are implemented. Specifically, the method described above can be stored as a program and copied. The computer-readable storage medium herein may be, but is not limited to, a tangible device that can hold and store instructions used by the instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electric storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. And a combination of devices including the above storage device.

As shown in FIG. 2, it is a comparison chart between the processing results of the haze image sharpening method of the present invention and the results of the existing processing methods, where column a is the original haze image, that is, the front fog to be cleared Haze image, b, c, and d are the clearing results of the existing haze image, and e is the processing result of the haze image clearing method of the present invention. It can be seen that the effect is better than that of the existing haze image. The result of the method is equivalent to that of the existing clearing method of the haze image. But the process is far superior to the existing methods of sharpening haze images.

It can be understood that the above embodiments only express the preferred embodiments of the present invention, and their descriptions are more specific and detailed, but they should not be construed as a limitation on the scope of the patent of the present invention; In other words, without departing from the concept of the present invention, the above technical features can be freely combined, and several modifications and improvements can be made, all of which belong to the protection scope of the present invention; All equivalent transformations and modifications made shall fall within the scope of the claims of the present invention.

Claims

A haze image sharpening method includes the following steps:

S1. Randomly select several first deep network parameters to perform deep learning and haze removal on a haze image to obtain a plurality of first haze images, and perform blind image quality evaluation on the plurality of first haze images to obtain Several first image quality scores;

S2. Select a first high-quality deep network parameter from the plurality of first deep network parameters according to the first image quality score;

S3. Perform a fast bacterial swarm optimization operation on the first high-quality deep network parameters to obtain several second deep network parameters.

S4. Perform deep learning and haze removal on the haze image through the plurality of second deep network parameters to obtain a plurality of second haze images, and perform a blind image quality evaluation on the plurality of second haze images to Obtaining several second image quality scores;

S5. Select a second high-quality deep network parameter from the plurality of second deep network parameters according to the second image quality score, and determine the second high-quality based on a second image quality score corresponding to the second high-quality deep network parameter. Whether the deep network parameters are optimal, if yes, go to step S6; if not, go to step S6-1;

S6. Output a second haze image corresponding to the second high-quality deep network parameter and end the process.

S6-1. Define the second high-quality deep network parameter as a new first high-quality deep network parameter and execute step S3.
The haze image sharpening method according to claim 1, wherein in the step S5, the second high-quality image is judged according to a second image quality score corresponding to the second high-quality deep network parameter. Whether the deep network parameters are optimal, including:

Calculating a difference between a second image quality score corresponding to the second high-quality deep network parameter and a first image quality score corresponding to the first high-quality deep network parameter, and determining whether the difference satisfies a first preset condition, and if yes, It is determined that the second high-quality deep network parameter is an optimal deep network parameter.
The haze image sharpening method according to claim 1, wherein in the step S5, the second high-quality image is judged according to a second image quality score corresponding to the second high-quality deep network parameter. Whether the deep network parameters are optimal, including:

Calculate whether a second image quality score corresponding to the second high-quality deep network parameter satisfies a second preset condition, and if so, determine that the second high-quality deep network parameter is an optimal deep network parameter.
The haze image sharpening method according to claim 1, wherein in the method, the first deep network parameter and the second deep network parameter are column vector parameters, and the column vector parameters are Each includes m parameter values, where m is greater than or equal to 1.
The haze image sharpening method according to claim 1, wherein in the method, the first deep network parameter and the second deep network parameter are row vector parameters, and the row vector parameters are Each includes n parameter values, where n is greater than or equal to 1.
The haze image sharpening method according to claim 1, wherein in the step S1, the blind image quality evaluation is performed on the plurality of first dehaze images to obtain a plurality of first image quality scores ,include:

Using the DIIVINE algorithm to perform blind image quality evaluation on the plurality of first haze images to obtain several first image quality scores;

In step S4, performing blind image quality evaluation on the plurality of second haze images to obtain a plurality of second image quality scores includes:

The DIIVINE algorithm is used to perform blind image quality evaluation on the plurality of second haze images to obtain a plurality of second image quality scores.
The haze image sharpening method according to claim 1, wherein in the step S3, the fast bacterial swarm optimization operation is performed on the first high-quality deep network parameters to obtain a plurality of second depths. Network parameters include: the constraints of the rapid bacterial swarm optimization operation satisfy:

When J i (j + 1, k, l) ＞ J min (j, k, l),

among them:

J min (j, k, l) is a first image quality score corresponding to the first high-quality network parameter,

Is the change of the second deep network parameter relative to the first high-quality network parameter,

θ i (j + 1, k, l) is the second deep network parameter,

C cc is an attraction factor, and is used to indicate a change factor when the second deep network parameter changes relative to the first high-quality network parameter,

θ b (j, k, l) is the first high-quality network parameter.
The haze image sharpening method according to claim 7, wherein the C cc includes a dynamic step size C (k, l), and a constraint condition of the dynamic step size is:

C (k, l) = L red / n k + l-1

among them:

L red is the initial chemotaxis step size,

n is the step gradient.
A haze image sharpening system is characterized in that it includes: a processor, a memory,

The memory is used to store program instructions,

The processor is configured to execute the steps of the method according to any one of claims 1-8 according to program instructions stored in the memory.
A computer-readable storage medium having stored thereon a computer program, characterized in that when the computer program is executed by a processor, the steps of the method according to any one of claims 1-8 are implemented.