CN116957984A - Method and system for monitoring unmanned aerial vehicle based on low-illuminance haze - Google Patents

Abstract

The application provides a method and a system for monitoring an unmanned aerial vehicle based on low-illumination haze, and relates to the field of image processing; the method comprises the steps of preprocessing a low-illumination haze image in a mixed mode of side window filtering and rapid edge protection filtering to obtain a more accurate ambient light estimated value; then combining the image and the obtained ambient light estimated value, setting a light source threshold value, and separately solving the transmittance of the light source region and the transmittance of the non-light source region through a self-adaptive light source matrix mechanism, so as to further refine and solve the light source region and better reserve the brightness value of the light source region; then, the transmissivity is subjected to fusion solution, and light source compensation is carried out on the transmissivity image in a gamma correction mode; and finally, defogging the image under the low-illumination haze by combining an atmospheric scattering model. The processed image is bright as a whole, the light source area has no halation effect, the sky area has small chromatic aberration, the image has clear texture details and vivid colors, and people can efficiently monitor the unmanned aerial vehicle and observe dangerous articles carried by the unmanned aerial vehicle.

Description

A method and system for monitoring drones under low illumination haze

Technical field

The present invention relates to the field of image processing technology, and specifically relates to a method and system for monitoring drones under low illumination haze.

Background technique

With the development of science and technology, people have paid more and more attention to drones in recent years. In addition to military applications, the application scenarios of drones are constantly expanding in the civilian field. For example, in the fields of entertainment aerial photography, agricultural plant protection, police security, power inspection, drone surveying and mapping, logistics and transportation, choreography and performances, drones have performed well. However, while drone technology brings benefits, it also has some drawbacks. For the aviation sector, there are problems such as difficulty in discovery, supervision, and law enforcement. During the period of take-off and landing of the aircraft, if the actions of the drone are not recognized in time and some dangerous items that may be carried by the drone are observed, it may easily cause the aircraft to crash and threaten people's lives and interests. Loss, and the prerequisite for controlling drones is to be able to effectively monitor drones.

For the working environment of drones. In daytime scenes, image collection equipment can easily capture drones, monitor their movements, and identify dangerous items they carry. However, when the drone is in an environment with low illumination and accompanied by haze, it becomes more difficult to monitor. The current low-illumination defogging method still has many shortcomings. There are problems such as poor color saturation, blurred texture details, and high noise. Existing Technology: To address the problem of uneven illumination at night, some researchers have proposed a dehazing algorithm that first compensates for illumination and then corrects the color. Although the color effect seems to be improved, due to the inaccurate illumination estimation during compensation, the shining area in the picture is not processed enough. Good, resulting in obvious halo and large noise in the restored image; some researchers proposed to use illumination compensation to achieve dehazing, and then color correction after dehazing, but failed to reasonably estimate the transmission, resulting in color distortion of the final restored image, and the dehazing effect Poor. In addition, some researchers believe that artificial light sources at night have phenomena such as glare and uneven illumination, so they add the glare layer to the standard daytime defogging model, remove the glare layer to obtain the layer separation results, and then re-block the nighttime atmospheric light to estimate the nighttime atmospheric light, using dark Channel theory estimates the transmittance and then obtains the restored image; although the above method has a better dehazing effect, due to the lack of illumination compensation, brightness enhancement and other processing, the restored image is overall dark and the texture details are not clear enough; and after the image is dehazed , the display is too dark, resulting in a loss of details.

Some published patent application documents also propose some solutions for monitoring drones, such as patent application CN108734670A and patent application CN115170404A. Although the above-mentioned public scheme has improved the low-illumination dehazing method, its application is too broad and cannot be used to process drone images in low-illumination haze situations. For example, the actual working status of the drone is not considered; when the drone is working under low illumination conditions, it is equipped with lights. These lights will affect the dehazed image and easily produce a halo effect. In addition, when there is a large sky area, serious effects such as color cast and artifacts will occur in the sky area after dehazing, resulting in poor image dehazing effect and making it difficult to better observe the drone. The patent application CN115616479A discloses the use of special devices and systems to monitor drones, but it is impossible to directly understand the dangerous items carried by the drone itself, and these dangerous items may cause major accidents.

Therefore, based on the current technical problems of drone monitoring in special scenes, a low-light defogging method that can be applied to special scenes is needed to achieve better monitoring of drones and identify the objects carried by drones. of dangerous goods.

Contents of the invention

The purpose of the present invention is to provide a method and system for monitoring UAVs under low illumination haze. Based on the dark channel prior theory, the dehazing method is improved to improve the quality of the output image and solve the problem of low illumination haze. For the problem of drone monitoring, the purpose of identifying dangerous items carried by drones under complex conditions is achieved.

In order to achieve the above objects, the present invention proposes the following technical solutions:

The first aspect is to disclose a method for monitoring drones under low illumination haze, including:

Obtain the pixels of the input image, use a hybrid method of edge window filtering and fast edge-preserving filtering to preprocess the input image, and obtain the ambient light estimate of the input image;

Set the light source threshold of the input image according to the ambient light estimate;

Divide the light source area and non-light source area of the input image according to the light source threshold, use an adaptive light source matrix mechanism to optimize the transmittance of the light source area and non-light source area respectively, and then fuse and solve the initial transmittance;

Perform light source compensation on the initial transmittance to obtain and output the final transmittance of the input image;

According to the ambient light estimate and the final transmittance, a defogging calculation is performed on the input image according to an atmospheric scattering model to obtain a dehazed output image.

Further, the process of processing the input image using a hybrid method of edge window filtering and fast edge-preserving filtering includes:

Obtain the pixels of the input image, use side window filtering to treat each pixel as a potential edge, and generate several edge windows around each pixel;

Processing the input image, including adjusting the brightness of the input image, obtaining and outputting an edge window with the smallest Euclidean distance between each pixel and the input image, so as to retain the edge information of the input image;

Obtain a filtered image according to the output edge window with the smallest distance, use fast edge-preserving filtering to process the filtered image, and obtain a preprocessed image;

Based on the preprocessed image, the ambient light estimate is calculated using a dark channel prior method.

Further, the process of setting the light source threshold of the input image according to the ambient light estimate includes:

Calculate the photometric value of each pixel in the input image;

The difference between the photometric value and the estimated ambient light value is calculated, and the maximum absolute value of the difference is used as the light source threshold.

Further, the process of dividing the light source area and non-light source area of the input image according to the light source threshold, using an adaptive light source matrix mechanism to respectively optimize the transmittance of the light source area and the non-light source area, and then fusion to solve the initial transmittance includes: :

For any pixel in the input image, classify the pixel as belonging to the light source area or non-light source area; wherein, when the luminosity value of the pixel is greater than the light source threshold, the pixel is classified as belonging to the light source area, otherwise the pixel is classified as belonging to the non-light source area;

When the pixel belongs to the light source area, an adaptive light source matrix mechanism is used to calculate and optimize the transmittance of the light source area; including:

According to the light source matrix mechanism, each pixel in the input image is numbered in descending order of photometric value, and the first light source influence matrix, the first light source influence matrix and the pixel light source influence matrix of each pixel are calculated; specifically, the input image is defined _{The size of} }, where T is the total number of pixels; then,

Under the value of x, the first light source influence matrix k _x is calculated as follows:

Under the value of y, the second light source influence matrix k _y is calculated as follows:

The pixel light source influence matrix K _x is calculated as follows:

Among them, C _x and C _y represent the luminosity value of the pixel point corresponding to the value of x and y, M is the light source threshold, d _{x and m} represent the distance from other pixels to the selected pixel point;

According to the pixel light source influence matrix, the adjustment correction coefficient w _x that optimizes the transmittance of the light source area is obtained,

Among them, α represents the adjustment correction factor of the input image, t _x initial transmittance;

When the pixel belongs to a non-light source area, the transmittance of the non-light source area is calculated through the dark channel prior theory;

Based on the optimized transmittance of the light source area and the transmittance of the non-light source area, the initial transmittance t _M is calculated through fusion,

Among them, Ω represents the light source area, represents the transmittance of the non-light source area, w _x t _x∈Ω represents the transmittance of the light source area.

Further, the process of performing light source compensation on the initial transmittance and obtaining and outputting the final transmittance of the input image is:

Using the gamma correction method, the initial transmittance is compensated for the light source by adjusting the compensation coefficient to obtain the final transmittance.

Further, the process of obtaining the dehazed output image includes:

According to the ambient light estimate and the final transmittance, any pixel in the input image is dehazed according to the atmospheric scattering model. The formula is:

Among them, I _x represents the initial image of any pixel in the input image, t _F is the final transmittance, and J _x represents the image after dehazing of the pixel;

The dehazed image of each pixel is combined to obtain the dehazed output image.

In the second aspect, a drone system for monitoring under low illumination haze is disclosed, including:

The acquisition processing module is used to acquire the pixels of the input image, preprocess the input image using a hybrid method of edge window filtering and fast edge-preserving filtering, and obtain the ambient light estimate of the input image;

A setting module configured to set the light source threshold of the input image according to the ambient light estimate;

A division and solution module, used to divide the light source area and non-light source area of the input image according to the light source threshold, use an adaptive light source matrix mechanism to respectively optimize the transmittance of the light source area and the non-light source area, and then fuse and solve the initial transmittance;

A compensation module, used to perform light source compensation on the initial transmittance, and obtain and output the final transmittance of the input image;

A calculation module configured to perform dehazing calculation on the input image based on the ambient light estimate and the final transmittance and an atmospheric scattering model to obtain a dehazed output image.

Further, the execution unit for obtaining the ambient light estimate of the input image by the acquisition processing module includes:

The acquisition unit is used to obtain the pixels of the input image, uses side window filtering to treat each pixel as a potential edge, and generates several edge windows around each pixel;

An adjustment unit, configured to process the input image, including adjusting the brightness of the input image, obtaining and outputting an edge window with the smallest Euclidean distance between each pixel and the input image, so as to retain the edge information of the input image;

A processing unit, configured to obtain a filtered image according to the output edge window with the smallest distance, and use fast edge-preserving filtering to process the filtered image to obtain a preprocessed image;

A calculation unit configured to calculate the ambient light estimate using a dark channel prior method based on the preprocessed image.

In a third aspect, a computer device is disclosed, including at least one processor, the processor being coupled to a memory, the memory storing programs or instructions running on the processor, and the programs or instructions being processed by the processor. When the device is executed, the above-mentioned steps based on the method of monitoring UAVs under low illumination haze are implemented.

The fourth aspect discloses a readable storage medium on which a program or instructions are stored, characterized in that when the program or instructions are executed by a processor, the above-mentioned method based on monitoring drones under low illumination haze is realized. step.

It can be seen from the above technical solutions that the technical solutions of the present invention achieve the following beneficial effects:

The method and system for monitoring drones under low illumination and haze disclosed in the present invention are intended to solve the problem of monitoring drones under low illumination and haze conditions. The advantages of specific applications are as follows:

(1) The solution uses a hybrid method of edge window filtering and fast edge-preserving filtering to preprocess low-illumination haze images to eliminate the impact of high light source areas and pseudo light source areas on the image to solve the ambient light estimation value, and improve the ambient light estimation The accuracy of the value; it solves the problem of low-illumination haze images being too dark after dehazing, allowing drones to be detected more clearly.

(2) The plan sets the light source threshold according to the brightness value of the image and the processed ambient light estimate, uses the light source matrix mechanism to solve the transmittance fusion of different light source areas, and then performs light source compensation through gamma correction to obtain the final transmission High-efficiency images solve the problem of the halo effect in the light source area of low-illumination images after dehazing, improve the texture details of the image, and can better detect drones in low-illumination haze situations and identify the objects carried by drones. of dangerous goods.

(3) The solution improves the display effect of the sky area by modifying the ambient light estimation and optimizing the solution process of transmittance, reduces the chromatic aberration, solves the problem of artifacts and noise, and prevents the monitoring effect of the drone from being affected. The dangerous goods carried cannot be clearly identified.

It should be understood that all combinations of the foregoing concepts as well as additional concepts described in more detail below can be considered part of the inventive subject matter of the present disclosure so long as such concepts are not inconsistent with each other.

The foregoing and other aspects, embodiments and features of the present teachings will be more fully understood from the following description taken in conjunction with the accompanying drawings. Additional aspects of the invention, such as features and/or advantages of the exemplary embodiments, will be apparent from the description which follows, or may be learned by practice of specific embodiments in accordance with the teachings of the invention.

Description of the drawings

The drawings are not meant to be drawn to actual reference scale. In the drawings, each identical or approximately identical component shown in various figures may be designated by the same reference numeral. For clarity, not every component is labeled in each figure. Embodiments of various aspects of the invention will now be described by way of example and with reference to the accompanying drawings, in which:

Figure 1 is a flow chart of a method for monitoring drones under low illumination haze disclosed by the present invention;

Figure 2 is a flow chart of the present invention for obtaining the ambient light estimation value of the input image;

Figure 3 is a flow chart for setting the light source threshold of the input image according to the present invention;

Figure 4 is a framework diagram of a UAV system for monitoring under low illumination haze disclosed by the present invention;

Figure 5 is a schematic diagram of an electronic device according to an embodiment of the present application;

(a), (b), and (c) in Figure 6 are all pictures taken by UAV monitoring under low illumination haze conditions in the embodiment;

(a), (b), and (c) in Figure 7 are the renderings after the hybrid filtering process corresponding to the drone monitoring in Figure 6;

(a), (b), and (c) in Figure 8 are the transmittance images corresponding to the fusion solution of the adaptive light source matrix mechanism in Figure 7;

(a), (b), and (c) in Figure 9 are corresponding to the UAV monitoring renderings after dehazing in Figure 8.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the described embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention. Unless otherwise defined, technical or scientific terms used herein shall have their ordinary meaning understood by a person of ordinary skill in the art to which this invention belongs.

"First", "second" and similar words used in the specification and claims of the patent application of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Likewise, the singular forms "a", "an" or "the" and similar words do not imply a limitation on quantity but rather indicate the presence of at least one, unless the context clearly dictates otherwise. The words "include" or "include" and similar words mean that the elements or things appearing before "include" or "include" include the features, integers, steps, operations, elements and/or listed after "include" or "include". or component, does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and/or collections thereof. "Up", "down", "left", "right", etc. are only used to express relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

Based on the existing technology, there are the following problems when using image technology to monitor drones in low-light haze conditions, including: 1) The existing dark channel prior theory, when processing daytime haze images, the selected ambient light The estimated value is relatively simple. At night, it is affected by various lights, causing deviations in the solution of ambient light values, and the dehazing effect is darker; 2) Although guided filtering can alleviate the problems caused by image blocking when processing transmittance The block effect problem of the transmittance image, but when the rough transmittance values of two adjacent blocks differ greatly, the residual block effect is still significant, which will lead to an obvious halo effect near strong light in the final output image; and Although the existing algorithm has been improved, it ignores the influence of the sky area, causing chromatic aberration and artifacts in the sky area of the image. Therefore, the present invention proposes a low-illumination defogging method for pictures that can be applied in special scenes such as low-illumination haze to solve the above problems and realize the monitoring of drones and the identification of dangerous items carried.

The following is a further detailed introduction to the method and system for monitoring drones under low illumination haze disclosed in the present invention with reference to the embodiments shown in the accompanying drawings.

As shown in Figure 1, the method disclosed in the embodiment for monitoring drones under low illumination haze includes the following steps:

Step S102, obtain the pixels of the input image, use a hybrid method of edge window filtering and fast edge-preserving filtering to preprocess the input image, and obtain the ambient light estimate of the input image; the purpose of this step is to detect the highlight points and pseudo light source points of the image Processing is performed to eliminate the impact on ambient light estimation.

Step S104, set the light source threshold of the input image according to the ambient light estimation value;

Step S106, divide the light source area and non-light source area of the input image according to the light source threshold, use an adaptive light source matrix mechanism to optimize the transmittance of the light source area and the non-light source area respectively, and then fuse and solve the initial transmittance;

Step S108, perform light source compensation on the initial transmittance, and obtain and output the final transmittance of the input image;

Step S110: Perform defogging calculation on the input image based on the ambient light estimate value and final transmittance based on the atmospheric scattering model to obtain a dehazed output image.

Specifically as shown in Figure 2, the process of processing the input image using a hybrid method of edge window filtering and fast edge-preserving filtering includes the following steps: Step S1022, obtain the pixels of the input image, and use edge window filtering to treat each pixel equally. as potential edges, and generate several edge windows around each pixel; step S1024, process the input image, including adjusting the brightness of the input image, and obtain and output the edge window with the smallest Euclidean distance between each pixel and its input image, so that Preserve the edge information of the input image; Step S1026, obtain a filtered image according to the output edge window with the smallest distance, use fast edge-preserving filtering to process the filtered image, and obtain a preprocessed image; Step S1028, according to the Preprocess the image and calculate the ambient light estimate using a dark channel prior.

Among them, the calculation process of using side window filtering to obtain the edge window with the smallest distance from step S1022 to step S1024 is as follows:

Among them, S = {U, D, L, R, NM, NE, SW, SE}, represents the set of eight edge windows generated around the pixel; n∈S, represents any edge window of the set S; I _n represents the edge The filter value of the window; q _i represents the weight of the pixel j near the target pixel i based on the kernel function F; w _ij is the pixel value of the pixel; N _n represents the sum of the pixel values of the eight edge windows; I _sw represents the Euclidean distance from the pixel input image Minimal edge window.

Step S104 is a process of setting the light source threshold of the input image according to the estimated ambient light value. As shown in Figure 3, it includes the following steps: Step S1042, calculate the photometric value of each pixel in the input image; Step S1044, calculate the photometric value The difference between the ambient light estimate and the ambient light estimate is the largest absolute value of the difference as the light source threshold.

That is, the luminosity value I _x of each pixel point in the image is obtained, and the difference is made with the ambient light estimated value A, and |I _x -A| _max is used as the light source threshold to distinguish the light source area from the non-light source area.

In a specific embodiment, the above-mentioned step S106 divides the light source area and non-light source area of the input image according to the light source threshold, uses an adaptive light source matrix mechanism to separately optimize the transmittance of the light source area and the non-light source area, and then fuses and solves the initial transmittance. process, including the following calculation process:

For any pixel in the input image, classify the pixel as belonging to the light source area or non-light source area; wherein, when the luminosity value of the pixel is greater than the light source threshold, the pixel is classified as belonging to the light source area, otherwise the pixel is classified as belonging to the non-light source area;

When the pixel belongs to the light source area, that is, I _x > |I _x -A| _max , the adaptive light source matrix mechanism is used to calculate and optimize the transmittance of the light source area; including:

According to the light source matrix mechanism, each pixel in the input image is numbered in descending order of photometric value, and the first light source influence matrix, the first light source influence matrix and the pixel light source influence matrix of each pixel are calculated; specifically, the input image is defined _{The size of} }, where T is the total number of pixels; then,

Under the value of x, the first light source influence matrix k _x is calculated as follows:

Under the value of y, the second light source influence matrix k _y is calculated as follows:

The pixel light source influence matrix K _x is calculated as follows:

Among them, C _x and C _y represent the luminosity value of the pixel point corresponding to the value of x and y, M is the light source threshold, d _{x and m} represent the distance from other pixels to the selected pixel point;

According to the pixel light source influence matrix, the adjustment correction coefficient w _x that optimizes the transmittance of the light source area is obtained,

Among them, α represents the adjustment correction factor of the input image, which is set to 0.5 in the embodiment; t _x represents the initial transmittance;

When the pixel belongs to the non-light source area, that is, I _x ≤ |I _x -A| _max , the transmittance of the non-light source area is calculated through the dark channel prior theory;

Based on the optimized transmittance of the light source area and the transmittance of the non-light source area, the initial transmittance t _M is calculated through fusion,

Among them, Ω represents the light source area, represents the transmittance of the non-light source area, w _x t _x∈Ω represents the transmittance of the light source area.

Further combined with the embodiment, the present invention uses a gamma correction method to perform light source compensation on the initial transmittance by adjusting the compensation coefficient, and finally outputs the final transmittance of the input image; during implementation, the compensation coefficient is set to 0.8.

The process of obtaining the defogged output image in step S110 of the present invention includes the following calculation process:

First, based on the ambient light estimate and final transmittance, any pixel in the input image is dehazed according to the atmospheric scattering model. The formula is:

Among them, I _x represents the initial image of any pixel in the input image, t _F is the final transmittance, and J _x represents the image after dehazing of the pixel;

Then, the dehazed image of each pixel is combined to obtain the dehazed output image.

The method disclosed by the present invention based on monitoring drones under low-illumination haze first pre-processes low-illumination haze images through a mixture of edge window filtering and fast edge-preserving filtering to eliminate the pairing of high light source areas and false light source areas. The purpose of image solving is to solve the impact of ambient light estimates; then set the light source threshold according to the image and ambient light estimates, use the light source matrix mechanism to solve the transmittance fusion of different light source areas, and then perform light source compensation through gamma correction to obtain the final transmission rate image to solve the problem of halo effect in the light source area of low-illumination images after dehazing; finally, by modifying the ambient light estimation and optimizing the solution process of transmittance, the display effect of the sky area is improved, chromatic aberration is reduced, and the image processing process is easily solved Problems with artifacts and noise. The method of the present invention can be effectively applied to the problem of UAV identification in special scenes such as low illumination haze and improves the safety management and control of UAVs.

In an embodiment of the present invention, a computer device is also provided. As shown in Figure 5, the device includes at least one processor, the processor is coupled to a memory, and the memory stores programs that run on the processor. or instructions. When the programs or instructions are executed by the processor, the steps of the method for monitoring UAVs under low illumination haze as disclosed in the above embodiments are implemented.

The above-mentioned program can be run in the processor, or can also be stored in the memory, that is, in a computer-readable medium. The computer-readable medium includes permanent and non-permanent, removable and non-removable media and can be stored in any method or technology. Implement information storage. Information may be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), and read-only memory. (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, Magnetic tape cassettes, tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary storage computer-readable media, such as modulated data signals and carrier waves.

These computer programs may also be loaded onto a computer or other programmable data processing device such that a series of operating steps are performed on the computer or other programmable device to produce computer-implemented processes, thereby causing instructions to be executed on the computer or other programmable device Steps are provided for implementing the functions specified in one process or multiple processes of the flowchart and/or one or more boxes of the block diagram, which may be implemented by different modules corresponding to different method steps.

In this embodiment, such a device or system is provided. The system can be called a drone system for monitoring drones under low illumination and haze. The system is shown in Figure 4 and includes: an acquisition processing module. In order to obtain the pixels of the input image, the input image is preprocessed using a hybrid method of edge window filtering and fast edge-preserving filtering to obtain an estimated ambient light value of the input image; a setting module is used to set the input image according to the estimated ambient light value. The light source threshold; the division and solution module is used to divide the light source area and the non-light source area of the input image according to the light source threshold, use the adaptive light source matrix mechanism to respectively optimize the transmittance of the light source area and the non-light source area, and then fuse and solve the initial transmission rate; the compensation module is used to perform light source compensation on the initial transmittance, and obtain and output the final transmittance of the input image; the calculation module is used to calculate the atmospheric scattering model according to the ambient light estimate and the final transmittance. Perform defogging calculation on the above input image to obtain the defogging output image.

This system is used to implement the steps of the drone system based on low illumination haze monitoring disclosed in the above embodiments. The steps have already been explained and will not be repeated here.

For example, the acquisition processing module obtains the execution unit of the ambient light estimate of the input image, including: an acquisition unit, used to obtain the pixels of the input image, using side window filtering to treat each pixel as a potential edge, and generate a signal around each pixel. Several edge windows; an adjustment unit for processing the input image, including adjusting the brightness of the input image, obtaining and outputting an edge window with the smallest Euclidean distance between each pixel and the input image, so as to retain the edge information of the input image ; The processing unit is used to obtain a filtered image according to the output edge window with the smallest distance, and uses fast edge-preserving filtering to process the filtered image to obtain a preprocessed image; the computing unit is used to obtain a preprocessed image according to the preprocessed image, The ambient light estimate is calculated using a dark channel prior.

Among them, the process of the adjustment unit obtaining the edge window with the smallest Euclidean distance between each pixel and its input image is as follows:

Among them, S = {U, D, L, R, NM, NE, SW, SE}, represents the set of eight edge windows generated around the pixel; n∈S, represents any edge window of the set S; I _n represents the edge The filter value of the window; q _i represents the weight of the pixel j near the target pixel i based on the kernel function F; w _ij is the pixel value of the pixel; N _n represents the sum of the pixel values of the eight edge windows; I _sw represents the Euclidean distance from the pixel input image Minimal edge window.

For another example, the setting module sets the light source threshold of the input image according to the estimated ambient light value, including the following execution unit: a second calculation unit, used to calculate the luminosity value of each pixel in the input image; a third calculation unit, It is used to calculate the difference between the photometric value and the estimated ambient light value, and the absolute value of the largest difference is used as the light source threshold.

For another example, the division and solution module divides the light source area and non-light source area of the input image according to the light source threshold, uses an adaptive light source matrix mechanism to respectively optimize the transmittance of the light source area and the non-light source area, and then merges and solves the process of initial transmittance. Including the following calculation process:

For any pixel in the input image, classify the pixel as belonging to the light source area or non-light source area; wherein, when the luminosity value of the pixel is greater than the light source threshold, the pixel is classified as belonging to the light source area, otherwise the pixel is classified as belonging to the non-light source area;

When the pixel belongs to the light source area, that is, I _x > |I _x -A| _max , the adaptive light source matrix mechanism is used to calculate and optimize the transmittance of the light source area; including:

According to the light source matrix mechanism, each pixel in the input image is numbered in descending order of photometric value, and the first light source influence matrix, the first light source influence matrix and the pixel light source influence matrix of each pixel are calculated; specifically, the input image is defined _{The size of} }, where T is the total number of pixels; then,

Under the value of x, the first light source influence matrix k _x is calculated as follows:

Under the value of y, the second light source influence matrix k _y is calculated as follows:

The pixel light source influence matrix K _x is calculated as follows:

Among them, C _x and C _y represent the luminosity value of the pixel point corresponding to the value of x and y, M is the light source threshold, d _{x and m} represent the distance from other pixels to the selected pixel point;

According to the pixel light source influence matrix, the adjustment correction coefficient w _x that optimizes the transmittance of the light source area is obtained,

Among them, α represents the adjustment correction factor of the input image, which is set to 0.5 in the embodiment; t _x represents the initial transmittance;

When the pixel belongs to the non-light source area, that is, I _x ≤ |I _x -A| _max , the transmittance of the non-light source area is calculated through the dark channel prior theory;

Based on the optimized transmittance of the light source area and the transmittance of the non-light source area, the initial transmittance t _M is calculated through fusion,

Among them, Ω represents the light source area, represents the transmittance of the non-light source area, w _x t _x∈Ω represents the transmittance of the light source area.

As another example, the process of the computing module obtaining the output image after dehazing includes the following execution units:

The fourth calculation unit is used to perform defogging calculations for any pixel in the input image based on the ambient light estimate and final transmittance and the atmospheric scattering model. The formula is:

Among them, I _x represents the initial image of any pixel in the input image, t _F is the final transmittance, and J _x represents the image after dehazing of the pixel;

The output unit is used to synthesize the dehazed image of each pixel point to obtain a dehazed output image.

The specific implementation process of the present invention is fully explained below in conjunction with the drone flight experiment in the low-illumination haze scene shown in the accompanying drawings.

First, a mixed processing method of edge window filtering and fast edge-preserving filtering is used to preprocess the light source points to eliminate the impact of the brightness values of high light source points and pseudo light source points on the solution of ambient light estimates in low-illumination haze scenes; specifically, As shown in Figures 6 and 7, Figure 6 is a direct shot under low illumination haze conditions, and Figure 7 is a processed image, which improves the accuracy of the ambient light value.

Secondly, the difference between the brightness value of each pixel in the image and the estimated ambient light value is calculated, and the maximum value is used as the light source threshold to distinguish the light source area from the non-light source area; for the non-light source area, the transmittance is solved using the traditional dark channel prior theory; For the light source area, after optimization using the light source matrix mechanism, the transmittance image is solved through fusion; then, the transmittance image is compensated for the light source through gamma correction to obtain the final transmittance image, as shown in Figure 8; from Figure It can be seen that the overall brightness and details have been greatly improved, so that the light source area is well preserved, and the halo effect is less likely to occur during dehazing.

Finally, the ambient light estimate and transmittance image processed in the above steps are used to dehaze the low-illumination haze image to obtain the output image after dehazing; as shown in Figure 9, the overall image is bright and clear, and the light source area The light is maintained well, without halo effect, reducing the problem of chromatic aberration in the sky area, without the influence of artifacts and noise. It can effectively monitor the drone in low illumination haze fields, and during the take-off and landing of the aircraft. During this time, the actions of nearby drones are identified to avoid major accidents.

Although the present invention has been disclosed above in terms of preferred embodiments, these are not intended to limit the present invention. Those with ordinary skill in the technical field to which the present invention belongs can make various modifications and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the claims.

Claims (10)

1. The method for monitoring the unmanned aerial vehicle based on low-illumination haze is characterized by comprising the following steps of:

acquiring pixels of an input image, and preprocessing the input image by adopting a mixed mode of edge window filtering and rapid edge protection filtering to obtain an ambient light estimated value of the input image;

setting a light source threshold of an input image according to the ambient light estimated value;

dividing a light source region and a non-light source region of an input image according to the light source threshold value, respectively optimizing the transmittance of the light source region and the non-light source region by adopting a self-adaptive light source matrix mechanism, and then fusing and solving the initial transmittance;

performing light source compensation on the initial transmittance to obtain and output the final transmittance of the input image;

and carrying out defogging calculation on the input image according to the ambient light estimated value and the final transmissivity and the atmospheric scattering model to obtain a defogged output image.

2. The method for monitoring the unmanned aerial vehicle under the low-illumination haze according to claim 1, wherein the process of processing the input image by adopting a mixed mode of edge window filtering and rapid edge protection filtering comprises the following steps:

acquiring pixels of an input image, adopting edge window filtering to treat each pixel as a potential edge, and generating a plurality of edge windows around each pixel;

processing the input image, including adjusting brightness of the input image, obtaining and outputting an edge window with minimum Euclidean distance between each pixel and the input image thereof so as to preserve edge information of the input image;

obtaining a filtered image according to the output edge window with the minimum distance, and processing the filtered image by adopting quick edge protection filtering to obtain a preprocessed image;

and calculating an ambient light estimated value in a dark channel priori mode according to the preprocessed image.

3. The method of claim 1, wherein the step of setting a light source threshold of an input image according to the ambient light estimate comprises:

calculating luminosity values of all pixel points in an input image;

and calculating the difference value between the luminosity value and the estimated value of the ambient light, and taking the absolute value of the largest difference value as the light source threshold value.

4. The method for monitoring the unmanned aerial vehicle under the low-illumination haze according to claim 1, wherein the process of dividing the light source region and the non-light source region of the input image according to the light source threshold value, respectively optimizing the transmittance of the light source region and the non-light source region by adopting an adaptive light source matrix mechanism, and then fusing and solving the initial transmittance comprises the following steps:

dividing any pixel point in the input image into a light source area or a non-light source area; when the luminosity value of the pixel point is larger than the light source threshold value, dividing the pixel point into a light source region, otherwise, dividing the pixel point into a non-light source region;

when the pixel points belong to a light source area, calculating and optimizing the transmissivity of the light source area by adopting a self-adaptive light source matrix mechanism; comprising the following steps:

numbering each pixel point in an input image according to a light source matrix mechanism in descending order of luminosity values, and calculating a first light source influence matrix, a first light source influence matrix and a pixel light source influence matrix of each pixel point; specifically, the size of the input image is defined as w.h, x.epsilon.1, w]，y∈[1，h]Numbering each pixel point in descending order of luminosity value as { m } ₀ ，m ₁ ，...，m _T-1 T is the total number of pixels; then the first time period of the first time period,

at the value of x, the first light source affects the matrix k _x The calculation is as follows:

at the value of y, the second light source affects the matrix k _y The calculation is as follows:

the pixel light source influence matrix K _x The calculation is as follows:

wherein C is _x 、C _y Representing luminosity values of the pixel points under corresponding x and y values, M is a light source threshold value, and d _x，m Representing the distance from other pixels to the selected pixel point;

obtaining an adjustment correction coefficient w for optimizing the transmissivity of the light source area according to the pixel light source influence matrix _x ，

Wherein α represents an adjustment correction factor, t, of the input image _x Representing the initial transmittance;

when the pixel points belong to a non-light source area, calculating the transmissivity of the non-light source area through a dark channel priori theory;

calculating initial transmittance t according to the optimized transmittance of the light source area and the transmittance of the non-light source area in a fusion way _M ，

Wherein the method comprises the steps ofOmega represents the area of the light source,indicating the transmittance of the non-light source region, w _x t _x∈Ω Indicating the transmittance of the light source region.

5. The method for monitoring the unmanned aerial vehicle under the low-illumination haze according to claim 1, wherein the process of performing light source compensation on the initial transmittance to obtain and output the final transmittance of the input image is as follows:

and adopting a gamma correction mode, and carrying out light source compensation on the initial transmittance by adjusting a compensation coefficient to obtain the final transmittance.

6. The method for monitoring a drone under low-light haze according to claim 1, wherein the process of obtaining the defogged output image comprises:

and according to the estimated value of the ambient light and the final transmissivity, defogging calculation is carried out on any pixel point in the input image according to an atmospheric scattering model, wherein the formula is as follows:

wherein I is _x An initial image representing any pixel point in an input image, t _F Is the final transmittance, J _x Representing the defogging processed image of the pixel point;

and synthesizing the defogging processed images of the pixel points to obtain defogged output images.

7. Monitoring unmanned aerial vehicle system based on low-light haze down, its characterized in that includes:

the acquisition processing module is used for acquiring pixels of the input image, preprocessing the input image by adopting a mixed mode of edge window filtering and rapid edge protection filtering, and acquiring an ambient light estimated value of the input image;

the setting module is used for setting a light source threshold value of an input image according to the ambient light estimated value;

the division solving module is used for dividing a light source region and a non-light source region of the input image according to the light source threshold value, respectively optimizing the transmittance of the light source region and the non-light source region by adopting a self-adaptive light source matrix mechanism, and then fusing and solving the initial transmittance;

the compensation module is used for carrying out light source compensation on the initial transmittance to obtain and output the final transmittance of the input image;

and the calculation module is used for carrying out defogging calculation on the input image according to the estimated value of the ambient light and the final transmissivity and the atmospheric scattering model to obtain a defogged output image.

8. The low-light haze-based unmanned aerial vehicle system of claim 7, wherein the execution unit that the acquisition processing module obtains an ambient light estimate of the input image comprises:

the acquisition unit is used for acquiring pixels of an input image, adopting edge window filtering to treat each pixel as a potential edge, and generating a plurality of edge windows around each pixel;

the adjusting unit is used for processing the input image and comprises adjusting the brightness of the input image, and obtaining and outputting an edge window with the minimum Euclidean distance between each pixel and the input image so as to keep the edge information of the input image;

the processing unit is used for obtaining a filtered image according to the output edge window with the minimum distance, and processing the filtered image by adopting quick edge protection filtering to obtain a preprocessed image;

and the calculating unit is used for calculating an ambient light estimated value in a dark channel prior mode according to the preprocessed image.

9. A computer device comprising at least one processor coupled to a memory storing a program or instructions that, when executed by the processor, implement the steps of the low-light haze-based drone method of any one of claims 1 to 6.

10. A readable storage medium having stored thereon a program or instructions which when executed by a processor performs the steps of the low-light haze-based drone method of any one of claims 1 to 6.