WO2020124355A1

WO2020124355A1 - Image processing method, image processing device, and unmanned aerial vehicle

Info

Publication number: WO2020124355A1
Application number: PCT/CN2018/121726
Authority: WO
Inventors: 李静; 袁一璟; 彭亮
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2018-12-18
Filing date: 2018-12-18
Publication date: 2020-06-25
Also published as: CN111344736A

Abstract

The present disclosure relates to an image processing method comprising: performing n-1 downsamplings on a raw image, so as to generate images of n respective resolutions; performing edge-preserving filtering on each of the images, so as to obtain high frequency information and filtering results; combining high frequency information of the ith image with high frequency information of the i-1th image, and obtaining a result of the i-1th high frequency fusion; and combining a filtering result of the ith image and a filtering result of the i-1th image according to the result of the i-1th high frequency fusion, and obtaining a result of the i-1th filtering fusion. According to the embodiments of the present disclosure, more details can be retained in the high frequency region of an image, while simultaneously achieving better noise reduction effects in the low frequency region of the image.

Description

Image processing method, image processing device and drone

Technical field

The present invention relates to the technical field of image processing, and in particular, to an image processing method, an image processing device, and a drone.

Background technique

In order to get an excellent display effect, the image needs to be denoised to remove the noise in the image.

However, based on the current method of image denoising, if a better denoising effect is to be achieved, more detailed information of the image will be lost, and if more detailed information of the image needs to be retained, the denoising effect will be weakened.

Summary of the invention

The invention provides an image processing method, an image processing device and an unmanned aerial vehicle to solve the technical problems in the related art.

According to a first aspect of the embodiments of the present disclosure, an image processing method is proposed, including:

The original image is downsampled n-1 times to determine n resolution images, where the resolution of the i-th image is less than the resolution of the i-1 image, 1<i≤n, the first image Is the original image;

Perform edge-preserving filtering for each image separately to obtain high-frequency information and filtering results;

Perform the following steps on the 2nd to nth images, respectively, until the first filter fusion result is obtained:

The high-frequency information of the i-th image and the high-frequency information of the i-1th image are fused to obtain the i-1 high-frequency fusion result, where, when i<n, the i-th high-frequency fusion result is taken as the High frequency information of i images;

According to the i-1 high-frequency fusion result, the filter result of the i-th image and the filter result of the i-1 image are fused to obtain the i-1 filter fusion result, where, when i<n, the The i-filter fusion result is used as the filter result of the i-th image.

According to a second aspect of the embodiments of the present disclosure, an image processing apparatus is proposed, including a processor, the processor is used for,

According to a third aspect of the embodiments of the present disclosure, a drone is proposed, including the image processing device described in the above embodiments.

According to an embodiment of the present disclosure, since the i-th image is obtained by downsampling based on the i-1th image, the resolution of the i-th image after downsampling is smaller than the resolution of the i-1th image, which may result in loss Part of the high-frequency information, so it has a better noise reduction effect, then the filtering result of the i-th image is less than the filtering result of the i-1 image, and the pixels with larger high-frequency fusion results correspond to The pixels in the filtered image are more likely to be edges, so more high-frequency information can be retained so that the edges can be correctly extracted.

Based on this embodiment, the filter result of the i-th image and the filter result of the i-1 image can be fused according to the i-1 high-frequency fusion result, for example, by setting the i-1 high-frequency fusion result and The weight value of the filtering result of the i-th image is inversely correlated and positively correlated with the weight value of the filtering result of the i-1th image, so that for the i-1 filtering fusion result, more details are retained in the high-frequency region , The denoising effect is higher in the low frequency region.

Furthermore, when i<n, the ith filter fusion result is used as the filtering result of the i-th image, and the second two steps in the above embodiment can be continuously performed on the second to n-1th images, so that according to the i-1 Filter fusion result to get the i-2 filter fusion result, according to the i-2 filter fusion result to get the i-3 filter fusion result, ..., until the first filter fusion result is obtained, then the first filter fusion result is relative to the original image In other words, at the same time, more details are retained in the high-frequency region, and the denoising effect is higher in the low-frequency region.

BRIEF DESCRIPTION

In order to more clearly explain the technical solutions in the embodiments of the present invention, the drawings required in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without paying any creative labor, other drawings can also be obtained based on these drawings.

FIG. 1 is a schematic flowchart of an image processing method according to an embodiment of the present disclosure.

FIG. 2 is a schematic flow chart of fusing the high-frequency information of the i-th image and the high-frequency information of the i-1th image according to an embodiment of the present disclosure to obtain the result of the i-1 high-frequency fusion.

FIG. 3 is a schematic flowchart of fusing the filter result of the i-th image and the filter result of the i-1 image according to the i-1 high-frequency fusion result to obtain the i-1 filter fusion result.

4 is a schematic flowchart of determining an i-1 second weight corresponding to an i-1 high-frequency fusion result.

FIG. 5 is a schematic flowchart of another image processing method according to an embodiment of the present disclosure.

detailed description

The technical solutions in the embodiments of the present invention will be described clearly and completely in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present invention. In addition, in the case of no conflict, the following embodiments and the features in the embodiments can be combined with each other.

FIG. 1 is a schematic flowchart of an image processing method according to an embodiment of the present disclosure. The image processing method shown in this embodiment can be applied to terminals, such as mobile phones, tablet computers, wearable devices, etc., can also be applied to servers, and can also be applied to other devices with data processing functions, such as drones.

As shown in FIG. 1, the image processing method may include the following steps:

In step S1, the original image is down-sampled n-1 times to determine n resolution images, where the resolution of the i-th image is less than the resolution of the i-1 image, 1<i≤n, the first One image is the original image.

In one embodiment, by downsampling the original image, an image with a lower resolution can be obtained. It should be noted that after each downsampling, the ratio of the resolution reduction of the image may be the same as before the downsampling Can also be different. The following is an exemplary description mainly in the case where the reduction ratio of the image resolution is the same.

For example, n=4, then the original image is down-sampled three times. After each down-sampling, the ratio of the resolution of the image is reduced by 1/2 relative to that before the down-sampling, that is, the resolution of the image after down-sampling, It is 1/2 the resolution of the image before downsampling. Then the resolution of the second image d2 after the first downsampling is 1/2 of the resolution of the original image d1, and the resolution of the third image d3 after the second downsampling is 1 of the resolution of d2 /2, the resolution of the fourth image d4 after the third downsampling is 1/2 of the resolution of d3.

In the following, the embodiment of the present disclosure will be exemplarily described in the case where n=4, and after each downsampling, the ratio of the resolution reduction of the image is 1/2 compared with that before the downsampling.

Step S2: Perform edge-preserving filtering for each image to obtain high-frequency information and filtering results.

In one embodiment, for each image, for example, d1, d2, d3, and d4, edge-preserving filtering can be performed separately, where edge-preserving filtering refers to a filtering method that can effectively retain edge information in the image during the filtering process , For example, bilateral filtering, guided filtering, weighted least squares filtering, etc. Taking bilateral filtering as an example, the minimum convolution kernel used for filtering is 3×3, for example, a 7×7 convolution kernel can be used.

After filtering the image through edge-preserving filtering, the high-frequency information and filtering result of the image can be obtained.

Among them, the high-frequency information of an area can express whether the area changes drastically, specifically whether there is more texture in the area, for example, the higher the high-frequency information corresponding to a pixel, the more likely that pixel belongs to the object in the image the edge of.

The filtering result is the result of removing some noise and details from the image before filtering. In theory, the signal-to-noise ratio of the filtering result is higher than that of the image before filtering. The filtering result of the edge-preserving filtering on the down-sampled image can reflect the low-frequency information of the image before down-sampling. The low-frequency information of a certain area can express whether the area changes smoothly, specifically whether there are large patches of color in the area.

For convenience of description, it is noted that the high-frequency information of the fourth image d4 is d4_diff and the filtering result is d4_filter; the high-frequency information of the third image d3 is d3_diff and the filtering result is d3_filter; the high-frequency information of the second image d2 is d2_diff, the filtering result is d2_filter; the high-frequency information of the first image d1 is d1_diff, and the filtering result is d1_filter.

Step S3, fusing the high-frequency information of the i-th image and the high-frequency information of the i-1-th image to obtain the i-1 high-frequency fusion result, where, when i<n, the i-th high-frequency fusion The result is the high frequency information of the i-th image;

In step S4, according to the i-1 high-frequency fusion result, the filter result of the i-th image and the filter result of the i-1 image are fused to obtain the i-1 filter fusion result. When i<n, The ith filter fusion result is used as the ith image filter result.

In one embodiment, starting from the nth image, the high-frequency information of the i-th image and the high-frequency information of the i-1th image are fused to obtain the i-1 high-frequency fusion result. For example, n=4, starting from the fourth image d4, the high-frequency information d4_diff of the fourth image and the high-frequency information d3_diff of the third image d3 are fused to obtain the third high-frequency fusion result d3_diff_merge.

Furthermore, when i<n, the i-th high-frequency fusion result is used as the high-frequency information of the i-th image. For example, when i=3, the third high-frequency fusion result d3_diff_merge is used as the high-frequency information of the third image d3_diff, i = 2, the second high-frequency fusion result d2_diff_merge is used as the high-frequency information d2_diff of the second image.

After obtaining the high-frequency information d2_diff of the second image, the high-frequency information d2_diff of the second image and the high-frequency information d1_diff of the first image can be further fused to obtain the first high-frequency fusion result d1_diff_merge.

The down-sampling of the image will reduce the image noise, so the first high-frequency fusion result obtained by successive fusion according to the high-frequency information of the images of different resolutions, relative to the original image, that is, the high-frequency information of the first image, can While retaining high-frequency information, reduce noise interference and improve the accuracy of edge detection.

Further, according to the i-1 high-frequency fusion result, the filter result of the i-th image and the filter result of the i-1 image are fused to obtain the i-1 filter fusion result. For example, fusion refers to weighted sum, then You can set the i-1 high-frequency fusion result to be inversely correlated with the weight of the filtering result of the i-th image, and positively correlated with the weight of the i-th image filtering result, so that the pixels with larger high-frequency fusion results The weight value of the filter result of the corresponding i-th image is small, and the weight value of the filter result of the i-th image is large.

Since the i-th image is based on the down-sampling of the i-1th image, the resolution of the i-th image after downsampling is less than the resolution of the i-1th image, which will cause some high-frequency information to be lost, so it has Better noise reduction effect, then the filtering result of the i-th image has less high-frequency information than the filtering result of the i-1th image, and the pixels with larger high-frequency fusion results correspond to the pixels in the filtered image are edges Is more likely, so more high-frequency information can be retained so that the edges can be correctly extracted.

Based on this embodiment, the filter result of the i-th image and the filter result of the i-1 image can be fused according to the i-1 high-frequency fusion result, for example, by setting the i-1 high-frequency fusion result and The weight of the filtering result of the i-th image is inversely correlated and positively correlated with the weight of the filtering result of the i-th image. When the i-1 high-frequency fusion result is large, there is a high possibility that the pixel is high-frequency information, corresponding to the filtering result, so that the weight of the filtering result of the i-1th image is larger, and the i The weight of the filtering result of each image is small; when the i-1 high-frequency fusion result is small, there is a high possibility that the pixel is low-frequency information, corresponding to the filtering result, so that the filtering result of the i-1 image The weight of the image is smaller, and the weight of the filtering result of the i-th image is larger; thus for the i-1 filter fusion result, more details are retained in the high-frequency region, and the denoising effect is higher in the low-frequency region. .

Further, when i<n, the i-th high-frequency fusion result is used as the high-frequency information of the i-th image, and the i-th filter fusion result is used as the filter result of the i-th image, so that when the second to n-1 In the process of performing the above steps S3 and S4, the high frequency information of the i-1th image and the high frequency information of the i-2th image can be fused to obtain the i-2 high frequency fusion result. -The high-frequency information of the 2 images and the high-frequency information of the i-3 image are fused to obtain the i-3 high-frequency fusion result, ..., and finally the first high-frequency fusion result is obtained; in addition, the i The filtering result of each image and the filtering result of the i-1th image are fused to obtain the filtering result of the i-1th filter, and the filtering result of the i-1th image and the filtering result of the i-2th image are fused, Obtain the i-2 filter fusion result, ..., and finally get the first filter fusion result. Compared with the original image, the first filter fusion result realizes the preservation of more details in the high frequency area and the denoising in the low frequency area. The effect is higher.

Based on the embodiment shown in FIG. 1, FIG. 2 shows a method for fusing the high-frequency information of the i-th image and the high-frequency information of the i-1th image according to an embodiment of the present disclosure to obtain the i -1 Schematic flow chart of high-frequency fusion results. As shown in FIG. 2, the high-frequency information of the i-th image and the high-frequency information of the i-1th image are fused, and the result of the i-1 high-frequency fusion includes:

Step S31, up-sampling the high-frequency information of the i-th image to obtain the i-th high-frequency up-sampling result;

In step S32, the i-th high-frequency up-sampling result and the high-frequency information of the i-1th image are weighted and summed according to the i-1 first weight to obtain the i-1 high-frequency fusion result.

In one embodiment, since the resolution of the i-th image is smaller than the resolution of the i-1th image, the resolution of the high-frequency information of the i-th image is smaller than the resolution of the high-frequency information of the i-1th image In order to weight and sum the high-frequency information of the i-th image and the high-frequency information of the i-1th image, it is necessary to increase the resolution of the high-frequency information of the i-th image. The high-frequency information of the image is up-sampled to obtain the i-th high-frequency up-sampling result, and the resolution of the i-th high-frequency up-sampling result is the same as the resolution of the high-frequency information of the i-1th image.

For example, for the high-frequency information d4_diff of the fourth image, up-sampling is performed to obtain the fourth high-frequency up-sampling result d4_up_diff. The resolution of d4_up_diff is the same as the resolution of the high-frequency information d3_diff of the third image.

Further, according to the i-1 first weight value, the i-th high-frequency up-sampling result and the i-th image high-frequency information are weighted and summed to obtain the i-1 high-frequency fusion result. The i-1 first weight can be set as needed. For example, when i=4, the third first weight is w3, and the third high-frequency fusion result d3_diff_merge is calculated as follows:

d3_diff_merge=d3_diff×w3+d4_up_diff×(1-w3);

Since the i-th image is obtained based on the down-sampling of the i-1 image, the resolution of the i-th image after down-sampling is less than the resolution of the i-1 image, then the high frequency of the i-1 image Compared with the high-frequency information of the i-th image, the information can express more high-frequency information. In this case, by setting the i-1 first weight, the denoising effect and details in the fusion process can be retained Adjust the degree. Generally, the first weight value is a preset value, and the first weight value w(i-1) can be determined according to the signal-to-noise ratio of the original image. If the original image has a high signal-to-noise ratio, then w(i-1) You can take a larger value. In general, you can set w(n-1), w(n-2)...w0 decreases sequentially.

For example, if the first weight of i-1 is set larger, then the high-frequency information of the i-1th image can be fused more, so that more high-frequency information of higher frequencies can be retained, that is, fusion The degree of detail retention is high in the process; accordingly, if the i-1 first weight is set smaller, the i-th high-frequency upsampling result can be fused more, resulting in less The high-frequency information is retained, so the denoising effect is relatively good.

It should be noted that for an i-1 high-frequency fusion result, it can be calculated by an i-1 first weight, for example, in d3_diff_merge=d3_diff×w3+d4_up_diff×(1-w3), for d3_diff and For each pixel in each d4_up_diff, w3 is a fixed value. However, it is also possible to set the i-1th first weight value to be variable as needed, wherein different i-1th first weight values can be set for pixels in different positions, for example, in d3_diff_merge=d3_diff×w3+d4_up_diff× In (1-w3), for each pixel in d3_diff and each d4_up_diff, w3 may be different based on the difference in pixel position.

Based on the embodiment shown in FIG. 1 or FIG. 2, FIG. 3 is based on the i-1 high-frequency fusion result, the filter result of the i-th image and the filter result of the i-1 image are fused to obtain the first The schematic flow chart of i-1 filter fusion result. As shown in FIG. 3, according to the i-1 high-frequency fusion result, the filter result of the i-th image and the filter result of the i-1 image are fused, and the i-1 filter fusion result includes:

Step S41, up-sampling the filtering result of the i-th image to obtain the i-th filtering up-sampling result;

Step S42: Determine the i-1 second weight value corresponding to the i-1 high frequency fusion result, where the i-1 high frequency fusion result is positively correlated with the i-1 second weight value;

Step S43: Weight the i-th filter up-sampling result according to the i-1 second weight, weight the filter result of the i-1th image according to the difference between 1 and the i-1 second weight, and sum according to the weight As a result, the i-1th filter fusion result is obtained.

In one embodiment, since the resolution of the i-th image is smaller than the resolution of the i-1th image, the resolution of the filtering result of the i-th image is smaller than the resolution of the filtering result of the i-1th image, In order to perform weighted summation of the filtering result of the i-th image and the filtering result of the i-1th image, it is necessary to increase the resolution of the filtering result of the i-th image, where the filtering result of the i-th image can be performed Up-sampling to obtain the i-th filter up-sampling result, the resolution of the i-th filter up-sampling result is the same as the resolution of the i-1 image filtering result. For example, for the filter result d4_filter of the fourth image, up-sampling is performed to obtain the fourth high-frequency up-sampling result d4_up_filter. The resolution of d4_up_filter is the same as the resolution of the filter result d4_filter of the third image.

Further, the i-1 second weight value corresponding to the i-1 high frequency fusion result can be determined, and by setting the weight value of the i-1 high frequency fusion result inversely correlated with the filtering result of the i th image, and The weight of the filtering result of the i-1th image is positively correlated, so that the weight of the filtering result of the ith image corresponding to the pixel with a higher high-frequency fusion result is smaller, and the filtering result of the i-1th image corresponds to Has a larger weight.

For example, in the case of n=4, the third second weight d3_weight may be determined according to the third high-frequency fusion result d3_diff_merge, where the determined d3_weight is different for pixels with different d3_diff_merge, then for the third image d3 Pixel (i,j), the corresponding fourth filter upsampling result d4_up_filter(i,j), the corresponding third high-frequency fusion result is d3_diff_merge(i,j), then the determined third second weight Is d3_weight(i,j), the i-1th high-frequency fusion result of pixel (i,j) d3_filter_new(i,j) is equal to:

d3_filter(i,j)*d3_weight(i,j)+d4_up_filter(i,j)*(1-d3_weight(i,j))

Based on the example in FIG. 1, since the fourth image is obtained based on the down-sampling of the third image, the resolution of the down-sampled fourth image is less than that of the third image, which may cause some high-frequency loss. Information, so it has better noise reduction effect, then the filter result d4_filter of the fourth image is less high-frequency information than the filter result d3_filter of the third image, and the pixels with larger high-frequency fusion result correspond to the filtered image Of pixels are more likely to be edges, so more high-frequency information can be retained so that the edges are correctly extracted.

Based on this embodiment, the filtering result of the fourth image and the filtering result of the third image can be weighted and summed according to the third high-frequency fusion result, for example, by setting the third high-frequency fusion result and the fourth image The weights of the filtering results are inversely correlated and positively correlated with the weights of the filtering results of the third image, so that for the third filtering fusion result, more details are retained in the high-frequency region and the denoising effect in the low-frequency region Higher.

Further, when i<4, the i-th high-frequency fusion result is used as the high-frequency information of the i-th image, and the i-th filter fusion result is used as the filter result of the i-th image, so that the second to third images are executed During the above steps S3 and S4, the high-frequency information of the third image and the high-frequency information of the second image can be weighted and summed to obtain the second high-frequency fusion result, and the second second weight can be determined accordingly Value d2_weight, and then weighted and sum the high-frequency information of the second image and the high-frequency information of the first image to obtain the first high-frequency fusion result, and determine the first second weight d1_weight based on this; in addition, The filtering result of the third image and the filtering result of the second image can be weighted and summed based on the second second weight d2_weight to obtain the second filter fusion result, and then based on the first second weight d1_weight, the The filtering results of the two images and the filtering result of the first image are weighted and summed, and finally the first filtering fusion result is obtained. Compared with the original image, the first filtering fusion result realizes the preservation of details in the high-frequency region. Many, the denoising effect is higher in the low frequency region.

Optionally, the i-1 high-frequency fusion result is inversely related to the weight of the filtering result of the i-th image, and positively related to the weight of the filtering result of the i-1 image.

It should be noted that the positive correlation in the embodiment of the present disclosure means that when A and B are positively correlated, A increases in the overall trend with the increase of B, and in a local interval, A can follow the B Increases and remains unchanged; the positive correlation in the embodiments of the present disclosure means that when A and B are inversely correlated, A increases or decreases with the increase of B in the overall trend, while in a local interval, A can increase with The increase in B remains unchanged.

Based on the embodiment shown in FIG. 3, FIG. 4 is a schematic flowchart of determining an i-1 second weight value corresponding to an i-1 high frequency fusion result. As shown in FIG. 4, the determining the i-1 second weight corresponding to the i-1 high frequency fusion result includes:

Step S421: Determine the i-1 second weight corresponding to the i-1 high-frequency fusion result according to the association table between the i-1 second weight and the i-1 high-frequency fusion result.

In an embodiment, the i-1 second weight value and the i-1 high-frequency fusion result may be stored in an association relationship table. In the association relationship table, different i-1 high-frequency fusion results may correspond to different I-1 second weight value, so that the i-1 second weight value corresponding to the i-1 high frequency fusion result can be subsequently queried according to the association relationship table.

Optionally, the manner of downsampling includes at least one of the following:

Gaussian downsampling, mean downsampling, maximum or minimum downsampling, and median downsampling.

In one embodiment, the adopted downsampling method may be selected according to needs, and the downsampling operation method may be the same or different each time.

Optionally, the manner of upsampling includes at least one of the following:

The nearest neighbor element method, bilinear interpolation method, cubic interpolation method.

In one embodiment, the adopted upsampling method may be selected according to needs, and the upsampling operation method may be the same or different each time.

Optionally, the edge-preserving filtering method includes at least one of the following:

Bilateral filtering, guided filtering, weighted least squares filtering.

In one embodiment, the edge-preserving filtering method used may be selected according to needs.

Based on any of the above embodiments, FIG. 5 is a schematic flowchart of another image processing method according to an embodiment of the present disclosure. As shown in FIG. 5, before the original image is downsampled n-1 times to determine n resolution images, the method further includes:

Step S5: Determine the value of n according to the resolution of the original image.

In one embodiment, the value of n can be determined based on the resolution of the original image, and then the number of downsampling times n-1 is determined. For example, the higher the resolution of the original image, the larger n can be, that is, the number of downsampling times n -1 can be larger, which can ensure that the low-frequency information of sufficiently low frequency is obtained, so as to ensure that the fused result has a good denoising effect.

Optionally, the high-frequency information includes a mean value of a sum of absolute values of pixel value differences between each pixel in the corresponding image and pixels in respective neighborhoods.

In one embodiment, for each pixel in the image, the average value of the sum of the absolute values of the difference between the pixel value of the pixel and the pixel in the neighborhood can be calculated, and the average value can express the difference between the pixel and the pixel value in the neighborhood If the average value is large, it means that the difference between the pixel value and the pixel value in the neighborhood is large, then the pixel is more likely to belong to the edge of the object in the image, so the average value can be used as the high-frequency information of the pixel.

Corresponding to the embodiments of the image processing method described above, the present disclosure also proposes embodiments of the image processing device.

An embodiment of the present disclosure proposes an image processing apparatus including a processor, the processor is used for,

In one embodiment, the processor is used to,

Up-sampling the high-frequency information of the i-th image to obtain the i-th high-frequency up-sampling result;

According to the i-1 first weight, the i-th high-frequency up-sampling result and the i-th image high-frequency information are weighted and summed to obtain the i-1 high-frequency fusion result.

In one embodiment, the processor is used to,

Up-sampling the filtering result of the i-th image to obtain the i-th filtering up-sampling result;

Determine the i-1 second weight corresponding to the i-1 high frequency fusion result, where the i-1 high frequency fusion result is positively correlated with the i-1 second weight;

The i-th filter up-sampling result is weighted according to the i-1 second weight, the i-1 image filtering result is weighted according to the difference between 1 and the i-1 second weight, and the first i-1 filter fusion results.

In one embodiment, the i-1 high-frequency fusion result is inversely correlated with the weight of the filter result of the i-th image, and positively correlated with the weight of the filter result of the i-1 image.

In one embodiment, the processor is used to,

The i-1 second weight value corresponding to the i-1 high frequency fusion result is determined according to the correlation table between the i-1 second weight value and the i-1 high frequency fusion result.

In one embodiment, the manner of downsampling includes at least one of the following:

In one embodiment, the manner of upsampling includes at least one of the following:

In one embodiment, the edge-preserving filtering method includes at least one of the following:

Bilateral filtering, guided filtering, weighted least squares filtering.

In one embodiment, the processor is further configured to determine the value of n according to the resolution of the original image.

In one embodiment, the high-frequency information includes an average value of the sum of the absolute values of the difference values of the pixel values of each pixel in the corresponding image and the pixels in the respective neighborhood.

An embodiment of the present disclosure proposes a drone, including the image processing device described in any of the above embodiments.

The system, device, module or unit explained in the above embodiments may be specifically implemented by a computer chip or entity, or implemented by a product having a certain function. For the convenience of description, when describing the above device, the functions are divided into various units and described separately. Of course, when implementing this application, the functions of each unit may be implemented in one or more software and/or hardware. Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

The embodiments in this specification are described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method embodiment.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations There is any such actual relationship or order. The terms "include", "include", or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a series of elements includes not only those elements, but also others that are not explicitly listed Elements, or also include elements inherent to such processes, methods, objects, or equipment. Without more restrictions, the element defined by the sentence "include one..." does not exclude that there are other identical elements in the process, method, article or equipment that includes the element.

The above is only an embodiment of the present application, and is not intended to limit the present application. For those skilled in the art, this application may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in the scope of the claims of this application.

Claims

An image processing method, which includes:

The original image is downsampled n-1 times to determine n resolution images, where the resolution of the i-th image is less than the resolution of the i-1 image, 1<i≤n, the first image Is the original image;

Perform edge-preserving filtering for each image separately to obtain high-frequency information and filtering results;

Perform the following steps on the 2nd to nth images, respectively, until the first filter fusion result is obtained:

The high-frequency information of the i-th image and the high-frequency information of the i-1th image are fused to obtain the i-1 high-frequency fusion result, where, when i<n, the i-th high-frequency fusion result is taken as the High frequency information of i images;

According to the i-1 high-frequency fusion result, the filter result of the i-th image and the filter result of the i-1 image are fused to obtain the i-1 filter fusion result, where, when i<n, the The i-filter fusion result is used as the filter result of the i-th image.
The method according to claim 1, wherein the fusing the high-frequency information of the i-th image and the high-frequency information of the i-1th image to obtain the i-1 high-frequency fusion result includes:

Up-sampling the high-frequency information of the i-th image to obtain the i-th high-frequency up-sampling result;

According to the i-1 first weight, the i-th high-frequency up-sampling result and the i-th image high-frequency information are weighted and summed to obtain the i-1 high-frequency fusion result.
The method according to claim 1, wherein the filtering result of the i-th image and the filtering result of the i-1th image are fused according to the i-1 high-frequency fusion result to obtain the i-th 1 Filter fusion results include:

Up-sampling the filtering result of the i-th image to obtain the i-th filtering up-sampling result;

Determine the ith second weight corresponding to the ith-1 high-frequency fusion result, where the ith-1 high-frequency fusion result is positively correlated with the ith-1 second weight;

The i-th filter up-sampling result is weighted according to the i-1 second weight, the i-1 image filtering result is weighted according to the difference between 1 and the i-1 second weight, and the first i-1 filter fusion results.
The method according to claim 3, wherein the i-1 high-frequency fusion result is inversely correlated with the weight of the filtering result of the i-th image, and positively correlated with the weight of the filtering result of the i-1th image .
The method according to claim 3, wherein the determining the i-1 second weight corresponding to the i-1 high frequency fusion result includes:

The i-1 second weight value corresponding to the i-1 high frequency fusion result is determined according to the correlation table between the i-1 second weight value and the i-1 high frequency fusion result.
The method according to any one of claims 1 to 5, wherein the manner of downsampling includes at least one of the following:

Gaussian downsampling, mean downsampling, maximum or minimum downsampling, and median downsampling.
The method according to any one of claims 1 to 5, wherein the manner of upsampling includes at least one of the following:

The nearest neighbor element method, bilinear interpolation method, cubic interpolation method.
The method according to any one of claims 1 to 5, wherein the edge-preserving filtering method includes at least one of the following:

Bilateral filtering, guided filtering, weighted least squares filtering.
The method according to any one of claims 1 to 5, wherein before down-sampling the original image n-1 times to determine n resolution images, the method further comprises:

According to the resolution of the original image, the value of n is determined.
The method according to any one of claims 1 to 5, wherein the high-frequency information includes an average value of a sum of absolute values of differences in pixel values of each pixel in the corresponding image and pixels in respective neighborhoods.
An image processing device, characterized by comprising a processor for down-sampling the original image n-1 times to determine n resolution images, wherein the resolution of the i-th image is less than The resolution of the i-1th image, 1<i≤n, the first image is the original image;

Perform edge-preserving filtering for each image separately to obtain high-frequency information and filtering results;

Perform the following steps on the 2nd to nth images, respectively, until the first filter fusion result is obtained:

The high-frequency information of the i-th image and the high-frequency information of the i-1th image are fused to obtain the i-1 high-frequency fusion result, where, when i<n, the i-th high-frequency fusion result is taken as the High frequency information of i images;

According to the i-1 high-frequency fusion result, the filter result of the i-th image and the filter result of the i-1 image are fused to obtain the i-1 filter fusion result, where, when i<n, the The i-filter fusion result is used as the filter result of the i-th image.
The apparatus according to claim 10, wherein the processor is used to:

Up-sampling the high-frequency information of the i-th image to obtain the i-th high-frequency up-sampling result;

According to the i-1 first weight, the i-th high-frequency up-sampling result and the i-th image high-frequency information are weighted and summed to obtain the i-1 high-frequency fusion result.
The apparatus according to claim 10, wherein the processor is used to:

Up-sampling the filtering result of the i-th image to obtain the i-th filtering up-sampling result;

Determine the i-1 second weight corresponding to the i-1 high frequency fusion result, where the i-1 high frequency fusion result is positively correlated with the i-1 second weight;

The i-th filter up-sampling result is weighted according to the i-1 second weight, the i-1 image filtering result is weighted according to the difference between 1 and the i-1 second weight, and the first i-1 filter fusion results.
The device according to claim 13, wherein the i-1 high-frequency fusion result is inversely related to the weight of the filtering result of the i-th image, and is positively related to the weight of the filtering result of the i-1 image .
The apparatus according to claim 14, wherein the processor is used for,

The i-1 second weight value corresponding to the i-1 high frequency fusion result is determined according to the correlation table between the i-1 second weight value and the i-1 high frequency fusion result.
The device according to any one of claims 11 to 15, wherein the manner of downsampling includes at least one of the following:

Gaussian downsampling, mean downsampling, maximum or minimum downsampling, and median downsampling.
The device according to any one of claims 11 to 15, wherein the manner of upsampling includes at least one of the following:

The nearest neighbor element method, bilinear interpolation method, cubic interpolation method.
The device according to any one of claims 11 to 15, wherein the manner of edge-preserving filtering includes at least one of the following:

Bilateral filtering, guided filtering, weighted least squares filtering.
The apparatus according to any one of claims 11 to 15, wherein the processor is further configured to determine the value of n according to the resolution of the original image.
The device according to any one of claims 11 to 15, wherein the high-frequency information includes an average value of a sum of absolute values of pixel value differences between each pixel in the corresponding image and pixels in respective neighborhoods.
An unmanned aerial vehicle, characterized by comprising the image processing device according to any one of claims 11 to 20.