WO2021217642A1

WO2021217642A1 - Infrared image processing method and apparatus, and movable platform

Info

Publication number: WO2021217642A1
Application number: PCT/CN2020/088469
Authority: WO
Inventors: 张青涛; 庹伟; 陈星�
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-04-30
Filing date: 2020-04-30
Publication date: 2021-11-04

Abstract

An infrared image processing method and apparatus, and a movable platform. The method comprises: extracting low-frequency components of an infrared image to be processed and corresponding high-frequency components of said infrared image in one or more extraction directions; determining grayscale gains of respective corresponding high-frequency components in the extraction directions, and performing enhancement processing on the high-frequency components according to the grayscale gains; performing contrast stretching processing on the low-frequency components; and fusing the low-frequency components subjected to the stretching processing and the high-frequency components subjected to the enhancement processing to obtain a processed infrared image. High-frequency components in different directions are extracted, and then grayscale gains of the high-frequency components in all the extraction directions are respectively determined, thereby making the enhancement of the high-frequency components more flexible and controllable, avoiding making the noise in the tangent direction of an edge more obvious in an infrared image enhancement process, and improving the infrared image processing effect.

Description

Infrared image processing method, device and movable platform

Technical field

This application relates to the field of image processing technology, and in particular, to an infrared image processing method, device, and movable platform.

Background technique

Infrared images collected by infrared sensors usually have relatively low resolution, narrow grayscale distribution, and contain a lot of noise, which cannot well reflect the details and contours of the imaged object. In order to better display the details of the imaged object and facilitate subsequent applications and sightings, the infrared image can be enhanced to make the contrast of the infrared image higher and the details and outline of the object clearer. However, when the infrared image is enhanced, it is easy to make the noise of the enhanced infrared image more obvious, for example, it is easy to introduce noise such as isolated points, or make the noise at the edge of the object more obvious, which affects the effect of the infrared image.

Summary of the invention

In view of this, this application provides an infrared image processing method, device and movable platform.

According to the first aspect of the present application, there is provided an infrared image processing method, the method including:

Extracting low-frequency components of the infrared image to be processed and corresponding high-frequency components of the infrared image to be processed in one or more extraction directions;

Determining the gray-scale gain of the high-frequency component corresponding to each of the extraction directions, and performing enhancement processing on the high-frequency component according to the gray-scale gain;

Performing contrast stretching processing on the low-frequency components;

The low-frequency components after the stretching process and the high-frequency components after the enhancement process are combined to obtain a processed infrared image.

According to a second aspect of the present application, there is provided an infrared image processing device, the device including a processor, a memory, and a computer program stored in the memory that can be executed by the processor, and the processor executes the computer The following steps are implemented during the program:

Performing contrast stretching processing on the low-frequency components;

According to a third aspect of the present application, a movable platform is provided, and the movable platform includes the infrared image processing device described in the second aspect.

According to a fourth aspect of the present application, there is provided a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the above-mentioned first aspect is implemented. The infrared image processing method.

Applying the solution provided by this application, when the infrared image is enhanced, the low-frequency components of the infrared image and the high-frequency components in one or more extraction directions can be extracted. For the low-frequency components, the stretching and enhancement processing can be carried out, and for each extraction For the high-frequency components corresponding to the directions, the corresponding gray-scale gains can be determined respectively. The high-frequency components are enhanced by the determined gray-scale gains, and then the low-frequency components after stretching and the enhanced high-frequency components are merged to obtain enhancement processing After the infrared image. By extracting the high-frequency components in different directions, and then separately determining the gray gains of the high-frequency components in each extraction direction, the enhancement of the high-frequency components can be made more flexible and controllable, avoiding the tangent direction of the object edge during the infrared image enhancement process The noise is more obvious, thereby enhancing the effect of infrared image processing.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings that need to be used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained from these drawings without creative labor.

FIG. 1 is a schematic diagram of the normal direction and the tangent direction of the edge of an object according to an embodiment of the present application.

Fig. 2 is a flowchart of an infrared image processing method according to an embodiment of the present application.

Fig. 3 is a schematic diagram of an infrared image processing method according to an embodiment of the present application.

FIG. 4 is a schematic diagram of the extraction direction of high-frequency components according to an embodiment of the present application.

Fig. 5 is a schematic diagram of determining the extraction direction of a pixel according to an embodiment of the present application.

Fig. 6 is a schematic diagram of an infrared image processing method according to an embodiment of the present application.

Fig. 7 is a schematic diagram of the logical structure of an infrared image processing device according to an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

Generally speaking, the infrared image collected by the infrared sensor has a low resolution, a narrow grayscale distribution and a large amount of noise, which cannot well reflect the details and contours of the imaged object. Therefore, it is necessary to perform enhancement processing on the collected original infrared image, improve the contrast of the infrared image, and enhance the gray scale of the edges and details of the object in the infrared image, so that the details and outline of the object are clearer. However, when the infrared image is enhanced, various noises are easily enhanced, making the noise of the enhanced infrared image more obvious, and in the process of enhancement processing, it is easy to introduce noise such as isolated points, or make The noise in the tangential direction of the edge of the object is enhanced, which affects the effect of the infrared image. For example, as shown in Figure 1, assuming that the black line in the image is the edge of the object, when the edge is enhanced, the tangent direction of the edge (the dotted line in the figure is the normal direction of the line, and the tangent direction is the direction of the line Coincidence) is easy to introduce noise, making the noise in the tangent direction of the edge too obvious.

Based on this, the present application provides an infrared image processing method, and the infrared image processing method will be introduced below in conjunction with FIG. 2 and FIG. 3. As shown in FIG. 2, it is a flowchart of the infrared image processing method, and the method includes the following steps:

S202: Extract low-frequency components of the infrared image to be processed and high-frequency components corresponding to the infrared image to be processed in one or more extraction directions;

S204: Determine the gray-scale gain of the high-frequency component corresponding to each of the extraction directions, and perform enhancement processing on the high-frequency component according to the gray-scale gain;

S206: Perform contrast stretching processing on the low-frequency component;

S208: Fusion the low frequency components after the stretching process and the high frequency components after the enhancement process to obtain a processed infrared image.

Fig. 3 is a schematic diagram of the infrared image processing method, in which multiple extraction directions are taken as an example. Since noise, edges or details of objects are parts of the image where the gray level changes more drastically, they are mostly concentrated in the high-frequency components of the infrared image, while the background area and flat area in the infrared image are the parts where the gray level changes relatively smoothly. Therefore, it is mostly concentrated on the low-frequency components of the infrared image. Therefore, the low-frequency component and high-frequency component of the infrared image to be processed can be extracted, and processed according to the characteristics of the low-frequency component and the high-frequency component respectively. Among them, the extraction of high-frequency components and low-frequency components in infrared images can use general image high-low-frequency component extraction methods, for example, the infrared image can be Fourier transformed to obtain the spectrogram of the infrared image, and the high frequency can be determined based on the spectrogram. Frequency components and low frequency components. Of course, in some embodiments, a high-pass filter can also be used to extract the high-frequency part, and a low-pass filter can be used to extract the low-frequency part. The filter can be a general-purpose filter or can be designed by itself.

As for the low-frequency components, since they are the parts that change smoothly in the image, the extraction direction can be ignored and the omnidirectional extraction can be directly performed when extracting. Of course, in some embodiments, in order to perform more precise and detailed processing on the infrared image, when extracting low-frequency components, it is also possible to extract from multiple extraction directions. After extracting the low-frequency components, the low-frequency components can be subjected to contrast stretching processing to enhance the contrast of the low-frequency components and make the infrared image more vivid. Among them, the low-frequency component can be improved by adaptive contrast stretching or histogram equalization to achieve the stretching and enhancement of the low-frequency component.

For high-frequency components, the high-frequency components mainly correspond to the parts that change drastically in the image, mainly noise, the edges of objects, and so on. In the process of sharpening and enhancing the edge of the object, it is easy to enhance the noise in the tangent direction of the object edge, or generate noise such as isolated points. Therefore, when extracting the high-frequency components of the infrared image, you can extract the corresponding high in one or more extraction directions. Frequency components, and then determine the respective gray gains for the high-frequency components of each extraction direction. Figure 3 shows an example of multiple extraction directions. Different directions can be extracted, such as direction 1, direction 2, and direction 3. High-frequency components, and then determine the gray gain 1 of the high-frequency components corresponding to direction 1, the gray gain 2 of the high-frequency components corresponding to the direction 2, and the gray gain 3 of the high-frequency components corresponding to the direction 3. The degree gain enhances the high-frequency components in each extraction direction, and then fuses the enhanced high-frequency components in all directions to obtain the high-frequency components after the infrared image enhancement processing. For high-frequency components in different extraction directions, the gray gain can be flexibly adjusted. For example, the gray gain in the edge tangent direction can be smaller, and the normal gray gain can be larger to avoid noise in the edge tangent direction. Of course, in some embodiments, it is also possible to directly determine an optimal extraction direction, such as the direction of the edge normal, and then extract high-frequency components based on the determined optimal extraction direction. In addition, for the high-frequency part, the gray-scale gain can be controlled in combination with various factors, so that the enhancement of the high-frequency component is more flexible and controllable, and a better enhancement effect can be achieved.

Through the infrared image method provided by this application, when the infrared image is enhanced, the low-frequency and high-frequency components of the infrared image can be extracted separately, and the low-frequency components can be subjected to contrast stretching processing. For the high-frequency components, one or more Extract the high-frequency components in the extraction direction, and determine the corresponding gray-scale gain for the high-frequency components in each extraction direction, so that the gray-scale gain in each extraction direction is more flexible and controllable, and can avoid the infrared image noise after the enhancement process For more obvious problems, improve the treatment effect.

In some embodiments, in order to avoid increasing the noise during the infrared image enhancement process, making the noise more obvious, therefore, before the infrared image is enhanced, the infrared image can be denoised first to remove the noise. Various noises in infrared images.

Because the part with sharp gray changes in the infrared image may be noise, or it may be the edge of the object, usually noise is often the most drastically changed part of the image, followed by the edges and details of the object in the image, and then the image in the image. Background and flat area. In order to improve the intensity of the edge of the object and at the same time to better suppress the intensity of the noise, and to prevent the enhanced infrared image noise from becoming more obvious, in some embodiments, two different high-pass components can be used when extracting high-frequency components. The filter extracts the high-frequency components in the infrared image to obtain the first high-frequency component and the second high-frequency component. The gray scale change degree of the image corresponding to the first high-frequency component is greater than that of the image corresponding to the second high-frequency component. The degree of grayscale change, that is to say, the first high-frequency component corresponds to the most severely changed part of the image, this part is likely to be noise, and the second high-frequency component corresponds to the more severely changed part of the image , This part is basically the edge or detail of the object.

For the first high-frequency part, since it is likely to be noise, it can be suppressed, while for the second high-frequency part, since it is basically the edges and details of the object, its gray gain can be appropriately enhanced. , To highlight the details of the image. Therefore, in some embodiments, when determining the gray-scale gain of the high-frequency component, the first high-frequency component and the second high-frequency component can be processed separately, and the gain of the first high-frequency component can be determined separately, which will be referred to as Is the first gray-scale gain and the gain of the second high-frequency component, hereinafter referred to as the second gray-scale gain, where the first gray-scale gain is smaller than the second gray-scale gain. By distinguishing the high-frequency components corresponding to the noise and the high-frequency components corresponding to the edges of the object, and determining their gray scale gains respectively, the details of the object can be enhanced, noise can be suppressed, and the processing effect can be improved.

In some embodiments, the extraction direction of the high-frequency components of the infrared image may be one or more preset directions, for example, it may be omnidirectional extraction, or it may be preset 2, 4, 8 or more. Multi-direction, taking 4 directions as an example, it can be horizontal direction, vertical direction, 45 angle direction and 135° angle direction. Then the template corresponding to each direction is used to extract the high-frequency components. In some embodiments, the extraction direction can also be determined in combination with the grayscale characteristics of each pixel of the infrared image, and the direction with a sharper grayscale change is selected as the extraction direction. For example, the degree of grayscale change corresponding to each pixel of the infrared image in multiple directions can be determined separately, and then the extraction direction can be selected from the multiple directions according to the degree of grayscale change corresponding to each pixel in these multiple directions. As shown in Figure 4, for the pixel point P0 in the infrared image, the gray scale change degree of the pixel point P0 in the eight directions from direction 1 to direction 8 can be determined, and then the gray scale change degree in these eight directions can be changed from The extraction direction is determined among these 8 directions. Of course, the directions shown in FIG. 4 are only illustrative examples. In practical applications, the number and angles of the multiple directions can be flexibly set according to requirements.

In some embodiments, the degree of grayscale change can be characterized by the gradient of each pixel in these multiple directions, or the grayscale of each pixel and the pixel's neighboring pixels in these multiple directions. Characterization of the variance of the grayscale. Of course, it can also be other parameters that can indicate the degree of change in the gray level of the pixel, which is not limited here.

In some embodiments, when the extraction direction of each pixel is determined according to the degree of grayscale change of each pixel, the direction with the largest degree of grayscale change among the multiple directions can be selected as the extraction direction of the pixel, or One or more directions in which the gray level change degree is greater than a preset threshold value are selected as the extraction direction.

In some embodiments, in order to ensure that the selected extraction direction is more reliable and more accurate, it is also possible to combine the extraction directions of multiple pixels adjacent to each pixel and the grayscale changes of each pixel in all directions. The degree determines the extraction direction of the pixel. As shown in Figure 5, assuming that the extraction direction of pixel P0 is to be determined, the extraction direction of adjacent pixels of P0 (the gray pixel in the figure, the arrow indicates the extraction direction) can be determined first, and then combined with P0 in all directions The degree of gray level change and the extraction direction of adjacent pixels determine the extraction direction of P0. Assuming that among the 8 adjacent pixels, the extraction direction of 6 pixels is the horizontal direction, and the other two are 45° angle directions. The probability that the extraction direction of P0 is the horizontal direction is also relatively high. Therefore, the adjacent pixels can be considered comprehensively. The extraction direction of the pixel and the degree of gray change of P0 in each direction are combined with the above two factors to determine the extraction direction.

In some embodiments, it is also possible to determine the respective confidence levels corresponding to the multiple directions according to the difference in the gray levels of each pixel in multiple directions, and then select the extraction direction from the multiple directions according to the confidence. Taking Figure 4 as an example, you can first determine the degree of grayscale change corresponding to the pixel point P0 in the direction 1 to the direction 8, assuming that they are R1-R8, and then determine the difference between R1 and R2-R7. If the difference is relatively large, indicate the direction 1. The probability of a direction that changes more drastically is greater, so the confidence of direction 1 is higher. If the difference is small, it means that the probability of direction 1 being a more drastically changing direction is small, so the confidence of direction 1 is small, and then the direction with the greatest confidence can be selected as the extraction direction, or the direction with the confidence greater than a certain threshold can be selected. As the extraction direction.

Of course, in some embodiments, the degree of grayscale change of each pixel in all directions, the extraction direction of neighboring pixels of each pixel, and the gray change of each pixel in all directions can also be integrated. The difference between the degree and the degree of gray change in the other directions comprehensively determines the confidence of each direction, and then selects the extraction direction from multiple directions according to the determined confidence.

In some embodiments, when extracting low-frequency components and high-frequency components in an infrared image, each image can be divided into multiple image blocks, and the high-frequency components and low-frequency components can be extracted in units of image blocks. . In some embodiments, it is also possible to use pixel points as the unit, for each pixel point, respectively extract its corresponding high-frequency components in one or more extraction directions, and then determine that each pixel point is in each extraction direction. Corresponding to the gray gain of the high-frequency components, the high-frequency components of each pixel are enhanced according to the gray gains of the high-frequency components corresponding to each pixel in each extraction direction, and then the high-frequency components in each extraction direction are enhanced. The frequency components are fused to obtain the high frequency components of each pixel after the enhancement processing.

In some embodiments, in order to more flexibly control the enhancement intensity of the high frequency part, the magnitude of the gray scale gain of the high frequency component corresponding to each pixel point in each extraction direction can be manually controlled or adjusted.

In some embodiments, when adjusting the gray scale gain of the corresponding high-frequency components of each pixel in different extraction directions, in order to enhance the edge of the object as far as possible along the normal direction of the object edge, avoiding The tangent direction of the edge of the object enhances the edge of the object, making the noise in the tangential direction of the edge more obvious. When determining the gray-scale gain of the high-frequency component corresponding to each extraction direction, the gray-scale gain of the high-frequency component corresponding to each extraction direction can be determined according to the corresponding confidence of each extraction direction, where the confidence can indicate that the direction is an edge The probability of the normal direction, or the probability that this direction is the direction with the most dramatic gray-scale change. The confidence level can be determined based on the difference between the gray level change degree of each pixel in the extraction direction and the gray level change degree in other multiple directions. The greater the difference, the higher the confidence level. After the confidence is determined, the gray-scale gain of the high-frequency components corresponding to each extraction direction can be determined according to the confidence. The greater the confidence, the higher the probability that the extraction direction is the edge normal direction. Therefore, its The gray scale gain can be as large as possible, and vice versa, the gray scale gain is as small as possible.

In some embodiments, when adjusting the gray gain of the corresponding high-frequency components of each pixel in different extraction directions, the high-frequency components in each extraction direction can also be determined according to one or more of the following information Corresponding gray-scale gain: the characteristic of the pixel itself, the characteristic of the neighborhood of the pixel, the characteristic of the infrared image, and the extraction information of the high-frequency component of the pixel.

In some embodiments, the gray scale gain of the high-frequency components of each pixel can be adjusted according to its own characteristics. The characteristics of each pixel include: the brightness of each pixel, the position distribution of each pixel in the infrared image, and / Or the maximum brightness change threshold of each pixel. Among them, if the brightness of the pixel is relatively large, the gray scale gain can be appropriately set to be smaller to prevent the pixel from being too bright after being enhanced. On the contrary, the gray scale gain can be appropriately set to be larger. In addition, the position distribution of pixels in the infrared image can also be used to determine its gray-scale gain. For example, the corners of the image have strong noise and have a pot-lid effect. Therefore, for the pixels distributed in the corners of the image , The gray gain can be appropriately smaller. In addition, in order to avoid the appearance of obvious black and white edges after the edge part is enhanced, the maximum brightness change threshold value of each pixel point after enhancement can also be preset. After the pixel point is enhanced, the brightness change cannot exceed the maximum brightness threshold value. The maximum change threshold is used to adjust the grayscale gain of the high-frequency components of each pixel.

In some embodiments, the gray gain of high-frequency components can be adjusted according to the characteristics of the neighborhood of each pixel. The neighborhood characteristics of the pixel include: the brightness of the neighborhood of each pixel, and the neighborhood of each pixel. Whether the domain is the user's region of interest, the amount of horizontal stripe noise contained in the neighborhood of each pixel, the signal-to-noise ratio of the neighborhood of each pixel, and/or whether there are thick edges in the neighborhood of each pixel. For example, for pixels located in the bright, gray, and dark areas of the image, you can adjust the gray gain of their high-frequency components separately to prevent the dark areas from being too noisy, and the bright areas are too sharp to cause black and white edge defects obvious. For the pixels in the dead black area and the overexposed area, the gray gain can be increased appropriately to make the details more obvious. For the pixels in the user's area of interest, the gray gain can be increased appropriately. For pixels in areas with low signal-to-noise ratio and a large number of horizontal stripes, the gray gain can be appropriately reduced. For pixels with thick edges around, you can appropriately reduce the gray scale gain, reduce the sharpness, and prevent black and white edges.

In some embodiments, when determining whether there is a thick border around a pixel, the gradient of each pixel in the neighborhood of the pixel can be determined, and whether there is a black border around it is determined according to the gradient of each pixel in the neighborhood. The range of the neighborhood can be set by yourself. In some embodiments, in order to reduce the amount of calculation and save processing resources, when detecting whether there are thick edges in the neighborhood, you can also downsample the image first, and then determine the pixel based on the downsampled image Whether there are rough edges around. After the down-sampling process, the range of the neighborhood can be appropriately reduced.

In some embodiments, the gray gain of the high frequency components of each pixel can be adjusted according to the characteristics of the infrared image. The characteristics of the infrared image include: the purpose of the infrared image, the collection time of the infrared image, the collection mode of the infrared image, and / Or the subsequent processing operation corresponding to the infrared image. For example, for infrared images used in scenes such as power inspections, the temperature linear correlation of infrared images is required in such scenes, so that high-temperature areas are easier to find, so the grayscale of high-frequency components of pixels can be appropriately reduced. Gain, weaken the degree of sharpening, and prevent misjudgment caused by sharpening. For infrared images used in security search and rescue scenes, the image details are required to be stronger, and the sharpness requirements are high, so the grayscale gain of the high-frequency components of the pixels can be appropriately larger. In addition, the grayscale gain can also be determined by combining the infrared image acquisition time and the acquisition mode. For example, for the infrared image acquired after the shutter is opened for a long time, the grayscale gain can be appropriately reduced, and the noise can be reduced by reducing the sharpness. The infrared image collected when the shutter is opened can appropriately enhance its gray gain. In addition, infrared sensors are divided into low-gain mode and high-gain mode. Among them, the low-gain mode has low signal-to-noise ratio and large noise, so the gray scale gain can be appropriately small to avoid excessive noise, and the high-gain mode has a high signal-to-noise ratio. The gray scale gain can be larger. Of course, you can also adjust the grayscale gain in combination with the subsequent processing operations corresponding to the infrared image. For example, if the enhanced infrared image needs to be compressed and encoded, you can moderately reduce the grayscale gain and reduce the sharpening intensity, thereby reducing Small bit stream reduces storage space and transmission bandwidth.

In some embodiments, the gray-scale gain of the high-frequency component of each pixel can be adjusted according to the extraction information of the high-frequency component of the pixel. The extraction information of the high-frequency component of the pixel includes the extraction of the high-frequency component of the pixel. The absolute value of the output result of the direction and/or filter for each pixel. For example, if the extraction direction is the direction in which the gray level changes most drastically, the gray level gain can be appropriately larger, and if it is omnidirectional extraction, the gray level gain can be appropriately smaller. In addition, the absolute value of the output result of each pixel can also be adjusted according to the high-pass filter (HPF). When the absolute value of the output result of the HPF is large, the gray scale gain can be smaller to prevent excessive Sharpening produces black and white edges.

In order to further explain the infrared image processing method provided by the present application, the following will be explained in conjunction with a specific embodiment.

Because the original infrared image collected by the infrared sensor has a narrow grayscale distribution, low contrast, and unclear details, it can be enhanced to improve the contrast of the infrared image, and sharpen the edges and details to obtain details Clear infrared image. In order to minimize the problem of excessive noise caused by edge sharpening processing, the following provides an infrared image processing method.

As shown in FIG. 6, it is a schematic diagram of the infrared image processing method of this application. After acquiring the infrared image to be processed, it can be denoised first to remove various noises. Then through the pre-designed high-pass filter (HPF), mid-pass filter (Migh-pass filter, MPF), low-pass filter (Ligh-pass filter, LPF) to extract each pixel of the infrared image The high-frequency component, intermediate-frequency component and low-frequency component. For the high-frequency and intermediate-frequency components of each pixel, it can be extracted from multiple extraction directions (only two extraction directions are shown in FIG. 6) to obtain the high-frequency and intermediate-frequency components corresponding to each extraction direction. Among them, the multiple extraction directions can be preset multiple directions, assuming that they are omnidirectional, horizontal, vertical, 45° angular direction, 135° angular direction, and each extraction direction corresponds to a template, and the template is used to extract The medium and high frequency components of each pixel in the corresponding extraction direction. Among them, for each extraction direction, the confidence level of the extraction direction corresponding to each pixel can be determined in advance. Specifically, the gradient or gray-scale variance of each pixel in the multiple extraction directions can be determined, and then each pixel can be determined. The difference between the gradient or variance of the extraction direction and the gradient or variance of other extraction directions. Further, the extraction direction of the neighboring pixels of the pixel can also be determined, and the gradient or variance of each pixel in each extraction direction can be integrated. The difference between the gradient or variance of the direction and the gradient or variance of other extraction directions, and the extraction direction of the neighboring pixels of the pixel can determine the confidence of each pixel in each direction. The greater the confidence, the extraction The greater the probability that the direction is the normal direction of the edge.

Then, it can be comprehensively determined according to factors such as the confidence of each pixel in each extraction direction, the characteristics of each pixel itself, the characteristics of each pixel's neighborhood, the characteristics of the infrared image to be processed, and the extraction information of the high-frequency components of each pixel. The gray-scale gains of the medium and high-frequency components corresponding to each extraction direction. The overall principles for setting the gray-scale gains of the medium and high-frequency components corresponding to each extraction direction are as follows:

(1) The greater the confidence of the extraction direction, the greater the probability that the extraction direction is the normal direction of the edge, the greater the gray gain of the medium and high frequency components corresponding to the extraction direction, and vice versa. The edge normal direction sharpens the edge to avoid obvious noise in the tangential direction. In addition, after each pixel is input into MPF and HPF, the absolute value of the output result is larger, and the gray scale gain can also be appropriately reduced.

(2) The high frequency component may be noise, and the intermediate frequency component is basically the edge details of the object. Therefore, the gray-scale gain of high-frequency components should be appropriately suppressed to avoid obvious noise. The gray gain of the intermediate frequency components should be as large as possible to highlight the details.

(3) The more noise in the infrared image area where the pixel is located, the gray gain should be as small as possible to avoid enhancing the noise. The area where the pixel is located is bright, there are thick edges around the pixel, and the pixel is located at the corner position, the gray gain should be as small as possible to avoid over-sharpening or black and white edges.

(4) The gray scale gain can be adjusted in combination with the application scene of the infrared image, and the gray scale gain can be larger for scenes that require higher details. The gray gain of the infrared image in the high gain mode can be larger, and the gray gain of the infrared image in the low gain mode can be smaller. If the infrared image needs to be compressed and encoded later, in order to save storage space and reduce the transmission bandwidth, the gray scale gain can also be appropriately smaller.

(5) A maximum grayscale change threshold can be set for each pixel, and the finally enhanced grayscale cannot exceed the maximum change threshold to avoid excessive enhancement. Therefore, the grayscale gain can be adjusted in conjunction with the maximum grayscale change threshold.

After determining the gray gains of the high-frequency components and intermediate-frequency components in each extraction direction, the determined gray-scale gains can be used to enhance the mid- and high-frequency components, and then the enhanced high-frequency components in each extraction direction of each pixel are merged. Obtain the high frequency component of the pixel, and fuse the enhanced intermediate frequency components in each extraction direction of each pixel to obtain the intermediate frequency component of the pixel. For the low-frequency component of each pixel, it can be adaptively stretched and enhanced to improve image contrast. Then, the enhanced high-frequency components, low-frequency components of each pixel and the low-frequency components after stretching are merged to obtain the enhanced gray scale of each pixel, and then the enhanced infrared image can be obtained.

The above infrared image enhancement method has the following advantages:

(1) Differentiating the high-frequency component, intermediate-frequency component, and low-frequency component of the infrared image. For the low-frequency component, which is mainly the background area and the flat area, it can be stretched adaptively to improve the contrast. For the middle and high frequency components, the gray scale gain can be controlled and adjusted manually, which is more flexible. For example, for high-frequency components, which may be noise, it should be suppressed. For intermediate-frequency components, which are mainly the edges and details of objects, the gray gain should be appropriately increased to highlight the details.

(2) When extracting medium and high frequency components, extract them in multiple directions. The gray gain can be determined based on the confidence of the extraction direction to ensure that the edge part is enhanced along the normal direction of the edge part as much as possible to avoid edge tangent direction noise obvious.

(3) Combining various factors to comprehensively adjust the gray gain of the medium and high frequency components of each pixel in each extraction direction, which can finely control the enhancement of each pixel in the image, and avoid the problem of obvious noise caused by the enhancement process , And greatly enhance the effect of the enhanced infrared image.

In addition, the present application also provides an infrared image processing device. As shown in FIG. 7, the device includes a processor 71, a memory 72, and a computer program stored on the memory 72 that can be executed by the processor 71, When the processor 71 executes the computer program, the following steps are implemented:

Performing contrast stretching processing on the low-frequency components;

In some embodiments, the extraction direction is determined in the following manner:

Respectively determining the degree of grayscale change corresponding to each pixel of the infrared image in multiple directions;

The extraction direction of each pixel point is determined from the multiple directions based on the degree of grayscale change.

In some embodiments, when the processor is configured to determine the extraction direction of each pixel from the multiple directions based on the degree of grayscale change, it is specifically configured to:

Determining the respective confidence levels corresponding to the multiple directions according to the difference in the degree of grayscale change in the multiple directions;

The extraction direction is determined from the multiple directions based on the confidence.

Selecting the direction with the smallest degree of grayscale change as the extraction direction; or

Determining the extraction direction of the neighboring pixel points of the pixel point;

The extraction direction is determined from the multiple directions according to the extraction direction of the adjacent pixel points and the degree of grayscale change.

In some embodiments, the degree of grayscale change is characterized by the following parameters:

The gradient of the pixel in multiple directions, or

The gray-scale variance of the pixel point and adjacent pixels in multiple directions.

In some embodiments, the processor is configured to determine the gray-scale gain of the high-frequency component corresponding to each of the extraction directions, and when performing enhancement processing on the high-frequency component according to the gray-scale gain, it is specifically used for:

Determining the gray scale gain of the high-frequency component corresponding to each pixel of the infrared image to be processed in the extraction direction;

The high-frequency component of each pixel is enhanced based on the gray scale gain of the high-frequency component corresponding to each pixel in the extraction direction to obtain the high-frequency component of each pixel after the enhancement processing.

In some embodiments, the gray scale gain of the high frequency component corresponding to each pixel point in each extraction direction is adjustable.

In some embodiments, the gray-scale gain of the high-frequency components corresponding to each pixel in each extraction direction is determined based on one or more of the following information: the characteristics of the pixel itself, the neighbors of the pixel The characteristics of the domain, the characteristics of the infrared image, and the extraction information of the high-frequency components of the pixels.

In some embodiments, the characteristics of the pixel point itself include: the brightness of the pixel point, the position distribution of the pixel point in the infrared image, and/or the maximum brightness change threshold of the pixel point.

In some embodiments, the neighborhood characteristics of the pixel point include: the brightness of the neighborhood, whether the neighborhood is a user region of interest, the amount of horizontal stripe noise contained in the neighborhood, and the neighborhood The signal-to-noise ratio and/or whether there are thick edges in the neighborhood.

In some embodiments, whether there are thick edges in the neighborhood is determined by the following methods:

Determining the gradient of each pixel in the neighborhood;

Determine whether there are thick edges in the neighborhood according to the gradient.

In some embodiments, the characteristics of the infrared image include: use of the infrared image, acquisition time of the infrared image, acquisition mode of the infrared image, and/or subsequent processing operations corresponding to the infrared image.

In some embodiments, the extraction information of the high-frequency component of the pixel point includes: an extraction direction of the high-frequency component of the pixel point and/or an extraction frequency threshold of the high-frequency component.

In some embodiments, when the processor is used to determine the gray scale gain of the high-frequency component corresponding to each of the extraction directions, it is specifically used to:

Determining the confidence level of the extraction direction according to the difference between the gray level change degree of each pixel of the infrared image to be processed in the extraction direction and the gray level change degree in other multiple directions;

The gray-scale gain of the high-frequency component in the extraction direction is determined based on the confidence.

In some embodiments, the high-frequency component includes a first high-frequency component and a second high-frequency component, and the grayscale change degree of the image corresponding to the first high-frequency component is greater than that of the second high-frequency component. The degree of grayscale change of the image.

In some embodiments, the determining the gray scale gain of the high-frequency component corresponding to the extraction direction includes:

Determine the first gray-scale gain of the first high-frequency component corresponding to the extraction direction, determine the second gray-scale gain of the second high-frequency component corresponding to the extraction direction, and the second gray-scale gain is greater than The first gray scale gain.

In some embodiments, the infrared image to be processed is an infrared image after denoising processing.

For the specific details of the infrared image processing device for enhancing the infrared image, reference may be made to the description in the foregoing embodiments, which will not be repeated here.

Optionally, the infrared image processing device further includes an infrared sensor for collecting infrared images.

The infrared image processing device may be an infrared camera, for example.

The infrared processing device mentioned in this application can be used in power inspection, industry inspection and other fields.

Further, this application also provides a movable platform, the movable platform may be an unmanned aerial vehicle, an unmanned boat, an unmanned car, etc., the movable platform includes the infrared image processing described in the above embodiments Device. Taking drones as an example, the drones can be equipped with infrared sensors and the above-mentioned infrared image processing devices for performing tasks such as temperature measurement, power inspection, and monitoring.

Correspondingly, an embodiment of this specification also provides a computer storage medium in which a program is stored, and the program is executed by a processor to implement the infrared image processing method in any of the above embodiments.

The embodiments of this specification may adopt the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program codes. Computer usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, Magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices.

For the device embodiment, since it basically corresponds to the method embodiment, the relevant part can refer to the part of the description of the method embodiment. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments. Those of ordinary skill in the art can understand and implement without creative work.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply one of these entities or operations. There is any such actual relationship or order between. The terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements that are not explicitly listed. Elements, or also include elements inherent to such processes, methods, articles, or equipment. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or equipment that includes the element.

The methods and devices provided by the embodiments of the present invention are described in detail above. Specific examples are used in this article to illustrate the principles and implementations of the present invention. The descriptions of the above embodiments are only used to help understand the methods and methods of the present invention. Core idea; At the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation and the scope of application. In summary, the content of this specification should not be construed as a limitation of the present invention .

Claims

An infrared image processing method, characterized in that the method includes:

Extracting low-frequency components of the infrared image to be processed and corresponding high-frequency components of the infrared image to be processed in one or more extraction directions;

Determining the gray-scale gain of the high-frequency component corresponding to each of the extraction directions, and performing enhancement processing on the high-frequency component according to the gray-scale gain;

Performing contrast stretching processing on the low-frequency components;

The low-frequency components after the stretching process and the high-frequency components after the enhancement process are combined to obtain a processed infrared image.
The method according to claim 1, wherein the extraction direction is determined in the following manner:

Respectively determining the degree of grayscale change corresponding to each pixel of the infrared image in multiple directions;

The extraction direction of each pixel point is determined from the multiple directions based on the degree of grayscale change.
The method according to claim 2, wherein determining the extraction direction of each pixel from the multiple directions based on the degree of grayscale change comprises:

Determining the respective confidence levels corresponding to the multiple directions according to the difference in the degree of grayscale change in the multiple directions;

The extraction direction is determined from the multiple directions based on the confidence.
The method according to claim 2, wherein determining the extraction direction of each pixel from the multiple directions based on the degree of grayscale change comprises:

Selecting the direction with the smallest degree of grayscale change as the extraction direction; or

Determining the extraction direction of the neighboring pixel points of the pixel point;

The extraction direction is determined from the multiple directions according to the extraction direction of the adjacent pixel points and the degree of grayscale change.
The method according to any one of claims 2-4, wherein the degree of grayscale change is characterized by the following parameters:

The gradient of the pixel in multiple directions, or

The gray-scale variance of the pixel point and adjacent pixels in multiple directions.
The method according to claim 1, wherein determining the gray scale gain of the high-frequency component corresponding to each of the extraction directions, and performing enhancement processing on the high-frequency component according to the gray scale gain, comprises:

Determining the gray scale gain of the high-frequency component corresponding to each pixel of the infrared image to be processed in the extraction direction;

The high-frequency component of each pixel is enhanced based on the gray gain of the high-frequency component corresponding to each pixel in the extraction direction, and the high-frequency component after the enhancement processing of each pixel is obtained.
The method according to claim 6, wherein the gray scale gain of the high frequency component corresponding to each pixel point in each extraction direction is adjustable.
The method according to claim 7, wherein the gray scale gain of the high-frequency component corresponding to the extraction direction of each pixel is determined based on one or more of the following information: the characteristics of the pixel itself , The characteristic of the neighborhood of the pixel, the characteristic of the infrared image, and the extraction information of the high-frequency component of the pixel.
The method according to claim 8, wherein the characteristics of the pixel point itself include: the brightness of the pixel point, the position distribution of the pixel point in the infrared image, and/or the maximum value of the pixel point. Brightness change threshold.
The method according to claim 8, wherein the neighborhood characteristics of the pixel point include: the brightness of the neighborhood, whether the neighborhood is a region of interest of the user, and the horizontal stripe noise contained in the neighborhood The number, the signal-to-noise ratio of the neighborhood and/or whether there are thick edges in the neighborhood.
The method according to claim 10, wherein whether there is a thick edge in the neighborhood is determined in the following manner:

Determining the gradient of each pixel in the neighborhood;

Determine whether there are thick edges in the neighborhood according to the gradient.
The method according to claim 8, wherein the characteristics of the infrared image include: the purpose of the infrared image, the collection time of the infrared image, the collection mode of the infrared image, and/or the infrared image Corresponding subsequent processing operations.
The method according to claim 8, wherein the extraction information of the high-frequency components of the pixel points comprises: the extraction direction of the high-frequency components of the pixel points and/or the filtering used to extract the high-frequency components The absolute value of the output result of the detector for the pixel.
The method according to claim 1, wherein determining the gray-scale gain of the high-frequency components corresponding to each of the extraction directions comprises:

Determining the confidence level of the extraction direction according to the difference between the gray level change degree of each pixel of the infrared image to be processed in the extraction direction and the gray level change degree in other multiple directions;

The gray-scale gain of the high-frequency component in the extraction direction is determined based on the confidence.
The method according to any one of claims 1-14, wherein the high-frequency component comprises a first high-frequency component and a second high-frequency component, and the gray level of the image corresponding to the first high-frequency component changes The degree is greater than the degree of grayscale change of the image corresponding to the second high-frequency component.
The method according to claim 15, wherein determining the gray scale gain of the high-frequency component corresponding to each of the extraction directions comprises:

Determine the first grayscale gain of the first high-frequency component corresponding to the extraction direction, determine the second grayscale gain of the second high-frequency component corresponding to the extraction direction, and the second grayscale gain is greater than the first Gray gain.
The method according to claim 1, wherein the infrared image to be processed is an infrared image after denoising processing.
An infrared image processing device, characterized in that the device includes a processor, a memory, and a computer program stored in the memory that can be executed by the processor, and the processor implements the following steps when the computer program is executed :

Extracting low-frequency components of the infrared image to be processed and corresponding high-frequency components of the infrared image to be processed in one or more extraction directions;

Determining the gray-scale gain of the high-frequency component corresponding to each of the extraction directions, and performing enhancement processing on the high-frequency component according to the gray-scale gain;

Performing contrast stretching processing on the low-frequency components;

The low-frequency components after the stretching process and the high-frequency components after the enhancement process are combined to obtain a processed infrared image.
The device according to claim 18, wherein the extraction direction is determined in the following manner:

Respectively determining the degree of grayscale change corresponding to each pixel of the infrared image in multiple directions;

The extraction direction of each pixel point is determined from the multiple directions based on the degree of grayscale change.
The device according to claim 19, wherein when the processor is configured to determine the extraction direction of each pixel from the multiple directions based on the degree of grayscale change, it is specifically configured to:

Determining the respective confidence levels corresponding to the multiple directions according to the difference in the degree of grayscale change in the multiple directions;

The extraction direction is determined from the multiple directions based on the confidence.
The device according to claim 20, wherein when the processor is configured to determine the extraction direction of each pixel point from the multiple directions based on the degree of grayscale change, it is specifically configured to:

Selecting the direction with the smallest degree of grayscale change as the extraction direction; or

Determining the extraction direction of the neighboring pixel points of the pixel point;

The extraction direction is determined from the multiple directions according to the extraction direction of the adjacent pixel points and the degree of grayscale change.
The device according to any one of claims 19-21, wherein the degree of grayscale change is characterized by the following parameters:

The gradient of the pixel in multiple directions, or

The gray-scale variance of the pixel point and adjacent pixels in multiple directions.
18. The device according to claim 18, wherein the processor is configured to determine the gray-scale gain of the high-frequency component corresponding to each of the extraction directions, and perform enhancement processing on the high-frequency component according to the gray-scale gain When, specifically used for:

Determining the gray scale gain of the high-frequency component corresponding to each pixel of the infrared image to be processed in the extraction direction;

The high-frequency component of each pixel is enhanced based on the gray gain of the high-frequency component corresponding to each pixel in the extraction direction, and the high-frequency component after the enhancement processing of each pixel is obtained.
The device according to claim 23, wherein the gray scale gain of the high frequency component corresponding to each pixel point in each extraction direction is adjustable.
The device according to claim 24, wherein the gray scale gain of the high-frequency component corresponding to each pixel in each extraction direction is determined based on one or more of the following information: the characteristics of the pixel itself, The characteristic of the neighborhood of the pixel, the characteristic of the infrared image, and the extraction information of the high-frequency component of the pixel.
The device according to claim 25, wherein the characteristics of the pixel point itself include: the brightness of the pixel point, the position distribution of the pixel point in the infrared image, and/or the maximum value of the pixel point. Brightness change threshold.
The device according to claim 25, wherein the neighborhood characteristics of the pixel point include: brightness of the neighborhood, whether the neighborhood is a region of interest of the user, and horizontal stripe noise contained in the neighborhood The number, the signal-to-noise ratio of the neighborhood and/or whether there are thick edges in the neighborhood.
The device according to claim 27, wherein whether there are thick edges in the neighborhood is determined in the following manner:

Determining the gradient of each pixel in the neighborhood;

Determine whether there are thick edges in the neighborhood according to the gradient.
The device according to claim 25, wherein the characteristics of the infrared image include: the purpose of the infrared image, the collection time of the infrared image, the collection mode of the infrared image, and/or the infrared image Corresponding subsequent processing operations.
The device according to claim 25, wherein the extraction information of the high-frequency component of the pixel point comprises: an extraction direction of the high-frequency component of the pixel point and/or a filter used to extract the high-frequency component The absolute value of the output result of the detector for the pixel.
The device according to claim 18, wherein when the processor is used to determine the gray scale gain of the respective high-frequency components corresponding to the extraction directions, it is specifically used to:

Determining the confidence level of the extraction direction according to the difference between the gray level change degree of each pixel of the infrared image to be processed in the extraction direction and the gray level change degree in other multiple directions;

The gray-scale gain of the high-frequency component in the extraction direction is determined based on the confidence.
The device according to any one of claims 18-31, wherein the high-frequency component comprises a first high-frequency component and a second high-frequency component, and the gray level of the image corresponding to the first high-frequency component changes The degree is greater than the degree of grayscale change of the image corresponding to the second high-frequency component.
The device according to claim 32, wherein the determining the gray scale gain of the high-frequency component corresponding to the extraction direction comprises:

Determine the first gray-scale gain of the first high-frequency component corresponding to the extraction direction, determine the second gray-scale gain of the second high-frequency component corresponding to the extraction direction, and the second gray-scale gain is greater than The first gray scale gain.
The device according to claim 18, wherein the infrared image to be processed is an infrared image after denoising processing.
A movable platform, wherein the movable platform comprises the infrared image processing device according to any one of claims 18-34.
A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the infrared image processing according to any one of claims 1 to 17 is realized method.