WO2021127972A1

WO2021127972A1 - Image processing method and apparatus, imaging device, and movable carrier

Info

Publication number: WO2021127972A1
Application number: PCT/CN2019/127865
Authority: WO
Inventors: 张青涛; 曹子晟; 庹伟; 杨磊; 赵新涛
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-12-24
Filing date: 2019-12-24
Publication date: 2021-07-01
Also published as: CN112544064A

Abstract

An image processing method, comprising: performing image correction and signal-to-noise ratio improvement on a grayscale map; and performing contrast stretching on the grayscale map after image correction and signal-to-noise ratio improvement, wherein the image correction comprises: performing a flat field correction on the grayscale map; and performing bad point correction on the flat field-corrected grayscale map; and the signal-to-noise ratio improvement comprises: performing time domain noise removal on the grayscale map.

Description

Image processing method, device, imaging equipment and movable carrier

Technical field

The present disclosure relates to the field of image processing technology, and in particular to an image processing method, device, imaging equipment, and a movable carrier.

Background technique

In the field of image processing, the images collected by the image sensor usually have some flaws. For example, in the existing infrared image processing device, due to the manufacturing process of the infrared sensor itself, there are problems such as obvious dead pixels and noise that cannot be removed cleanly. The contrast and details of the grayscale image directly output by the infrared sensor are not ideal. Therefore, the image processing process needs to be improved to show the details of the scene objects and obtain high-quality image output.

Public content

The present disclosure provides an image processing method, which includes performing image correction and signal-to-noise ratio enhancement on a grayscale image; performing contrast stretching on the grayscale image after image correction and signal-to-noise ratio enhancement; wherein the image correction includes: Perform flat-field correction on the grayscale image; and perform dead pixel correction on the grayscale image after the flat-field correction; the signal-to-noise ratio improvement includes: removing temporal noise on the grayscale image.

The present disclosure also provides an image processing device, including: a correction unit for performing image correction on a grayscale image; a noise reduction unit for improving a signal-to-noise ratio of the grayscale image; and a stretching unit for performing image correction on the grayscale image. The grayscale image after image correction and signal-to-noise ratio enhancement is contrast stretched; wherein, the correction unit includes: a flat field correction unit for performing flat field correction on the grayscale image; a dead pixel correction unit for performing flat field correction on the grayscale image; The flat-field corrected gray image is corrected for dead pixels; the noise reduction unit includes: a time-domain noise reduction unit for removing time-domain noise on the gray image.

The present disclosure also provides an imaging device. The imaging device includes an image sensor and an image processing device. The image sensor is connected to the image processing device. The image processing device includes: a correction unit for grayscale image processing. Image correction; a noise reduction unit for improving the signal-to-noise ratio of the grayscale image; and a stretching unit for performing contrast stretching on the grayscale image after the image correction and the signal-to-noise ratio increase; wherein, the The correction unit includes: a flat-field correction unit, which is used to perform flat-field correction on a grayscale image; a dead pixel correction unit, which is used to perform dead-point correction on a grayscale image after flat-field correction; and the noise reduction unit includes: time domain The noise reduction unit is used to remove the time domain noise of the gray image.

The present disclosure also provides a movable carrier, including: a body and an imaging device, the imaging device is installed in the body, the imaging device includes an image sensor and an image processing device, the image sensor is connected to the An image processing device, the image processing device comprising: a correction unit for performing image correction on the grayscale image; a noise reduction unit for improving the signal-to-noise ratio of the grayscale image; and a stretching unit for performing image correction on the image The grayscale image after the correction and signal-to-noise ratio increase is contrast-stretched; wherein, the correction unit includes: a flat field correction unit for performing flat field correction on the grayscale image; a dead pixel correction unit for performing flat field correction on the grayscale image; The corrected gray image is corrected for dead pixels; the noise reduction unit includes: a time domain noise reduction unit for removing time domain noise on the gray image.

The image processing method and device of the present disclosure solve the problems of dead pixels, obvious fixed pattern noise and random noise, low contrast, low signal-to-noise ratio, and few image details in the grayscale image output by the image sensor, resulting in fewer defects , Images with higher contrast, more detail, and better quality.

Description of the drawings

The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure, but do not constitute a limitation to the present disclosure. In the attached picture:

FIG. 1 is a flowchart of an image processing method according to an embodiment of the disclosure.

Fig. 2a is a flowchart of an image correction method according to an embodiment of the disclosure.

Fig. 2b is a flowchart of a method for improving a signal-to-noise ratio according to an embodiment of the disclosure.

FIG. 2c is a flowchart of contrast and detail enhancement according to an embodiment of the disclosure.

FIG. 3 is a flowchart of a specific implementation of the image processing method of the present disclosure.

Fig. 4a is a schematic diagram of an image processing device according to an embodiment of the present disclosure.

Fig. 4b is a schematic diagram of an image processing apparatus according to another embodiment of the present disclosure.

FIG. 5 is a schematic structural diagram of an imaging device according to an embodiment of the disclosure.

FIG. 6 is a schematic structural diagram of a movable carrier according to an embodiment of the disclosure.

Detailed ways

The present disclosure provides an image processing method, device, imaging equipment and movable carrier. The image processing method, device, imaging device, and movable carrier of the present disclosure can overcome the problems of obvious dead pixels, fixed pattern noise, random horizontal stripes, low contrast, and inconspicuous details in the output of the existing image processing device, and provide high High-quality image output.

The technical solutions of the present disclosure will be clearly and completely described below in conjunction with the embodiments and the drawings in the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present disclosure.

In the first exemplary embodiment of the present disclosure, an image processing method is provided. The processing object of the image processing method in this embodiment is an example of a grayscale image output by an infrared sensor, where the infrared sensor can be configured to detect and form an image based on infrared radiation (for example, having a wavelength between 700 nm and 1 mm) .

In this embodiment, the image processing method may first perform the dynamic range correction of the infrared sensor. The dynamic range of an infrared sensor can be understood as the ratio of the brightness of the brightest part to the darkest part of the infrared image. When performing image processing, you can first store the received direct output data of the infrared sensor, check the dynamic range of the infrared sensor, and perform the dynamic range correction of the infrared sensor according to the dynamic range check result, and then correct the gray after the dynamic range correction. Image processing is performed on the degree map.

The image processing method of this embodiment will be described in detail below.

FIG. 1 is a flowchart of an image processing method according to an embodiment of the disclosure. As shown in Figure 1, the image processing method includes:

Step S1: Perform image correction and signal-to-noise ratio improvement on the grayscale image.

Step S2: Perform contrast stretching on the grayscale image after image correction and signal-to-noise ratio enhancement.

Wherein, the image correction and signal-to-noise ratio improvement in step S1 are performed before the contrast stretching of the grayscale image in step S2, so that the contrast stretching process can more accurately process the effective image information and reduce the image content. The noise interference, so the output image picture is cleaner. Wherein, the order of the image correction step and the signal-to-noise ratio improvement step on the gray image in the step S1 can be exchanged.

As an embodiment, for the grayscale output of the infrared sensor, image correction may be performed first, for example, to correct the consistency of the response curve of the photosensitive unit of the image sensor or to correct the dead pixels of the image sensor. On the basis of the degree map, the signal-to-noise ratio is improved, such as removing the time-domain noise in the grayscale image. Through the above-mentioned image processing method of first image correction and then noise reduction, it is possible to obtain more accurate image information in the subsequent image processing process.

As another embodiment, for the grayscale image output by the infrared sensor, the signal-to-noise ratio can be improved first, for example, to remove the time-domain noise in the grayscale image, and on the basis of the grayscale image after the signal-to-noise ratio is improved , And then perform image correction, such as correcting the consistency of the response curve of the photosensitive unit of the image sensor or correcting the dead pixels of the image sensor. Through the above-mentioned image processing method of noise reduction first and image correction, the image can retain more edge characteristics and maintain image details.

Fig. 2a is a flowchart of an image correction method according to an embodiment of the disclosure. As shown in Figure 2a, in the step S1, performing image correction on the grayscale image includes:

Step S101: Perform flat field correction on the grayscale image.

Since the response of the photosensitive unit of the image sensor is not strictly consistent, the uniformity of the response curve of the image sensor can be corrected by flat-field correction, which can change the slope and offset of the response curve of each photosensitive unit of the image sensor, so as to eliminate the response. Interference with inconsistent curves.

In this embodiment, when performing flat-field correction on the grayscale image, a shutter located between the lens and the infrared sensor is used to capture a scene image for flat-field correction, and the infrared sensor is corrected. Specifically, by taking the image frame collected when the shutter is closed as a reference grayscale image, storing the reference grayscale image in a memory (such as a DDR memory), performing multi-frame averaging and then outputting backwards to obtain a pixel-by-pixel offset correction Use the flat field frame to perform flat field correction of the infrared sensor.

It should be noted that in other embodiments, the flat field correction may also be completed without using the shutter. For example, a blurred image frame can be obtained by accumulating multiple image frames captured by the infrared sensor, or a blurred image frame can be obtained by deliberately defocusing the optical elements or other parts of the infrared sensor, and by processing the blurred image Frame to determine the appropriate correction term to be applied to the captured image frame. Alternatively, when an appropriate trigger event and/or condition is detected, which can indicate the existence of an object or scene acting as a shutter (for example, a virtual shutter), a virtual shutter correction program can be started to generate correction items for flat-field correction and so on. It can be understood that the completion of flat-field correction without using a shutter is not limited to the above-mentioned method.

Step S102: Perform dead pixel correction on the gray image after flat field correction.

Due to the influence of factors such as manufacturing process, transportation links, and service life, image sensors have a certain number of bad pixels, and the brightness values of these pixels cannot reflect the captured images, which are usually called dead pixels. In the early stage of image processing, especially after flat-field correction, dead pixel correction can effectively prevent the spread of dead pixels.

According to the probability of occurrence, dead pixels are divided into two types, namely static dead pixels and dynamic dead pixels. Regarding the above two types of dead pixels, in step S102, performing dead pixel correction on the flat-field corrected grayscale image includes:

S1021: Correct the pixels corresponding to the static dead pixels of the image sensor.

Static dead point correction includes static bright point correction and static dark point correction. Among them, the static bright spot and the static dark spot are calibrated in advance. When the static dead spot is calibrated in advance, the camera lens can be blacked out, and the straight-out photo of the specific exposure parameters including all the effective pixels of the image sensor can be taken, and then the software can be used Analyze all pixels, find out the coordinates of the pixels whose brightness is higher than the average value of the surrounding similar pixels by a certain threshold, mark these pixels as static bright spots, and the coordinates of the pixels whose brightness is lower than the average value of the surrounding similar pixels by a certain threshold , Mark these pixels as static dark spots, and save the coordinates of the static bright spots and static dark spots.

When the infrared sensor takes pictures and videos normally, it can be calibrated according to the calibrated static dead point coordinates. Specifically, in the image processing process, the value of the pixel marked as a static dead pixel will not be used, but the value obtained by a certain calculation of the value of the same pixel around it will replace the value of the pixel marked as a bright spot. value.

S1022: Detect dynamic dead pixels of the image sensor, and correct pixels corresponding to the dynamic dead pixels.

The dynamic dead pixel correction is carried out based on the dead pixels detected online during normal photographing or video recording. When performing dynamic bad pixel correction, analyze and compare the difference between each pixel and the surrounding pixels of the same type for one frame of the grayscale. If the surrounding pixel values are more consistent, but the pixel value is quite different, the pixel is marked as bad Point, for this photo or video frame, the value of the pixel will not be used, but the value of the same pixel around it after a certain calculation to replace the value of the pixel marked as a bad point. Because the dynamic dead pixel correction is calculating the grayscale image that is taken normally, the conditions for finding the location of the dead pixel are not ideal conditions, so the dynamic dead pixel correction is likely to miss the dead pixels or catch some noise points by mistake, and the position of the pixel is also Will not be saved. Dynamic dead point correction can detect bright and dark spots at the same time.

In some embodiments, in the image correction process in the step S1, after the step 101 and before the step 102, the method further includes:

S101', non-linear correction is performed on the gray image after flat field correction.

The straight-out image of the infrared sensor usually has random stripes. The appearance of random stripes is caused by the non-uniformity of the infrared sensor's focal plane. Due to the limitation of the infrared sensor production process, the non-uniformity of the generation process cannot be avoided. Therefore, the quality of infrared image output can be improved through nonlinear correction in the engineering application process. Wherein, the non-linear correction generally includes calibration-based non-linear correction and scene-based non-linear correction.

Specifically, the non-linear correction based on calibration can further perform pixel-level responsivity correction based on the pixel-by-pixel responsivity difference of the infrared sensor calibrated in advance, and at the same time, correct the pixel-level offset of the grayscale image, and finally output to the next level. The responsivity and bias of the entire image are kept consistent with the grayscale image. Among them, the pixel-level responsivity refers to the output signal voltage generated per unit radiation power. Typical algorithms for non-linear correction based on calibration generally use two-point correction and single-point correction. Scene-based nonlinear correction does not need to be calibrated in advance to obtain reference sources. Typical algorithms generally include high-pass filtering, constant statistics, algebraic correction, Kalman filtering, and neural network algorithms.

By performing nonlinear correction on the infrared image, irregular stripes in the image can be effectively removed, a clearer output image can be obtained, and the imaging quality of the infrared sensor can be improved.

In the step S1, in addition to the step of performing image correction on the gray image, it also includes the step of improving the signal-to-noise ratio of the gray image. Fig. 2b is a flowchart of a method for improving a signal-to-noise ratio according to an embodiment of the disclosure. As shown in Figure 2b, the improvement of the signal-to-noise ratio of the grayscale image includes:

S103: Perform time-domain noise removal according to the time-domain noise characteristics of the infrared sensor, including at least one of time-domain random single-point noise, time-domain random row noise, or time-domain random column noise. Specifically, a memory can be used to buffer the image frames before and after denoising, and the similarity and difference between the two frames can be used for filtering to improve the signal-to-noise ratio.

Generally, the time-domain noise reduction adopts the multi-image averaging method. Since the random noise introduced by the infrared sensor appears as zero-mean additive noise in time, the multi-image averaging method can effectively remove the noise. In the time-domain noise reduction process, by analyzing and calculating multiple frames of images, it is possible to prevent the residual motion in the image.

In some embodiments, the denoising the grayscale image further includes:

S104: Remove fixed pattern noise in the grayscale image.

Infrared sensors usually have inherent deviations such as sensitivity deviations, circuit gains and offsets, so the infrared direct image generally has fixed pattern noise interference. Wherein, the fixed pattern noise includes column fixed pattern noise and row fixed pattern noise. Exemplarily, the line-fixed mode noise can appear as periodic horizontal stripes, which is due to the parasitic resistance and capacitance in the analog accumulator circuit. Circuit mismatch will cause uneven brightness of the output image in the scanning direction and periodic attenuation. ; And the column fixed pattern noise appears as vertical stripes with light and dark changes. The reason is that the system structure of the sensor column parallel readout circuit is prone to mismatch between columns due to process deviations, which causes the output image to be in the scanning direction of the TDI The vertical direction) the brightness is not uniform.

As an implementation manner, the step S104 to remove the fixed pattern noise in the grayscale image may be performed after the time domain noise reduction is performed in step S103, so that the image is removed from the motion residue after the time domain noise reduction, which is beneficial to the subsequent accurate finding The fixed pattern noise of the image is removed. As another implementation manner, the step S104 to remove the fixed pattern noise in the grayscale image may also be performed before the time domain noise reduction is performed in the step S103.

S105, removing random noise in the spatial domain in the grayscale image.

After the time domain noise reduction is performed, since the motion of the local area in the image is not the same, the degree of noise reduction in each area after the time domain noise reduction is also different, and the moving area usually has a larger noise residue than the static area. Therefore, spatial noise reduction can be further used to improve image quality. Different from the temporal noise reduction process that uses the temporal correlation of multiple frames of images to reduce noise, spatial noise reduction is to sample a single frame image and use the spatial correlation within the image to reduce noise, that is, use the current pixel and The similarity and difference between the neighborhoods are filtered to improve the signal-to-noise ratio.

Low-pass filtering can be used for spatial noise reduction. Low-pass filtering is a method of processing image signals in the frequency domain. When analyzing the frequency characteristics of an image signal, the high-frequency component of an image includes random noise of the image, the edge and step part of the image, while the background area belongs to the low-frequency component of the image, and the image is filtered by low-pass filtering. The high-frequency part of the signal can achieve the purpose of removing random noise. However, since the edge part and step part of the image are also in the high-frequency area, this part of the signal is also filtered out after low-pass filtering, resulting in image edges and jumps Part of it appears blurred.

It is understandable that the noise reduction process of spatial noise reduction will lose more details. Therefore, if you need to obtain the temperature measurement data of the infrared sensor, you need to use the data before the spatial noise reduction, such as removing the fixed in the grayscale image. Data after mode noise (especially temperature measurement data for absolute temperature measurement).

In order to alleviate the blur caused by low-pass filtering in spatial noise reduction, a filter with boundary preservation is usually used. The main idea of this filter is to set a threshold according to the noise variance, and only use the neighborhood points that fall within the threshold for low-pass Filtering, when the boundary jump amplitude is greater than the threshold, there will be no blur, and the boundary with the jump amplitude less than the threshold will also be blurred. The use of boundary preservation filtering for noise reduction requires accurate estimation of the variance of the noise to achieve a better noise reduction effect. However, whether it is a low-pass filter noise reduction method or a noise reduction method with boundary preservation, spatial noise reduction is inevitable The larger the noise, the more the texture is submerged by the noise, and the more serious the blur of the image after noise reduction. It is difficult to obtain satisfactory results only by relying on spatial correlation. Therefore, spatial noise reduction needs to be time-dependent. Cooperate with domain noise reduction, and use the correlation of the time domain to use the information of multiple frames to compensate each other, so as to achieve a better noise reduction effect.

FIG. 2c is a flowchart of contrast and detail enhancement according to an embodiment of the disclosure. As shown in Figure 2c, after image correction and signal-to-noise ratio increase and noise reduction are performed in step S1, it further includes:

S1', perform spatial frequency separation on the gray image after image correction and signal-to-noise ratio enhancement and noise reduction;

S2', performing spatial frequency synthesis on the stretched grayscale image, and using the frequency-separated grayscale image to enhance the medium and high frequency components in the stretched grayscale image.

The step S1' is executed after the image is corrected and the signal-to-noise ratio is improved in the step S1. The frequency separation in the spatial domain is performed to separate the middle and high frequency components representing the edges and step parts of the image from the low frequencies representing the background area of the image. The components are separated to prepare for the subsequent contrast stretching and detail enhancement to reduce noise and enhance details; after performing the step S1', perform step S2 to perform contrast stretching on the image, and after performing contrast stretching on the image, perform In the step S2', the spatial frequency synthesis is performed on the stretched grayscale image; at the same time, the spatial frequency separated grayscale image obtained in the step S1' can be used to compare the middle and high frequencies in the stretched grayscale image. The frequency component is enhanced, and the details are enhanced by compensating the frequency-separated gray image, so as to output an infrared gray image with enhanced contrast and details.

Since the frequency separation in the spatial domain is carried out before the contrast stretching, the high-frequency part and the low-frequency part in the image are distinguished, and only the low-frequency part of the image is contrast stretched, thus avoiding the random noise contained in the high-frequency part of the image. The image is stretched, causing the image to be blurred; further, the high-frequency part of the image is superimposed with the contrast-stretched image through frequency synthesis, so that the edges and step details contained in the high-frequency part of the image can be used to reduce the contrast. Compensation for the loss of detail in the process of stretching. As a result, it is ensured that the image details are enhanced while the image contrast is enhanced.

In the step S2, performing contrast stretching on the gray image after image correction and signal-to-noise ratio enhancement includes:

S201. Perform the first-level contrast stretching, including performing first gray-level histogram statistics on the gray-level value of each pixel in the gray-level image after frequency separation, and performing the first gray-level transformation on the pixels according to the statistical results. Pixel gray value after contrast stretching;

S202. Perform a second-level contrast stretching, including performing second gray-scale histogram statistics on the gray value of each pixel in the gray-scale image after the first gray-scale transformation, and perform second gray-scale statistics on the pixels according to the statistical results. The degree transformation obtains the pixel gray value after the contrast stretching, wherein the second gray scale transformation is different from the first gray scale transformation.

Specifically, the first grayscale transformation is a linear grayscale transformation or a nonlinear grayscale transformation; the second grayscale transformation may also be a linear grayscale transformation or a nonlinear grayscale transformation. Exemplarily, the first gray-scale transformation or the second gray-scale transformation are both piecewise linear transformations. When the first gray-scale transformation is performed, the contrast of the first useful data of the image is enhanced; and the second gray-scale transformation is performed. During the degree transformation, the contrast of the second useful data of the image is enhanced. Through the above method, the gray value of the image can be better distributed on the histogram to achieve the image enhancement effect.

It is understandable that the above-mentioned contrast stretching takes two-level contrast stretching as an example. In other embodiments, it is also possible to perform more than two-level contrast stretching steps according to the specific conditions of the grayscale image that needs to be contrast-enhanced. Multi-level contrast stretching.

Preferably, in the step S2, the performing contrast stretching on the gray image after the image correction and the signal-to-noise ratio increase further includes:

S200, before the statistics of the first grayscale histogram, linearly transform the grayscale values of the frequency-separated grayscale images, and perform preliminary linear stretching, which is the subsequent first-level contrast stretching and second-level contrast Get ready for stretching. Exemplarily, an image whose gray value of all pixels is originally only in a certain part of the grayscale area can be grayscale transformed by linear stretching, and stretched to the entire grayscale space of 0-255, so that the contrast of the image can be greatly improved. Enhanced.

It is understandable that the above-mentioned preliminary linear stretching, first-level contrast stretching, and second-level contrast stretching are respectively responsible for different degrees of stretching. Illustratively, linear stretching is performed first to achieve global contrast stretching. , The first-level contrast stretching and the second-level contrast stretching are used to achieve flexible two-level stretching to obtain better contrast. According to different image processing needs, the three stretching processes can be combined, or one, two or more of them can be used.

In some embodiments, the image processing method may further include:

S3: Perform scene analysis on the grayscale image that achieves image enhancement after frequency synthesis. For example, scene analysis is performed to obtain indoor, outdoor, black body, woods, seaside and other scene analysis results, and the scene analysis results are fed back to the previous image correction, noise reduction and/or stretching process to correct image correction, noise reduction and/or stretch The parameters of the stretching process are adjusted so that the entire system forms a feedback system through the feedback of scene analysis, which can adaptively perform appropriate correction, denoising, contrast enhancement and detail enhancement for different scenes. Exemplarily, if the scene analysis result obtained by the scene analysis process is a forest, the scene characteristics corresponding to the forest are obtained, and the noise reduction and contrast and detail enhancement processes are fed back according to the scene characteristics, and the noise reduction or image enhancement parameters of the aforementioned process are adjusted , Thereby further improving the quality of the output image. Among them, the scene characteristics corresponding to the forest can be pre-stored in the program or stored in the cloud server.

Among them, the scene analysis is carried out after the image frequency synthesis. The image after frequency synthesis has undergone contrast enhancement and detail enhancement. Therefore, the amount of data corresponding to the image is small, and the information contained is richer. After feedback, the parameters used in the process of noise reduction, contrast enhancement and detail enhancement can be better realized.

In addition, the image processing method may further include:

S4: Perform pseudo-color mapping on the gray image that has achieved image enhancement after frequency synthesis. Mapping the grayscale image to the YUV color map, on the one hand highlights the temperature distribution information, on the other hand highlights the details of the object; among them, the pseudo-color mapping is also performed after the image frequency synthesis, also because the image after the frequency synthesis has undergone contrast enhancement and details Enhanced, so that the information contained is richer, so the pseudo-color mapped image also contains the richest details and better contrast, which is conducive to achieving high-quality image output.

It should be noted that when scene analysis is performed at the same time, the input data of the scene analysis feedback unit should use the data before the pseudo-color mapping, so as to avoid excessive amount of information processed by the scene analysis.

Specifically, when the YUV color map is a color map of YUV444 type, the color map of YUV444 can also be transcoded into a color map of YUV422 or 420, and output backward, which is convenient for subsequent encoding and saves storage space.

FIG. 3 is a flowchart of an implementation manner of an image processing method according to an embodiment of the disclosure. Through the image processing method of this embodiment, the problems of dead pixels, obvious fixed pattern noise and random noise, low contrast, low signal-to-noise ratio, and few image details in the grayscale image output by the infrared sensor are solved, thereby obtaining more flaws. Infrared images with less, higher contrast, more detail, and better picture quality.

In a second exemplary embodiment of the present disclosure, an image processing device is provided. The processing object of the image processing device in this embodiment is a grayscale image output by an infrared sensor as an example.

The image processing device may include a signal receiving and control unit. The signal receiving and control unit is used to receive the direct output data of the infrared sensor, control the infrared sensor, and store the infrared direct output image frame in a memory, such as a DDR memory. Before image processing, the signal receiving and control unit can first perform dynamic range correction of the infrared sensor to improve the problem of insufficient dynamic range of the infrared sensor, and then perform image processing on the grayscale image after the dynamic range correction.

The image processing apparatus of this embodiment will be described in detail below.

4a-4b are schematic diagrams of an image processing device according to an embodiment of the disclosure. As shown in FIGS. 4a-4b, the image processing device includes an image correction unit, a noise reduction unit, and a stretching unit. Among them, the image correction unit is used to correct the gray image; the noise reduction unit is used to improve the signal-to-noise ratio of the gray image; the stretching unit is used to contrast the gray image after the image correction and the signal-to-noise ratio increase Stretch.

Figures 4a-4b show two different implementations of the image processing device. Wherein, the order of the image correction unit and the noise reduction unit can be exchanged, that is, the order of performing image correction and signal-to-noise ratio improvement on the grayscale image can be exchanged. The image correction of the image correction unit and the improvement of the signal-to-noise ratio of the noise reduction unit are performed before the stretching unit performs contrast stretching on the grayscale image, thereby enabling the contrast stretching process to process effective image information more accurately , Reduce the noise interference in the image, so the output image picture is cleaner.

As shown in Figure 4a, as an implementation manner, for the grayscale output of the infrared sensor, an image correction unit can be used to perform image correction, for example, to correct the consistency of the response curve of the photosensitive unit of the image sensor or to correct the image sensor. The dead pixels are corrected, and on the basis of the corrected gray image, the noise reduction unit is used to improve the signal-to-noise ratio, for example, to remove the time-domain noise in the gray image. Through the above-mentioned image processing method of first image correction and then noise reduction, it is possible to obtain more accurate image information in the subsequent image processing process.

As shown in Figure 4b, as another implementation manner, for the grayscale image output by the infrared sensor, the noise reduction unit can also be used to improve the signal-to-noise ratio, such as removing the time domain noise in the grayscale image, and reducing the noise On the basis of the latter grayscale image, the image correction unit is then used to perform image correction, such as correcting the consistency of the response curve of the photosensitive unit of the image sensor or correcting the dead pixels of the image sensor. Through the above-mentioned image processing method of noise reduction first and image correction, the image can retain more edge characteristics and maintain image details.

Specifically, the image correction unit further includes a flat field correction unit and a dead pixel correction unit.

Among them, the flat field correction unit is used to perform flat field correction on the grayscale image. The flat field correction unit corrects the consistency of the response curve of the image sensor, which can change the slope and offset of the response curve of each photosensitive unit, so as to eliminate the interference of the inconsistent response curve.

In this embodiment, when the flat field correction unit performs flat field correction on the grayscale image, it uses the shutter to capture a substantially uniform scene image when the shutter is in front of the infrared sensor to correct the infrared sensor. Specifically, by taking the image frame collected when the shutter is closed as a reference grayscale image, using the reference grayscale image as a substantially uniform scene image, storing the image frame in the DDR, performing multi-frame averaging, and outputting backwards to obtain A flat field frame used for pixel-by-pixel offset correction, and the flat field frame is used to perform flat field correction of the infrared sensor. It should be noted that in other embodiments, the flat field correction unit may also complete flat field correction without using a shutter.

The dead pixel correction unit is used to perform dead pixel correction on the gray image after flat field correction. The dead pixel correction unit is used to detect that the brightness value of the pixels in the image sensor cannot reflect the dead pixels of the captured image, and perform corresponding processing.

According to the probability of occurrence, dead pixels are divided into two types, namely static dead pixels and dynamic dead pixels: static dead pixels are those that must appear under certain conditions (such as the exposure time is long enough and the image sensor reaches a certain temperature); dynamic dead pixels are Even if a certain condition is met, it is a probability of occurrence of dead pixels (that is, the dead pixels appear in this photo, and may be normal next time).

For the above two types of dead pixels, the dead pixel correction unit further includes a static dead pixel correction sub-unit and a dynamic dead pixel correction sub-unit.

Wherein, the static dead pixel correction subunit is used to correct the pixels corresponding to the static dead pixels of the image sensor. Static dead pixel correction is to analyze the position of the dead pixel under ideal test conditions, and the coordinate position of the dead pixel is fixed. Among them, the static bright spot and the static dark spot are calibrated in advance, for example, when the non-linear correction unit performs the responsivity correction, the marking of the static dead spot is completed. When the infrared sensor takes pictures and videos normally, it can be calibrated according to the calibrated static dead point coordinates. Specifically, in the image processing process, the value of the pixel marked as a static dead pixel will not be used, but the value obtained by a certain calculation of the value of the same pixel around it will replace the value of the pixel marked as a bright spot. value.

The dynamic dead pixel correction subunit is used to detect the dynamic dead pixels of the image sensor and correct the pixels corresponding to the dynamic dead pixels. The dynamic dead pixel correction is carried out based on the dead pixels detected online during normal photographing or video recording. When performing dynamic bad pixel correction, analyze and compare the difference between each pixel and the surrounding pixels of the same type for one frame of the grayscale. If the surrounding pixel values are more consistent, but the pixel value is quite different, the pixel is marked as bad Point, for this photo or video frame, the value of the pixel will not be used, but the value of the same pixel around it after a certain calculation to replace the value of the pixel marked as bad. Because the dynamic dead pixel correction is calculating the grayscale image that is taken normally, the conditions for finding the location of the dead pixel are not ideal conditions, so the dynamic dead pixel correction is likely to miss the dead pixels or catch some noise points by mistake, and the position of the pixel is also Will not be saved. Dynamic dead point correction can detect bright and dark spots at the same time.

In some embodiments, the image correction unit further includes a non-linear correction unit, which is used to perform non-linear correction on the flat-field corrected grayscale image. By performing nonlinear correction on the infrared image, irregular stripes in the image can be effectively removed, a clearer output image can be obtained, and the imaging quality of the infrared sensor can be improved.

The noise reduction unit is used to improve the signal-to-noise ratio of the grayscale image, including a time-domain noise reduction unit. The time-domain noise reduction unit is used to remove time-domain noise according to the time-domain noise characteristics of the infrared sensor, including time-domain random single-point noise, time-domain random row noise, or time-domain random column noise. Specifically, a DDR memory can be used to buffer the image frames before and after denoising, and the similarity and difference between the two frames can be used for filtering to improve the signal-to-noise ratio.

Generally, the time-domain noise reduction unit adopts the multi-image averaging method. Since the random noise introduced by the infrared sensor appears as zero-mean additive noise in time, the multi-image averaging method can effectively remove the noise. In the time-domain noise reduction process, by analyzing and calculating multiple frames of images, it is possible to prevent the residual motion in the image.

In some embodiments, the noise reduction unit further includes a fixed pattern noise removing unit for removing fixed pattern noise in the grayscale image. Wherein, the fixed pattern noise includes column fixed pattern noise and row fixed pattern noise.

As an implementation manner, the fixed-pattern noise removal unit removing the fixed-pattern noise in the grayscale image may be performed after the time-domain noise reduction unit performs the time-domain noise reduction. As another implementation manner, the fixed-pattern noise removal unit removing the fixed-pattern noise in the grayscale image may be performed before the time-domain noise reduction unit performs the time-domain noise reduction. Preferably, the time-domain noise reduction may be performed by the noise reduction unit first, and then the fixed-pattern noise removal unit may perform the fixed-pattern noise removal, so that after the image has undergone time-domain noise reduction, the motion residue is removed, which is beneficial to the subsequent accurate finding of image fixation. Pattern noise is removed.

In some embodiments, the noise reduction unit further includes a spatial noise reduction unit for removing random noise in the spatial domain.

After the temporal denoising is first performed in the temporal denoising, since the motion of the local area in the image is not the same, the degree of noise reduction in each area after the temporal denoising is not the same, and the moving area is usually more than the static area. Larger noise residue, so the spatial noise reduction unit can be further used to improve the quality of the image. Corresponding to time domain noise reduction, the difference is that the spatial denoising unit samples a single frame image and uses the spatial correlation within the image to reduce noise, that is, using the current pixel The similarity and difference between the neighborhood and the neighborhood are filtered to improve the signal-to-noise ratio.

The spatial noise reduction unit may include a low-pass filter. Filtering out the high-frequency part of the image signal by a low-pass filter can achieve the purpose of removing random noise. However, since the edge part and step part of the image are also in the high-frequency area, this part of the signal is also filtered out after the low-pass filter. , Resulting in blurring of the edges and jumps of the image.

It is understandable that more details will be lost through the airspace noise reduction unit. Therefore, if you need to obtain the temperature measurement data of the infrared sensor, you need to use the data before airspace noise reduction.

In order to alleviate the blur caused by low-pass filtering in spatial noise reduction, a filter with boundary preservation is usually used. The main idea of this filter is to set a threshold according to the noise variance, and only use the neighborhood points that fall within the threshold for low-pass Filtering, when the boundary jump amplitude is greater than the threshold, there will be no blur, and the boundary with the jump amplitude less than the threshold will also be blurred. The use of boundary preservation filtering for noise reduction requires accurate estimation of the variance of the noise to achieve a better noise reduction effect. However, whether it is a low-pass filter noise reduction method or a noise reduction method with boundary preservation, spatial noise reduction is inevitable The larger the noise, the more the texture is submerged by the noise, and the more serious the blur of the image after noise reduction. It is difficult to obtain satisfactory results only by relying on spatial correlation. Therefore, the spatial noise reduction unit needs to be combined with The time-domain noise reduction unit cooperates, and uses the correlation of the time domain to use the information of multiple frames to compensate each other, so as to achieve a better noise reduction effect.

Further, the image processing device further includes a frequency separation unit and a frequency synthesis unit, wherein the frequency separation unit is used to perform spatial frequency separation on the gray image after image correction and signal-to-noise ratio enhancement and noise reduction; the frequency synthesis unit is used for Perform spatial frequency synthesis on the stretched grayscale image, and use the frequency-separated grayscale image to enhance the middle and high frequency components in the stretched grayscale image.

Specifically, the frequency separation unit performs frequency separation in the spatial domain after the image is corrected and the signal-to-noise ratio is improved, and separates the middle and high frequency components representing the edges and step parts of the image from the low frequency components representing the background area of the image, as Preparing for post-contrast stretching and detail enhancement to reduce noise and improve details; after that, the stretching unit performs contrast stretching on the image, and after the image is contrast stretched, the frequency synthesis unit performs spatial domain on the stretched grayscale image At the same time, the spatial frequency separated gray image obtained by the frequency separation unit can be used to enhance the medium and high frequency components in the stretched gray image, and the details can be improved by compensating the frequency separated gray image. In this way, the infrared gray image with enhanced contrast and details is output.

Since the frequency separation unit performs spatial frequency separation before the contrast stretching, it distinguishes the high frequency part from the low frequency part in the image. The stretching unit only performs contrast stretching on the low frequency part of the image, thereby avoiding the high frequency in the image. Part of the random noise contained in the image is stretched at the same time, causing the image to be blurred; further, the high-frequency part of the image is superimposed with the contrast-stretched image through the frequency synthesis unit, so that the edges and jumps contained in the high-frequency part of the image can be used. To compensate for the loss of detail in the process of contrast stretching. As a result, it is ensured that the image details are enhanced while the image contrast is enhanced.

In some embodiments, the stretching unit includes a first contrast stretching unit and a second contrast stretching unit. Wherein, the first contrast stretching unit is used to perform the first-level contrast stretching, including performing first grayscale histogram statistics on the grayscale value of each pixel in the frequency-separated grayscale image, and performing the statistics according to the statistical results. The pixel performs the first gray scale transformation to obtain the pixel gray value after contrast stretching;

The second contrast stretching unit is used to perform the second-level contrast stretching, including performing second gray-scale histogram statistics on the gray value of each pixel in the gray-scale image after the first gray-scale transformation, and according to the statistical result Performing a second grayscale transformation on the pixel to obtain a pixel grayscale value after contrast stretching, wherein the second grayscale transformation is different from the first grayscale transformation.

Specifically, the first grayscale transformation is a linear grayscale transformation or a nonlinear grayscale transformation; the second grayscale transformation may also be a linear grayscale transformation or a nonlinear grayscale transformation. Exemplarily, the first gray-scale transformation or the second gray-scale transformation are both piecewise linear transformations. When the first gray-scale transformation is performed, the contrast of the first useful data of the image is enhanced; and the second gray-scale transformation is performed. During the degree transformation, the contrast of the second useful data of the image is enhanced. Through the above device, the gray value of the image can be better distributed on the histogram to achieve the image enhancement effect.

It is understandable that the above-mentioned contrast stretching takes a two-level contrast stretching unit as an example. In other embodiments, the grayscale image for contrast enhancement can also be performed according to the specific situation of the contrast enhancement, and it can also include more than two levels of contrast stretching. Unit to achieve multi-level contrast stretching.

Preferably, the stretching unit further includes a linear stretching unit for performing a linear transformation on the gray value of the grayscale image after frequency separation before the statistics of the first grayscale histogram, and performing a preliminary linear stretching. Stretching to prepare for the subsequent first-level contrast stretching and second-level contrast stretching. Exemplarily, an image whose gray value of all pixels is originally only in a certain part of the grayscale area can be grayscale transformed by linear stretching, and stretched to the entire grayscale space of 0-255, so that the contrast of the image can be greatly improved. Enhanced.

It can be understood that the above-mentioned preliminary linear stretching unit, the first contrast stretching unit, and the second contrast stretching unit are respectively responsible for different degrees of stretching. Illustratively, the linear stretching unit first performs global contrast stretching. The first-stage contrast stretching unit and the second-stage contrast stretching unit are used to realize flexible two-stage stretching and obtain better contrast. According to different image processing needs, the three stretching processes can be combined, or one, two or more of them can be used.

In some embodiments, the image processing device further includes a scene analysis feedback unit. The scene analysis unit is used to perform scene analysis on the grayscale images that have realized image enhancement after frequency synthesis, such as indoor, outdoor, black body, woods, seaside, etc., and feed back the results of the scene analysis to the previous image correction, noise reduction and/or In the stretching process, the parameters of the image correction, noise reduction and/or stretching process are adjusted, so that the entire system forms a feedback system through the feedback of scene analysis, which can adaptively perform appropriate correction, denoising, and contrast for different scenes. Enhancement and detail enhancement. Exemplarily, if the scene analysis result obtained by the scene analysis process is a forest, the scene characteristics corresponding to the forest are obtained, and the noise reduction and contrast and detail enhancement processes are fed back according to the scene characteristics, and the noise reduction or image enhancement parameters of the aforementioned process are adjusted , Thereby further improving the quality of the output image. Wherein, the scene characteristics corresponding to the forest can be pre-stored in the program or stored in the cloud server, and the scene analysis unit can execute the call of the scene characteristics.

Among them, the scene analysis of the scene analysis feedback unit is performed after the image frequency synthesis. Since the frequency synthesized image has undergone contrast enhancement and detail enhancement, the information contained in it is richer. After the information contained in it is fed back, it can be Better realize the adjustment of the parameters used in the process of noise reduction, contrast enhancement and detail enhancement.

In addition, the image processing device further includes a pseudo-color mapping unit for mapping the grayscale image that has achieved image enhancement after frequency synthesis into a YUV color image. Therefore, on the one hand, the temperature distribution information is highlighted, on the other hand, the details of the object are highlighted; among them, the pseudo-color mapping is also performed after the image frequency synthesis, also because the image after frequency synthesis has undergone contrast enhancement and detail enhancement, so that the information contained Richer, so the pseudo-color mapping image also contains the richest details and better contrast, which is conducive to achieving high-quality image output.

It should be noted that when a scene analysis feedback unit is provided at the same time, the input data of the scene analysis feedback unit should adopt the data before the pseudo-color mapping, so as to avoid the excessive amount of information processed by the scene analysis.

Specifically, when the YUV color map is a color map of YUV444 type, a transcoding unit may also be used to transcode the color map of YUV444 into a color map of YUV422 or 420, and output it backward, which is convenient for subsequent encoding and saves storage space.

The image processing device may include any type of data processing device with data processing capabilities, such as but not limited to CPU, DSP, FPGA, CPLD, etc.

Through the image processing device of this embodiment, the problems of dead pixels, obvious fixed pattern noise and random noise, low contrast, low signal-to-noise ratio, and less image details in the grayscale image output by the infrared sensor are solved, thereby obtaining more flaws. Infrared images with less, higher contrast, more detail, and better picture quality.

A third exemplary embodiment of the present disclosure provides an image processing device, including a reading storage medium and one or more processors. The readable storage medium is used to store executable instructions; one or more processors are used to execute the executable instructions to execute the image processing method described in the first embodiment.

The fourth exemplary embodiment of the present disclosure provides an imaging device. FIG. 5 is a schematic structural diagram of an imaging device according to an embodiment of the disclosure. As shown in FIG. 5, the imaging device includes an image sensor and an image processing device, the image sensor is connected to the image processing device, and the image processing device adopts the image processing device described in the second embodiment.

The fifth embodiment of the present disclosure provides a movable carrier. FIG. 6 is a schematic structural diagram of a movable carrier according to an embodiment of the disclosure. As shown in FIG. 6, the movable carrier includes: a body and an imaging device, the imaging device is installed in the body, the imaging device includes an image sensor and an image processing device, the image sensor is connected to the An image processing device, the image processing device adopts the image processing device described in the second embodiment. Wherein, the movable carrier may be an unmanned aerial vehicle, an unmanned vehicle, an unmanned ship, and the like.

The sixth embodiment of the present disclosure provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor realizes the same as described in the first embodiment. The image processing method described.

Those skilled in the art can clearly understand that for the convenience and conciseness of the description, only the division of the above-mentioned functional modules is used as an example. In practical applications, the above-mentioned functions can be allocated by different functional modules as required, that is, the device The internal structure is divided into different functional modules to complete all or part of the functions described above. For the specific working process of the device described above, reference may be made to the corresponding process in the foregoing method embodiment, which is not repeated here.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, not to limit it; although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions recorded in the foregoing embodiments can still be modified, or some or all of the technical features can be equivalently replaced; in the case of no conflict, the features in the embodiments of the present disclosure can be combined arbitrarily; and these modifications or replacements It does not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present disclosure.

Claims

An image processing method, characterized in that it comprises:

Perform image correction and signal-to-noise ratio improvement on grayscale images;

Perform contrast stretching on the grayscale image after image correction and signal-to-noise ratio enhancement; among them,

The image correction includes:

Perform flat-field correction on grayscale images; and

Perform dead pixel correction on the gray image after flat field correction;

The signal-to-noise ratio improvement includes:

Removal of time-domain noise on grayscale images.
5. The image processing method of claim 1, wherein the image processing method further comprises:

Perform spatial frequency separation on the grayscale image after image correction and signal-to-noise ratio enhancement, and stretch the low-frequency part of the grayscale image after frequency separation;

Perform spatial frequency synthesis on the stretched grayscale image.
3. The image processing method of claim 2, wherein the image processing method further comprises:

The frequency-separated grayscale image is used to enhance the medium and high frequency components in the stretched grayscale image.
3. The image processing method according to claim 2, wherein said performing contrast stretching on the gray image after image correction and signal-to-noise ratio increase comprises:

Perform a first grayscale histogram statistics on the grayscale value of each pixel in the grayscale image after frequency separation, and perform the first grayscale transformation on the pixel according to the statistical result to obtain the pixel grayscale value after contrast stretching;

Perform a second grayscale histogram statistics on the grayscale value of each pixel in the grayscale image after the first grayscale transformation, and perform the second grayscale transformation on the pixels according to the statistical results to obtain the pixel grayscale after contrast stretching Value, wherein the second grayscale transformation is different from the first grayscale transformation.
The image processing method according to claim 4, wherein the first grayscale transformation is a linear grayscale transformation or a nonlinear grayscale transformation; and/or the second grayscale transformation is a linear grayscale transformation Or nonlinear gray scale transformation.
The image processing method according to claim 4, wherein said performing contrast stretching on the gray image after image correction and signal-to-noise ratio increase further comprises:

Before the statistics of the first grayscale histogram, the grayscale value of the grayscale image after frequency separation is linearly transformed.
The image processing method of claim 1, wherein the flat-field correction of the grayscale image comprises: using a reference grayscale image to perform flat-field correction on the grayscale image collected by the image sensor, and The reference grayscale image is a grayscale image collected by the image sensor when the shutter is closed.
8. The image processing method according to claim 1, wherein said performing dead pixel correction on the gray image after flat field correction comprises:

Correct the pixels corresponding to the static dead pixels of the image sensor; and/or

The dynamic dead pixels of the image sensor are detected, and the pixels corresponding to the dynamic dead pixels are corrected.
8. The image processing method of claim 1, wherein the signal-to-noise ratio improvement further comprises:

After removing the time domain noise, remove the fixed pattern noise in the grayscale image after the time domain noise reduction;

or;

Remove the fixed pattern noise in the grayscale image, and then remove the time domain noise on the grayscale image from which the fixed pattern noise has been removed.
9. The image processing method of claim 9, wherein the signal-to-noise ratio improvement further comprises:

After removing the time domain noise and the fixed pattern noise, the gray image is removed from the spatial domain random noise.
8. The image processing method of claim 1, wherein the signal-to-noise ratio improvement further comprises:

After removing the time domain noise, the spatial random noise in the grayscale image after the time domain noise reduction is removed.
8. The image processing method according to claim 1, wherein the time domain noise comprises at least one of time domain random single point noise, time domain random row noise, and time domain random column noise.
8. The image processing method of claim 1, wherein the image correction further comprises:

Before the dead pixel correction, non-linear correction is performed on the gray image after the flat field correction.
8. The image processing method of claim 1, further comprising:

Perform scene information analysis on the grayscale image after contrast stretching, and feed back the analysis result to at least one of the image correction, signal-to-noise ratio improvement, and contrast stretching, and adaptively adjust the image correction, S/N ratio improvement, contrast stretching at least one of the parameters.
8. The image processing method of claim 1, further comprising:

The grayscale image after contrast stretching is mapped to the YUV color image.
15. The image processing method of claim 15, further comprising:

When the output of the pseudo-color mapping is the YUV444 color map, the YUV444 color map is transcoded into the YUV422 or YUV420 color map.
8. The image processing method of claim 1, further comprising:

Before performing the flat field correction, receiving the grayscale image collected by the image sensor, controlling the image sensor, and checking and correcting the dynamic range of the image sensor.
8. The image processing method of claim 1 or 10, further comprising:

Determine the absolute temperature according to the gray image after removing the time domain noise or fixed pattern noise.
An image processing device, characterized in that it comprises:

Correction unit for image correction of grayscale images;

The noise reduction unit is used to improve the signal-to-noise ratio of the grayscale image; and

The stretching unit is used to perform contrast stretching on the grayscale image after the image correction and the signal-to-noise ratio increase;

Wherein, the correction unit includes:

Flat-field correction unit for flat-field correction of grayscale images;

The dead pixel correction unit is used to perform dead pixel correction on the gray image after flat field correction;

The noise reduction unit includes:

The time domain noise reduction unit is used to remove the time domain noise of the gray image.
19. The image processing device of claim 19, wherein the image processing device further comprises:

The frequency separation unit is used to perform spatial frequency separation on the gray image after the image correction and the signal-to-noise ratio increase, and output the low frequency part of the gray image after the frequency separation to the stretching unit;

The frequency synthesis unit is used to perform spatial frequency synthesis on the gray image output by the stretching unit.
The image processing device according to claim 20, wherein the frequency synthesis unit is further configured to receive the frequency-separated grayscale image, and use the frequency-separated grayscale image to output to the stretching unit The middle and high frequency components in the grayscale image are enhanced.
22. The image processing device according to claim 20, wherein the stretching unit comprises:

The first contrast stretching unit, the first contrast stretching unit is used to perform the first grayscale histogram statistics on the grayscale value of each pixel in the grayscale image after frequency separation, and perform the statistics on the pixels according to the statistical results The first gray scale transformation obtains the pixel gray value after contrast stretching;

The second contrast stretching unit is used to perform a second grayscale histogram statistics on the grayscale value of each pixel in the grayscale image output by the first contrast stretching unit, and perform a second grayscale on the pixel according to the statistical result The pixel gray value after the contrast stretching is obtained by transformation, wherein the second gray scale transformation is different from the first gray scale transformation.
The image processing device according to claim 22, wherein the first grayscale transformation is a linear grayscale transformation or a nonlinear grayscale transformation; and/or the second grayscale transformation is a linear grayscale transformation Or nonlinear gray scale transformation.
22. The image processing device according to claim 22, wherein the stretching unit further comprises:

Linear stretching unit, used to linearly transform the gray value of the gray image after frequency separation;

The first contrast stretching unit is also used to perform first gray-scale histogram statistics on the gray-scale value of each pixel that has been linearly transformed.
The image processing device according to claim 19, wherein the flat field correction unit is configured to perform flat field correction on the gray image collected by the image sensor by using a reference gray image, and the reference gray image is A grayscale image collected by the image sensor when the shutter is closed.
19. The image processing device of claim 19, wherein the dead pixel correction unit comprises:

The static dead pixel correction subunit is used to correct the pixels corresponding to the static dead pixels of the image sensor; and/or

The dynamic dead pixel correction subunit is used to detect the dynamic dead pixels of the image sensor and correct the pixels corresponding to the dynamic dead pixels.
The image processing device according to claim 19, wherein the noise reduction unit further comprises:

The fixed pattern noise removal unit is used to remove fixed pattern noise in the grayscale output from the bad pixel correction unit or the time domain noise reduction unit.
The image processing device according to claim 27, wherein the noise reduction unit further comprises:

The spatial noise reduction unit is used to remove the spatial random noise of the gray image after removing the time domain noise and the fixed pattern noise.
19. The image processing device of claim 19, wherein the noise reduction unit further comprises:

The spatial noise reduction unit is used to remove the spatial random noise of the gray image output by the time domain noise reduction unit.
The image processing device according to claim 19, wherein the time-domain noise reduction unit is used to remove at least one of time-domain random single point noise, time-domain random row noise, and time-domain random column noise.
19. The image processing device according to claim 19, wherein the correction unit further comprises:

A non-linear correction unit, after the flat-field corrected gray image is non-linearly corrected by the non-linear correction unit, the bad pixel correction unit performs bad pixel correction.
19. The image processing device of claim 19, further comprising:

The scene analysis feedback unit is used to analyze the scene information of the gray scale image after the contrast stretching, and feed back the analysis result to at least one of the image correction unit, the noise reduction unit and/or the stretching unit, for different scenes The parameters of at least one of the image correction unit, noise reduction unit, and stretching unit are adaptively adjusted.
19. The image processing device of claim 19, further comprising:

The pseudo-color mapping unit is used to map the grayscale image after the contrast stretching to the YUV color image.
The image processing device according to claim 33, further comprising:

The transcoding unit is used to transcode the YUV444 color map into a YUV422 or YUV420 color map when the pseudo color mapping unit outputs a YUV444 color map.
19. The image processing device of claim 19, further comprising:

The signal receiving and control unit is used to receive and save the grayscale image collected by the image sensor, to control the image sensor, and to check and correct the dynamic range of the image sensor.
The image processing device according to claim 19 or 27, further comprising:

The temperature measurement unit is used to determine the absolute temperature according to the grayscale image output by the time domain noise reduction unit or the fixed pattern noise removal unit.
An image processing device, characterized in that it comprises:

A readable storage medium for storing executable instructions;

One or more processors, configured to execute the executable instructions to execute the image processing method according to any one of claims 1-18.
An imaging device, characterized by comprising: an image sensor and an image processing device, the image sensor is connected to the image processing device, and the image processing device includes:

Correction unit for image correction of grayscale images;

The noise reduction unit is used to improve the signal-to-noise ratio of the grayscale image; and

The stretching unit is used to perform contrast stretching on the grayscale image after the image correction and the signal-to-noise ratio increase;

Wherein, the correction unit includes:

Flat-field correction unit for flat-field correction of grayscale images;

The dead pixel correction unit is used to perform dead pixel correction on the gray image after flat field correction;

The noise reduction unit includes:

The time domain noise reduction unit is used to remove the time domain noise of the gray image.
The imaging device according to claim 38, wherein the image processing device further comprises:

The frequency separation unit is used to perform spatial frequency separation on the grayscale image after image correction and signal-to-noise ratio enhancement, and output the frequency-separated grayscale image to the stretching unit;

The frequency synthesis unit is used to perform spatial frequency synthesis on the gray image output by the stretching unit.
The imaging device of claim 39, wherein the frequency synthesis unit is further configured to receive the frequency-separated grayscale image, and use the frequency-separated grayscale image to compare the grayscale output from the stretching unit The mid- and high-frequency components in the degree diagram are enhanced.
The imaging device according to claim 39, wherein the stretching unit comprises:

The first contrast stretching unit, the first contrast stretching unit is used to perform the first grayscale histogram statistics on the grayscale value of each pixel in the grayscale image after frequency separation, and perform the statistics on the pixels according to the statistical results The first gray scale transformation obtains the pixel gray value after contrast stretching;

The second contrast stretching unit is used to perform a second grayscale histogram statistics on the grayscale value of each pixel in the grayscale image output by the first contrast stretching unit, and perform a second grayscale on the pixel according to the statistical result The pixel gray value after the contrast stretching is obtained by transformation, wherein the second gray scale transformation is different from the first gray scale transformation.
The imaging device according to claim 41, wherein the first gray-scale transformation is a linear gray-scale transformation or a nonlinear gray-scale transformation; and/or the second gray-scale transformation is a linear gray-scale transformation or Non-linear gray scale transformation.
41. The imaging device according to claim 41, wherein the stretching unit further comprises:

Linear stretching unit, used to linearly transform the gray value of the gray image after frequency separation;

The first contrast stretching unit is also used to perform first gray-scale histogram statistics on the gray-scale value of each pixel that has been linearly transformed.
The imaging device according to claim 38, wherein the flat-field correction unit is configured to perform flat-field correction on the gray-scale image collected by the image sensor by using a reference gray-scale image, and the reference gray-scale image is The grayscale image collected by the image sensor when the shutter is closed.
The imaging device according to claim 38, wherein the dead pixel correction unit comprises:

The static dead pixel correction subunit is used to correct the pixels corresponding to the static dead pixels of the image sensor; and/or

The dynamic dead pixel correction subunit is used to detect the dynamic dead pixels of the image sensor and correct the pixels corresponding to the dynamic dead pixels.
The imaging device of claim 38, wherein the noise reduction unit further comprises:

The fixed pattern noise removal unit is used to remove fixed pattern noise in the grayscale output from the bad pixel correction unit or the time domain noise reduction unit.
The imaging device of claim 46, wherein the noise reduction unit further comprises:

The spatial noise reduction unit is used to remove the spatial random noise of the gray image after removing the time domain noise and the fixed pattern noise.
The imaging device of claim 38, wherein the noise reduction unit further comprises:

The spatial noise reduction unit is used to remove the spatial random noise of the gray image output by the time domain noise reduction unit.
The imaging device according to claim 38, wherein the time-domain noise reduction unit is used to remove at least one of time-domain random single-point noise, time-domain random row noise, and time-domain random column noise.
The imaging device according to claim 38, wherein the correction unit further comprises:

A non-linear correction unit, after the flat-field corrected gray image is non-linearly corrected by the non-linear correction unit, the bad pixel correction unit performs bad pixel correction.
The imaging device of claim 38, further comprising:

The scene analysis feedback unit is used to analyze the scene information of the gray image after the contrast stretching, and feed back the analysis result to at least one of the image correction unit, the noise reduction unit, and the stretching unit, and is adaptive to different scenes The parameters of at least one of the image correction unit, noise reduction unit, and stretching unit are adjusted.
The imaging device of claim 38, further comprising:

The pseudo-color mapping unit is used to map the grayscale image after the contrast stretching to the YUV color image.
52. The imaging device of claim 52, further comprising:

The transcoding unit is used to transcode the YUV444 color map into a YUV422 or YUV420 color map when the pseudo color mapping unit outputs a YUV444 color map.
The imaging device of claim 38, further comprising:

The signal receiving and control unit is used to receive and save the grayscale image collected by the image sensor, control the image sensor, and check and correct the dynamic range of the image sensor.
The imaging device according to claim 38 or 46, further comprising:

The temperature measurement unit is used to determine the absolute temperature according to the grayscale image output by the time domain noise reduction unit or the fixed pattern noise removal unit.
A movable carrier, comprising: a body and an imaging device, the imaging device is installed in the body, the imaging device includes an image sensor and an image processing device, the image sensor is connected to the image A processing device, the image processing device includes:

Correction unit for image correction of grayscale images;

The noise reduction unit is used to improve the signal-to-noise ratio of the grayscale image; and

The stretching unit is used to perform contrast stretching on the grayscale image after the image correction and the signal-to-noise ratio increase;

Wherein, the correction unit includes:

Flat-field correction unit for flat-field correction of grayscale images;

The dead pixel correction unit is used to perform dead pixel correction on the gray image after flat field correction;

The noise reduction unit includes:

The time domain noise reduction unit is used to remove the time domain noise of the gray image.
The movable carrier according to claim 56, wherein the image processing device further comprises:

The frequency separation unit is used to perform spatial frequency separation on the grayscale image after image correction and signal-to-noise ratio enhancement, and output the frequency-separated grayscale image to the stretching unit;

The frequency synthesis unit is used to perform spatial frequency synthesis on the gray image output by the stretching unit.
The removable carrier according to claim 57, wherein the frequency synthesis unit is further configured to receive the frequency-separated grayscale image, and use the frequency-separated grayscale image to output to the stretching unit The middle and high frequency components in the grayscale image are enhanced.
The movable carrier according to claim 57, wherein the stretching unit comprises:

The first contrast stretching unit, the first contrast stretching unit is used to perform the first grayscale histogram statistics on the grayscale value of each pixel in the grayscale image after frequency separation, and perform the statistics on the pixels according to the statistical results The first gray scale transformation obtains the pixel gray value after contrast stretching;

The second contrast stretching unit is used to perform a second grayscale histogram statistics on the grayscale value of each pixel in the grayscale image output by the first contrast stretching unit, and perform a second grayscale on the pixel according to the statistical result The pixel gray value after the contrast stretching is obtained by transformation, wherein the second gray scale transformation is different from the first gray scale transformation.
The removable carrier according to claim 59, wherein the first grayscale transformation is a linear grayscale transformation or a nonlinear grayscale transformation; and/or the second grayscale transformation is a linear grayscale transformation Or non-linear gray scale transformation.
The movable carrier according to claim 59, wherein the stretching unit further comprises:

Linear stretching unit, used to linearly transform the gray value of the gray image after frequency separation;

The first contrast stretching unit is also used to perform first gray-scale histogram statistics on the gray-scale value of each pixel that has been linearly transformed.
The movable carrier according to claim 56, wherein the flat field correction unit is configured to use a reference gray image to perform flat field correction on the gray image collected by the image sensor, and the reference gray image is A grayscale image collected by the image sensor when the shutter is closed.
The removable carrier according to claim 56, wherein the dead pixel correction unit comprises:

The static dead pixel correction subunit is used to correct the pixels corresponding to the static dead pixels of the image sensor; and/or

The dynamic dead pixel correction subunit is used to detect the dynamic dead pixels of the image sensor and correct the pixels corresponding to the dynamic dead pixels.
The movable carrier according to claim 56, wherein the noise reduction unit further comprises:

The fixed pattern noise removal unit is used to remove fixed pattern noise in the grayscale output from the bad pixel correction unit or the time domain noise reduction unit.
The movable carrier according to claim 64, wherein the noise reduction unit further comprises:

The spatial noise reduction unit is used to remove the spatial random noise of the gray image after removing the time domain noise and the fixed pattern noise.
The movable carrier according to claim 56, wherein the noise reduction unit further comprises:

The spatial noise reduction unit is used to remove the spatial random noise of the gray image output by the time domain noise reduction unit.
The movable carrier according to claim 56, wherein the time domain noise reduction unit is used to remove at least one of time domain random single point noise, time domain random row noise, and time domain random column noise.
The movable carrier according to claim 56, wherein the correction unit further comprises:

A non-linear correction unit, after the flat-field corrected gray image is non-linearly corrected by the non-linear correction unit, the bad pixel correction unit performs bad pixel correction.
The movable carrier according to claim 56, characterized in that it further comprises:

The scene analysis feedback unit is used to analyze the scene information of the gray image after the contrast stretching, and feed back the analysis result to at least one of the image correction unit, the noise reduction unit, and the stretching unit, and is adaptive to different scenes The parameters of at least one of the image correction unit, noise reduction unit, and stretching unit are adjusted.
The movable carrier according to claim 56, characterized in that it further comprises:

The pseudo-color mapping unit is used to map the grayscale image after the contrast stretching to the YUV color image.
The movable carrier according to claim 70, further comprising:

The transcoding unit is used to transcode the YUV444 color map into a YUV422 or YUV420 color map when the pseudo color mapping unit outputs a YUV444 color map.
The movable carrier according to claim 56, characterized in that it further comprises:

The signal receiving and control unit is used to receive and save the grayscale image collected by the image sensor, to control the image sensor, and to check and correct the dynamic range of the image sensor.
The movable carrier according to claim 56 or 64, further comprising:

The temperature measurement unit is used to determine the absolute temperature according to the grayscale image output by the time domain noise reduction unit or the fixed pattern noise removal unit.
The movable carrier according to claim 56, wherein the movable carrier includes at least one of an unmanned aerial vehicle, an unmanned vehicle, and an unmanned ship.
A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements any one of claims 1-18 Image processing method.