WO2021127972A1 - Image processing method and apparatus, imaging device, and movable carrier - Google Patents

Image processing method and apparatus, imaging device, and movable carrier Download PDF

Info

Publication number
WO2021127972A1
WO2021127972A1 PCT/CN2019/127865 CN2019127865W WO2021127972A1 WO 2021127972 A1 WO2021127972 A1 WO 2021127972A1 CN 2019127865 W CN2019127865 W CN 2019127865W WO 2021127972 A1 WO2021127972 A1 WO 2021127972A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
unit
grayscale
correction
gray
Prior art date
Application number
PCT/CN2019/127865
Other languages
French (fr)
Chinese (zh)
Inventor
张青涛
曹子晟
庹伟
杨磊
赵新涛
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN201980050132.6A priority Critical patent/CN112544064A/en
Priority to PCT/CN2019/127865 priority patent/WO2021127972A1/en
Publication of WO2021127972A1 publication Critical patent/WO2021127972A1/en

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/14Picture signal circuitry for video frequency region
    • H04N5/20Circuitry for controlling amplitude response
    • H04N5/202Gamma control
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/64Circuits for processing colour signals
    • H04N9/646Circuits for processing colour signals for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters

Definitions

  • the present disclosure relates to the field of image processing technology, and in particular to an image processing method, device, imaging equipment, and a movable carrier.
  • the images collected by the image sensor usually have some flaws.
  • the existing infrared image processing device due to the manufacturing process of the infrared sensor itself, there are problems such as obvious dead pixels and noise that cannot be removed cleanly.
  • the contrast and details of the grayscale image directly output by the infrared sensor are not ideal. Therefore, the image processing process needs to be improved to show the details of the scene objects and obtain high-quality image output.
  • the present disclosure provides an image processing method, which includes performing image correction and signal-to-noise ratio enhancement on a grayscale image; performing contrast stretching on the grayscale image after image correction and signal-to-noise ratio enhancement; wherein the image correction includes: Perform flat-field correction on the grayscale image; and perform dead pixel correction on the grayscale image after the flat-field correction; the signal-to-noise ratio improvement includes: removing temporal noise on the grayscale image.
  • the present disclosure also provides an image processing device, including: a correction unit for performing image correction on a grayscale image; a noise reduction unit for improving a signal-to-noise ratio of the grayscale image; and a stretching unit for performing image correction on the grayscale image.
  • the grayscale image after image correction and signal-to-noise ratio enhancement is contrast stretched; wherein, the correction unit includes: a flat field correction unit for performing flat field correction on the grayscale image; a dead pixel correction unit for performing flat field correction on the grayscale image;
  • the flat-field corrected gray image is corrected for dead pixels;
  • the noise reduction unit includes: a time-domain noise reduction unit for removing time-domain noise on the gray image.
  • the present disclosure also provides an imaging device.
  • the imaging device includes an image sensor and an image processing device.
  • the image sensor is connected to the image processing device.
  • the image processing device includes: a correction unit for grayscale image processing. Image correction; a noise reduction unit for improving the signal-to-noise ratio of the grayscale image; and a stretching unit for performing contrast stretching on the grayscale image after the image correction and the signal-to-noise ratio increase; wherein, the The correction unit includes: a flat-field correction unit, which is used to perform flat-field correction on a grayscale image; a dead pixel correction unit, which is used to perform dead-point correction on a grayscale image after flat-field correction; and the noise reduction unit includes: time domain The noise reduction unit is used to remove the time domain noise of the gray image.
  • the present disclosure also provides a movable carrier, including: a body and an imaging device, the imaging device is installed in the body, the imaging device includes an image sensor and an image processing device, the image sensor is connected to the An image processing device, the image processing device comprising: a correction unit for performing image correction on the grayscale image; a noise reduction unit for improving the signal-to-noise ratio of the grayscale image; and a stretching unit for performing image correction on the image
  • the grayscale image after the correction and signal-to-noise ratio increase is contrast-stretched; wherein, the correction unit includes: a flat field correction unit for performing flat field correction on the grayscale image; a dead pixel correction unit for performing flat field correction on the grayscale image;
  • the corrected gray image is corrected for dead pixels;
  • the noise reduction unit includes: a time domain noise reduction unit for removing time domain noise on the gray image.
  • the image processing method and device of the present disclosure solve the problems of dead pixels, obvious fixed pattern noise and random noise, low contrast, low signal-to-noise ratio, and few image details in the grayscale image output by the image sensor, resulting in fewer defects , Images with higher contrast, more detail, and better quality.
  • FIG. 1 is a flowchart of an image processing method according to an embodiment of the disclosure.
  • Fig. 2a is a flowchart of an image correction method according to an embodiment of the disclosure.
  • Fig. 2b is a flowchart of a method for improving a signal-to-noise ratio according to an embodiment of the disclosure.
  • FIG. 2c is a flowchart of contrast and detail enhancement according to an embodiment of the disclosure.
  • FIG. 3 is a flowchart of a specific implementation of the image processing method of the present disclosure.
  • Fig. 4a is a schematic diagram of an image processing device according to an embodiment of the present disclosure.
  • Fig. 4b is a schematic diagram of an image processing apparatus according to another embodiment of the present disclosure.
  • FIG. 5 is a schematic structural diagram of an imaging device according to an embodiment of the disclosure.
  • FIG. 6 is a schematic structural diagram of a movable carrier according to an embodiment of the disclosure.
  • the present disclosure provides an image processing method, device, imaging equipment and movable carrier.
  • the image processing method, device, imaging device, and movable carrier of the present disclosure can overcome the problems of obvious dead pixels, fixed pattern noise, random horizontal stripes, low contrast, and inconspicuous details in the output of the existing image processing device, and provide high High-quality image output.
  • an image processing method is provided.
  • the processing object of the image processing method in this embodiment is an example of a grayscale image output by an infrared sensor, where the infrared sensor can be configured to detect and form an image based on infrared radiation (for example, having a wavelength between 700 nm and 1 mm) .
  • the image processing method may first perform the dynamic range correction of the infrared sensor.
  • the dynamic range of an infrared sensor can be understood as the ratio of the brightness of the brightest part to the darkest part of the infrared image.
  • Image processing is performed on the degree map.
  • FIG. 1 is a flowchart of an image processing method according to an embodiment of the disclosure. As shown in Figure 1, the image processing method includes:
  • Step S1 Perform image correction and signal-to-noise ratio improvement on the grayscale image.
  • Step S2 Perform contrast stretching on the grayscale image after image correction and signal-to-noise ratio enhancement.
  • the image correction and signal-to-noise ratio improvement in step S1 are performed before the contrast stretching of the grayscale image in step S2, so that the contrast stretching process can more accurately process the effective image information and reduce the image content.
  • the noise interference, so the output image picture is cleaner.
  • the order of the image correction step and the signal-to-noise ratio improvement step on the gray image in the step S1 can be exchanged.
  • image correction may be performed first, for example, to correct the consistency of the response curve of the photosensitive unit of the image sensor or to correct the dead pixels of the image sensor.
  • the signal-to-noise ratio is improved, such as removing the time-domain noise in the grayscale image.
  • the signal-to-noise ratio can be improved first, for example, to remove the time-domain noise in the grayscale image, and on the basis of the grayscale image after the signal-to-noise ratio is improved , And then perform image correction, such as correcting the consistency of the response curve of the photosensitive unit of the image sensor or correcting the dead pixels of the image sensor.
  • image processing method of noise reduction first and image correction the image can retain more edge characteristics and maintain image details.
  • Fig. 2a is a flowchart of an image correction method according to an embodiment of the disclosure. As shown in Figure 2a, in the step S1, performing image correction on the grayscale image includes:
  • Step S101 Perform flat field correction on the grayscale image.
  • the uniformity of the response curve of the image sensor can be corrected by flat-field correction, which can change the slope and offset of the response curve of each photosensitive unit of the image sensor, so as to eliminate the response. Interference with inconsistent curves.
  • a shutter located between the lens and the infrared sensor is used to capture a scene image for flat-field correction, and the infrared sensor is corrected.
  • a memory such as a DDR memory
  • performing multi-frame averaging and then outputting backwards to obtain a pixel-by-pixel offset correction Use the flat field frame to perform flat field correction of the infrared sensor.
  • the flat field correction may also be completed without using the shutter.
  • a blurred image frame can be obtained by accumulating multiple image frames captured by the infrared sensor, or a blurred image frame can be obtained by deliberately defocusing the optical elements or other parts of the infrared sensor, and by processing the blurred image Frame to determine the appropriate correction term to be applied to the captured image frame.
  • a virtual shutter correction program can be started to generate correction items for flat-field correction and so on. It can be understood that the completion of flat-field correction without using a shutter is not limited to the above-mentioned method.
  • Step S102 Perform dead pixel correction on the gray image after flat field correction.
  • dead pixels Due to the influence of factors such as manufacturing process, transportation links, and service life, image sensors have a certain number of bad pixels, and the brightness values of these pixels cannot reflect the captured images, which are usually called dead pixels. In the early stage of image processing, especially after flat-field correction, dead pixel correction can effectively prevent the spread of dead pixels.
  • step S102 performing dead pixel correction on the flat-field corrected grayscale image includes:
  • Static dead point correction includes static bright point correction and static dark point correction.
  • the static bright spot and the static dark spot are calibrated in advance.
  • the camera lens can be blacked out, and the straight-out photo of the specific exposure parameters including all the effective pixels of the image sensor can be taken, and then the software can be used Analyze all pixels, find out the coordinates of the pixels whose brightness is higher than the average value of the surrounding similar pixels by a certain threshold, mark these pixels as static bright spots, and the coordinates of the pixels whose brightness is lower than the average value of the surrounding similar pixels by a certain threshold , Mark these pixels as static dark spots, and save the coordinates of the static bright spots and static dark spots.
  • the infrared sensor When the infrared sensor takes pictures and videos normally, it can be calibrated according to the calibrated static dead point coordinates. Specifically, in the image processing process, the value of the pixel marked as a static dead pixel will not be used, but the value obtained by a certain calculation of the value of the same pixel around it will replace the value of the pixel marked as a bright spot. value.
  • S1022 Detect dynamic dead pixels of the image sensor, and correct pixels corresponding to the dynamic dead pixels.
  • the dynamic dead pixel correction is carried out based on the dead pixels detected online during normal photographing or video recording.
  • dynamic bad pixel correction analyze and compare the difference between each pixel and the surrounding pixels of the same type for one frame of the grayscale. If the surrounding pixel values are more consistent, but the pixel value is quite different, the pixel is marked as bad Point, for this photo or video frame, the value of the pixel will not be used, but the value of the same pixel around it after a certain calculation to replace the value of the pixel marked as a bad point.
  • the dynamic dead pixel correction is calculating the grayscale image that is taken normally, the conditions for finding the location of the dead pixel are not ideal conditions, so the dynamic dead pixel correction is likely to miss the dead pixels or catch some noise points by mistake, and the position of the pixel is also Will not be saved. Dynamic dead point correction can detect bright and dark spots at the same time.
  • the method further includes:
  • the straight-out image of the infrared sensor usually has random stripes.
  • the appearance of random stripes is caused by the non-uniformity of the infrared sensor's focal plane. Due to the limitation of the infrared sensor production process, the non-uniformity of the generation process cannot be avoided. Therefore, the quality of infrared image output can be improved through nonlinear correction in the engineering application process.
  • the non-linear correction generally includes calibration-based non-linear correction and scene-based non-linear correction.
  • the non-linear correction based on calibration can further perform pixel-level responsivity correction based on the pixel-by-pixel responsivity difference of the infrared sensor calibrated in advance, and at the same time, correct the pixel-level offset of the grayscale image, and finally output to the next level.
  • the responsivity and bias of the entire image are kept consistent with the grayscale image.
  • the pixel-level responsivity refers to the output signal voltage generated per unit radiation power.
  • Typical algorithms for non-linear correction based on calibration generally use two-point correction and single-point correction. Scene-based nonlinear correction does not need to be calibrated in advance to obtain reference sources.
  • Typical algorithms generally include high-pass filtering, constant statistics, algebraic correction, Kalman filtering, and neural network algorithms.
  • Fig. 2b is a flowchart of a method for improving a signal-to-noise ratio according to an embodiment of the disclosure. As shown in Figure 2b, the improvement of the signal-to-noise ratio of the grayscale image includes:
  • S103 Perform time-domain noise removal according to the time-domain noise characteristics of the infrared sensor, including at least one of time-domain random single-point noise, time-domain random row noise, or time-domain random column noise.
  • a memory can be used to buffer the image frames before and after denoising, and the similarity and difference between the two frames can be used for filtering to improve the signal-to-noise ratio.
  • the time-domain noise reduction adopts the multi-image averaging method. Since the random noise introduced by the infrared sensor appears as zero-mean additive noise in time, the multi-image averaging method can effectively remove the noise. In the time-domain noise reduction process, by analyzing and calculating multiple frames of images, it is possible to prevent the residual motion in the image.
  • the denoising the grayscale image further includes:
  • the fixed pattern noise includes column fixed pattern noise and row fixed pattern noise.
  • the line-fixed mode noise can appear as periodic horizontal stripes, which is due to the parasitic resistance and capacitance in the analog accumulator circuit. Circuit mismatch will cause uneven brightness of the output image in the scanning direction and periodic attenuation. ; And the column fixed pattern noise appears as vertical stripes with light and dark changes. The reason is that the system structure of the sensor column parallel readout circuit is prone to mismatch between columns due to process deviations, which causes the output image to be in the scanning direction of the TDI The vertical direction) the brightness is not uniform.
  • the step S104 to remove the fixed pattern noise in the grayscale image may be performed after the time domain noise reduction is performed in step S103, so that the image is removed from the motion residue after the time domain noise reduction, which is beneficial to the subsequent accurate finding The fixed pattern noise of the image is removed.
  • the step S104 to remove the fixed pattern noise in the grayscale image may also be performed before the time domain noise reduction is performed in the step S103.
  • spatial noise reduction can be further used to improve image quality. Different from the temporal noise reduction process that uses the temporal correlation of multiple frames of images to reduce noise, spatial noise reduction is to sample a single frame image and use the spatial correlation within the image to reduce noise, that is, use the current pixel and The similarity and difference between the neighborhoods are filtered to improve the signal-to-noise ratio.
  • Low-pass filtering can be used for spatial noise reduction.
  • Low-pass filtering is a method of processing image signals in the frequency domain.
  • the high-frequency component of an image includes random noise of the image, the edge and step part of the image, while the background area belongs to the low-frequency component of the image, and the image is filtered by low-pass filtering.
  • the high-frequency part of the signal can achieve the purpose of removing random noise.
  • this part of the signal is also filtered out after low-pass filtering, resulting in image edges and jumps Part of it appears blurred.
  • a filter with boundary preservation is usually used.
  • the main idea of this filter is to set a threshold according to the noise variance, and only use the neighborhood points that fall within the threshold for low-pass Filtering, when the boundary jump amplitude is greater than the threshold, there will be no blur, and the boundary with the jump amplitude less than the threshold will also be blurred.
  • boundary preservation filtering for noise reduction requires accurate estimation of the variance of the noise to achieve a better noise reduction effect.
  • spatial noise reduction is inevitable The larger the noise, the more the texture is submerged by the noise, and the more serious the blur of the image after noise reduction.
  • FIG. 2c is a flowchart of contrast and detail enhancement according to an embodiment of the disclosure. As shown in Figure 2c, after image correction and signal-to-noise ratio increase and noise reduction are performed in step S1, it further includes:
  • S1' perform spatial frequency separation on the gray image after image correction and signal-to-noise ratio enhancement and noise reduction;
  • the step S1' is executed after the image is corrected and the signal-to-noise ratio is improved in the step S1.
  • the frequency separation in the spatial domain is performed to separate the middle and high frequency components representing the edges and step parts of the image from the low frequencies representing the background area of the image.
  • the components are separated to prepare for the subsequent contrast stretching and detail enhancement to reduce noise and enhance details; after performing the step S1', perform step S2 to perform contrast stretching on the image, and after performing contrast stretching on the image, perform
  • the spatial frequency synthesis is performed on the stretched grayscale image; at the same time, the spatial frequency separated grayscale image obtained in the step S1' can be used to compare the middle and high frequencies in the stretched grayscale image.
  • the frequency component is enhanced, and the details are enhanced by compensating the frequency-separated gray image, so as to output an infrared gray image with enhanced contrast and details.
  • the frequency separation in the spatial domain is carried out before the contrast stretching, the high-frequency part and the low-frequency part in the image are distinguished, and only the low-frequency part of the image is contrast stretched, thus avoiding the random noise contained in the high-frequency part of the image.
  • the image is stretched, causing the image to be blurred; further, the high-frequency part of the image is superimposed with the contrast-stretched image through frequency synthesis, so that the edges and step details contained in the high-frequency part of the image can be used to reduce the contrast. Compensation for the loss of detail in the process of stretching. As a result, it is ensured that the image details are enhanced while the image contrast is enhanced.
  • step S2 performing contrast stretching on the gray image after image correction and signal-to-noise ratio enhancement includes:
  • S202 Perform a second-level contrast stretching, including performing second gray-scale histogram statistics on the gray value of each pixel in the gray-scale image after the first gray-scale transformation, and perform second gray-scale statistics on the pixels according to the statistical results.
  • the degree transformation obtains the pixel gray value after the contrast stretching, wherein the second gray scale transformation is different from the first gray scale transformation.
  • the first grayscale transformation is a linear grayscale transformation or a nonlinear grayscale transformation
  • the second grayscale transformation may also be a linear grayscale transformation or a nonlinear grayscale transformation.
  • the first gray-scale transformation or the second gray-scale transformation are both piecewise linear transformations.
  • contrast stretching takes two-level contrast stretching as an example. In other embodiments, it is also possible to perform more than two-level contrast stretching steps according to the specific conditions of the grayscale image that needs to be contrast-enhanced. Multi-level contrast stretching.
  • the performing contrast stretching on the gray image after the image correction and the signal-to-noise ratio increase further includes:
  • first-level contrast stretching and second-level contrast stretching are respectively responsible for different degrees of stretching.
  • linear stretching is performed first to achieve global contrast stretching.
  • the first-level contrast stretching and the second-level contrast stretching are used to achieve flexible two-level stretching to obtain better contrast.
  • the three stretching processes can be combined, or one, two or more of them can be used.
  • the image processing method may further include:
  • S3 Perform scene analysis on the grayscale image that achieves image enhancement after frequency synthesis. For example, scene analysis is performed to obtain indoor, outdoor, black body, woods, seaside and other scene analysis results, and the scene analysis results are fed back to the previous image correction, noise reduction and/or stretching process to correct image correction, noise reduction and/or stretch The parameters of the stretching process are adjusted so that the entire system forms a feedback system through the feedback of scene analysis, which can adaptively perform appropriate correction, denoising, contrast enhancement and detail enhancement for different scenes.
  • the scene characteristics corresponding to the forest are obtained, and the noise reduction and contrast and detail enhancement processes are fed back according to the scene characteristics, and the noise reduction or image enhancement parameters of the aforementioned process are adjusted , Thereby further improving the quality of the output image.
  • the scene characteristics corresponding to the forest can be pre-stored in the program or stored in the cloud server.
  • the scene analysis is carried out after the image frequency synthesis.
  • the image after frequency synthesis has undergone contrast enhancement and detail enhancement. Therefore, the amount of data corresponding to the image is small, and the information contained is richer. After feedback, the parameters used in the process of noise reduction, contrast enhancement and detail enhancement can be better realized.
  • the image processing method may further include:
  • S4 Perform pseudo-color mapping on the gray image that has achieved image enhancement after frequency synthesis. Mapping the grayscale image to the YUV color map, on the one hand highlights the temperature distribution information, on the other hand highlights the details of the object; among them, the pseudo-color mapping is also performed after the image frequency synthesis, also because the image after the frequency synthesis has undergone contrast enhancement and details Enhanced, so that the information contained is richer, so the pseudo-color mapped image also contains the richest details and better contrast, which is conducive to achieving high-quality image output.
  • the input data of the scene analysis feedback unit should use the data before the pseudo-color mapping, so as to avoid excessive amount of information processed by the scene analysis.
  • the color map of YUV444 can also be transcoded into a color map of YUV422 or 420, and output backward, which is convenient for subsequent encoding and saves storage space.
  • FIG. 3 is a flowchart of an implementation manner of an image processing method according to an embodiment of the disclosure.
  • the problems of dead pixels, obvious fixed pattern noise and random noise, low contrast, low signal-to-noise ratio, and few image details in the grayscale image output by the infrared sensor are solved, thereby obtaining more flaws. Infrared images with less, higher contrast, more detail, and better picture quality.
  • an image processing device is provided.
  • the processing object of the image processing device in this embodiment is a grayscale image output by an infrared sensor as an example.
  • the image processing device may include a signal receiving and control unit.
  • the signal receiving and control unit is used to receive the direct output data of the infrared sensor, control the infrared sensor, and store the infrared direct output image frame in a memory, such as a DDR memory. Before image processing, the signal receiving and control unit can first perform dynamic range correction of the infrared sensor to improve the problem of insufficient dynamic range of the infrared sensor, and then perform image processing on the grayscale image after the dynamic range correction.
  • the image processing device includes an image correction unit, a noise reduction unit, and a stretching unit.
  • the image correction unit is used to correct the gray image
  • the noise reduction unit is used to improve the signal-to-noise ratio of the gray image
  • the stretching unit is used to contrast the gray image after the image correction and the signal-to-noise ratio increase Stretch.
  • Figures 4a-4b show two different implementations of the image processing device.
  • the order of the image correction unit and the noise reduction unit can be exchanged, that is, the order of performing image correction and signal-to-noise ratio improvement on the grayscale image can be exchanged.
  • the image correction of the image correction unit and the improvement of the signal-to-noise ratio of the noise reduction unit are performed before the stretching unit performs contrast stretching on the grayscale image, thereby enabling the contrast stretching process to process effective image information more accurately , Reduce the noise interference in the image, so the output image picture is cleaner.
  • an image correction unit can be used to perform image correction, for example, to correct the consistency of the response curve of the photosensitive unit of the image sensor or to correct the image sensor.
  • the dead pixels are corrected, and on the basis of the corrected gray image, the noise reduction unit is used to improve the signal-to-noise ratio, for example, to remove the time-domain noise in the gray image.
  • the noise reduction unit can also be used to improve the signal-to-noise ratio, such as removing the time domain noise in the grayscale image, and reducing the noise
  • the image correction unit is then used to perform image correction, such as correcting the consistency of the response curve of the photosensitive unit of the image sensor or correcting the dead pixels of the image sensor.
  • the image correction unit further includes a flat field correction unit and a dead pixel correction unit.
  • the flat field correction unit is used to perform flat field correction on the grayscale image.
  • the flat field correction unit corrects the consistency of the response curve of the image sensor, which can change the slope and offset of the response curve of each photosensitive unit, so as to eliminate the interference of the inconsistent response curve.
  • the flat field correction unit when it performs flat field correction on the grayscale image, it uses the shutter to capture a substantially uniform scene image when the shutter is in front of the infrared sensor to correct the infrared sensor. Specifically, by taking the image frame collected when the shutter is closed as a reference grayscale image, using the reference grayscale image as a substantially uniform scene image, storing the image frame in the DDR, performing multi-frame averaging, and outputting backwards to obtain A flat field frame used for pixel-by-pixel offset correction, and the flat field frame is used to perform flat field correction of the infrared sensor. It should be noted that in other embodiments, the flat field correction unit may also complete flat field correction without using a shutter.
  • the dead pixel correction unit is used to perform dead pixel correction on the gray image after flat field correction.
  • the dead pixel correction unit is used to detect that the brightness value of the pixels in the image sensor cannot reflect the dead pixels of the captured image, and perform corresponding processing.
  • dead pixels are divided into two types, namely static dead pixels and dynamic dead pixels: static dead pixels are those that must appear under certain conditions (such as the exposure time is long enough and the image sensor reaches a certain temperature); dynamic dead pixels are Even if a certain condition is met, it is a probability of occurrence of dead pixels (that is, the dead pixels appear in this photo, and may be normal next time).
  • the dead pixel correction unit further includes a static dead pixel correction sub-unit and a dynamic dead pixel correction sub-unit.
  • the static dead pixel correction subunit is used to correct the pixels corresponding to the static dead pixels of the image sensor.
  • Static dead pixel correction is to analyze the position of the dead pixel under ideal test conditions, and the coordinate position of the dead pixel is fixed.
  • the static bright spot and the static dark spot are calibrated in advance, for example, when the non-linear correction unit performs the responsivity correction, the marking of the static dead spot is completed.
  • the infrared sensor takes pictures and videos normally, it can be calibrated according to the calibrated static dead point coordinates.
  • the value of the pixel marked as a static dead pixel will not be used, but the value obtained by a certain calculation of the value of the same pixel around it will replace the value of the pixel marked as a bright spot. value.
  • the dynamic dead pixel correction subunit is used to detect the dynamic dead pixels of the image sensor and correct the pixels corresponding to the dynamic dead pixels.
  • the dynamic dead pixel correction is carried out based on the dead pixels detected online during normal photographing or video recording.
  • dynamic bad pixel correction analyze and compare the difference between each pixel and the surrounding pixels of the same type for one frame of the grayscale. If the surrounding pixel values are more consistent, but the pixel value is quite different, the pixel is marked as bad Point, for this photo or video frame, the value of the pixel will not be used, but the value of the same pixel around it after a certain calculation to replace the value of the pixel marked as bad.
  • the dynamic dead pixel correction is calculating the grayscale image that is taken normally, the conditions for finding the location of the dead pixel are not ideal conditions, so the dynamic dead pixel correction is likely to miss the dead pixels or catch some noise points by mistake, and the position of the pixel is also Will not be saved. Dynamic dead point correction can detect bright and dark spots at the same time.
  • the image correction unit further includes a non-linear correction unit, which is used to perform non-linear correction on the flat-field corrected grayscale image.
  • a non-linear correction unit which is used to perform non-linear correction on the flat-field corrected grayscale image.
  • the noise reduction unit is used to improve the signal-to-noise ratio of the grayscale image, including a time-domain noise reduction unit.
  • the time-domain noise reduction unit is used to remove time-domain noise according to the time-domain noise characteristics of the infrared sensor, including time-domain random single-point noise, time-domain random row noise, or time-domain random column noise.
  • a DDR memory can be used to buffer the image frames before and after denoising, and the similarity and difference between the two frames can be used for filtering to improve the signal-to-noise ratio.
  • the time-domain noise reduction unit adopts the multi-image averaging method. Since the random noise introduced by the infrared sensor appears as zero-mean additive noise in time, the multi-image averaging method can effectively remove the noise. In the time-domain noise reduction process, by analyzing and calculating multiple frames of images, it is possible to prevent the residual motion in the image.
  • the noise reduction unit further includes a fixed pattern noise removing unit for removing fixed pattern noise in the grayscale image.
  • the fixed pattern noise includes column fixed pattern noise and row fixed pattern noise.
  • the fixed-pattern noise removal unit removing the fixed-pattern noise in the grayscale image may be performed after the time-domain noise reduction unit performs the time-domain noise reduction.
  • the fixed-pattern noise removal unit removing the fixed-pattern noise in the grayscale image may be performed before the time-domain noise reduction unit performs the time-domain noise reduction.
  • the time-domain noise reduction may be performed by the noise reduction unit first, and then the fixed-pattern noise removal unit may perform the fixed-pattern noise removal, so that after the image has undergone time-domain noise reduction, the motion residue is removed, which is beneficial to the subsequent accurate finding of image fixation. Pattern noise is removed.
  • the noise reduction unit further includes a spatial noise reduction unit for removing random noise in the spatial domain.
  • the spatial denoising unit samples a single frame image and uses the spatial correlation within the image to reduce noise, that is, using the current pixel
  • the similarity and difference between the neighborhood and the neighborhood are filtered to improve the signal-to-noise ratio.
  • the spatial noise reduction unit may include a low-pass filter. Filtering out the high-frequency part of the image signal by a low-pass filter can achieve the purpose of removing random noise. However, since the edge part and step part of the image are also in the high-frequency area, this part of the signal is also filtered out after the low-pass filter. , Resulting in blurring of the edges and jumps of the image.
  • a filter with boundary preservation is usually used.
  • the main idea of this filter is to set a threshold according to the noise variance, and only use the neighborhood points that fall within the threshold for low-pass Filtering, when the boundary jump amplitude is greater than the threshold, there will be no blur, and the boundary with the jump amplitude less than the threshold will also be blurred.
  • boundary preservation filtering for noise reduction requires accurate estimation of the variance of the noise to achieve a better noise reduction effect.
  • spatial noise reduction is inevitable The larger the noise, the more the texture is submerged by the noise, and the more serious the blur of the image after noise reduction.
  • the spatial noise reduction unit needs to be combined with The time-domain noise reduction unit cooperates, and uses the correlation of the time domain to use the information of multiple frames to compensate each other, so as to achieve a better noise reduction effect.
  • the image processing device further includes a frequency separation unit and a frequency synthesis unit, wherein the frequency separation unit is used to perform spatial frequency separation on the gray image after image correction and signal-to-noise ratio enhancement and noise reduction; the frequency synthesis unit is used for Perform spatial frequency synthesis on the stretched grayscale image, and use the frequency-separated grayscale image to enhance the middle and high frequency components in the stretched grayscale image.
  • the frequency separation unit is used to perform spatial frequency separation on the gray image after image correction and signal-to-noise ratio enhancement and noise reduction
  • the frequency synthesis unit is used for Perform spatial frequency synthesis on the stretched grayscale image, and use the frequency-separated grayscale image to enhance the middle and high frequency components in the stretched grayscale image.
  • the frequency separation unit performs frequency separation in the spatial domain after the image is corrected and the signal-to-noise ratio is improved, and separates the middle and high frequency components representing the edges and step parts of the image from the low frequency components representing the background area of the image, as Preparing for post-contrast stretching and detail enhancement to reduce noise and improve details; after that, the stretching unit performs contrast stretching on the image, and after the image is contrast stretched, the frequency synthesis unit performs spatial domain on the stretched grayscale image
  • the spatial frequency separated gray image obtained by the frequency separation unit can be used to enhance the medium and high frequency components in the stretched gray image, and the details can be improved by compensating the frequency separated gray image. In this way, the infrared gray image with enhanced contrast and details is output.
  • the frequency separation unit Since the frequency separation unit performs spatial frequency separation before the contrast stretching, it distinguishes the high frequency part from the low frequency part in the image.
  • the stretching unit only performs contrast stretching on the low frequency part of the image, thereby avoiding the high frequency in the image. Part of the random noise contained in the image is stretched at the same time, causing the image to be blurred; further, the high-frequency part of the image is superimposed with the contrast-stretched image through the frequency synthesis unit, so that the edges and jumps contained in the high-frequency part of the image can be used.
  • the stretching unit includes a first contrast stretching unit and a second contrast stretching unit.
  • the first contrast stretching unit is used to perform the first-level contrast stretching, including performing first grayscale histogram statistics on the grayscale value of each pixel in the frequency-separated grayscale image, and performing the statistics according to the statistical results.
  • the pixel performs the first gray scale transformation to obtain the pixel gray value after contrast stretching;
  • the second contrast stretching unit is used to perform the second-level contrast stretching, including performing second gray-scale histogram statistics on the gray value of each pixel in the gray-scale image after the first gray-scale transformation, and according to the statistical result Performing a second grayscale transformation on the pixel to obtain a pixel grayscale value after contrast stretching, wherein the second grayscale transformation is different from the first grayscale transformation.
  • the first grayscale transformation is a linear grayscale transformation or a nonlinear grayscale transformation
  • the second grayscale transformation may also be a linear grayscale transformation or a nonlinear grayscale transformation.
  • the first gray-scale transformation or the second gray-scale transformation are both piecewise linear transformations.
  • the above-mentioned contrast stretching takes a two-level contrast stretching unit as an example.
  • the grayscale image for contrast enhancement can also be performed according to the specific situation of the contrast enhancement, and it can also include more than two levels of contrast stretching. Unit to achieve multi-level contrast stretching.
  • the stretching unit further includes a linear stretching unit for performing a linear transformation on the gray value of the grayscale image after frequency separation before the statistics of the first grayscale histogram, and performing a preliminary linear stretching. Stretching to prepare for the subsequent first-level contrast stretching and second-level contrast stretching.
  • a linear stretching unit for performing a linear transformation on the gray value of the grayscale image after frequency separation before the statistics of the first grayscale histogram, and performing a preliminary linear stretching. Stretching to prepare for the subsequent first-level contrast stretching and second-level contrast stretching.
  • an image whose gray value of all pixels is originally only in a certain part of the grayscale area can be grayscale transformed by linear stretching, and stretched to the entire grayscale space of 0-255, so that the contrast of the image can be greatly improved. Enhanced.
  • the above-mentioned preliminary linear stretching unit, the first contrast stretching unit, and the second contrast stretching unit are respectively responsible for different degrees of stretching.
  • the linear stretching unit first performs global contrast stretching.
  • the first-stage contrast stretching unit and the second-stage contrast stretching unit are used to realize flexible two-stage stretching and obtain better contrast. According to different image processing needs, the three stretching processes can be combined, or one, two or more of them can be used.
  • the image processing device further includes a scene analysis feedback unit.
  • the scene analysis unit is used to perform scene analysis on the grayscale images that have realized image enhancement after frequency synthesis, such as indoor, outdoor, black body, woods, seaside, etc., and feed back the results of the scene analysis to the previous image correction, noise reduction and/or
  • the parameters of the image correction, noise reduction and/or stretching process are adjusted, so that the entire system forms a feedback system through the feedback of scene analysis, which can adaptively perform appropriate correction, denoising, and contrast for different scenes. Enhancement and detail enhancement.
  • the scene analysis result obtained by the scene analysis process is a forest
  • the scene characteristics corresponding to the forest are obtained, and the noise reduction and contrast and detail enhancement processes are fed back according to the scene characteristics, and the noise reduction or image enhancement parameters of the aforementioned process are adjusted , Thereby further improving the quality of the output image.
  • the scene characteristics corresponding to the forest can be pre-stored in the program or stored in the cloud server, and the scene analysis unit can execute the call of the scene characteristics.
  • the scene analysis of the scene analysis feedback unit is performed after the image frequency synthesis. Since the frequency synthesized image has undergone contrast enhancement and detail enhancement, the information contained in it is richer. After the information contained in it is fed back, it can be Better realize the adjustment of the parameters used in the process of noise reduction, contrast enhancement and detail enhancement.
  • the image processing device further includes a pseudo-color mapping unit for mapping the grayscale image that has achieved image enhancement after frequency synthesis into a YUV color image. Therefore, on the one hand, the temperature distribution information is highlighted, on the other hand, the details of the object are highlighted; among them, the pseudo-color mapping is also performed after the image frequency synthesis, also because the image after frequency synthesis has undergone contrast enhancement and detail enhancement, so that the information contained Richer, so the pseudo-color mapping image also contains the richest details and better contrast, which is conducive to achieving high-quality image output.
  • a pseudo-color mapping unit for mapping the grayscale image that has achieved image enhancement after frequency synthesis into a YUV color image. Therefore, on the one hand, the temperature distribution information is highlighted, on the other hand, the details of the object are highlighted; among them, the pseudo-color mapping is also performed after the image frequency synthesis, also because the image after frequency synthesis has undergone contrast enhancement and detail enhancement, so that the information contained Richer, so the pseudo-color mapping image also contains the richest details and
  • the input data of the scene analysis feedback unit should adopt the data before the pseudo-color mapping, so as to avoid the excessive amount of information processed by the scene analysis.
  • a transcoding unit may also be used to transcode the color map of YUV444 into a color map of YUV422 or 420, and output it backward, which is convenient for subsequent encoding and saves storage space.
  • the image processing device may include any type of data processing device with data processing capabilities, such as but not limited to CPU, DSP, FPGA, CPLD, etc.
  • the problems of dead pixels, obvious fixed pattern noise and random noise, low contrast, low signal-to-noise ratio, and less image details in the grayscale image output by the infrared sensor are solved, thereby obtaining more flaws. Infrared images with less, higher contrast, more detail, and better picture quality.
  • a third exemplary embodiment of the present disclosure provides an image processing device, including a reading storage medium and one or more processors.
  • the readable storage medium is used to store executable instructions; one or more processors are used to execute the executable instructions to execute the image processing method described in the first embodiment.
  • FIG. 5 is a schematic structural diagram of an imaging device according to an embodiment of the disclosure. As shown in FIG. 5, the imaging device includes an image sensor and an image processing device, the image sensor is connected to the image processing device, and the image processing device adopts the image processing device described in the second embodiment.
  • FIG. 6 is a schematic structural diagram of a movable carrier according to an embodiment of the disclosure.
  • the movable carrier includes: a body and an imaging device, the imaging device is installed in the body, the imaging device includes an image sensor and an image processing device, the image sensor is connected to the An image processing device, the image processing device adopts the image processing device described in the second embodiment.
  • the movable carrier may be an unmanned aerial vehicle, an unmanned vehicle, an unmanned ship, and the like.
  • the sixth embodiment of the present disclosure provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor realizes the same as described in the first embodiment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)

Abstract

An image processing method, comprising: performing image correction and signal-to-noise ratio improvement on a grayscale map; and performing contrast stretching on the grayscale map after image correction and signal-to-noise ratio improvement, wherein the image correction comprises: performing a flat field correction on the grayscale map; and performing bad point correction on the flat field-corrected grayscale map; and the signal-to-noise ratio improvement comprises: performing time domain noise removal on the grayscale map.

Description

一种图像处理方法、装置、成像设备及可移动载体Image processing method, device, imaging equipment and movable carrier 技术领域Technical field
本公开涉及图像处理技术领域,尤其涉及一种图像处理方法、装置、成像设备及可移动载体。The present disclosure relates to the field of image processing technology, and in particular to an image processing method, device, imaging equipment, and a movable carrier.
背景技术Background technique
在图像处理领域,由于图像传感器采集的图像通常会存在一些瑕疵,例如现有的红外图像处理装置,由于红外传感器本身制造过程的原因,出图存在坏点明显、噪声不能去除干净等问题,且红外传感器直出的灰度图像的对比度及细节呈现也并不理想,因此,需要改进图像处理过程才能展现出场景物体的细节,得到高画质的图像输出。In the field of image processing, the images collected by the image sensor usually have some flaws. For example, in the existing infrared image processing device, due to the manufacturing process of the infrared sensor itself, there are problems such as obvious dead pixels and noise that cannot be removed cleanly. The contrast and details of the grayscale image directly output by the infrared sensor are not ideal. Therefore, the image processing process needs to be improved to show the details of the scene objects and obtain high-quality image output.
公开内容Public content
本公开提供了一种图像处理方法,包括对灰度图进行图像校正及信噪比提升;对图像校正及信噪比提升后的灰度图进行对比度拉伸;其中,所述图像校正包括:对灰度图进行平场校正;以及对平场校正后的灰度图进行坏点校正;所述信噪比提升包括:对灰度图进行时域噪声的去除。The present disclosure provides an image processing method, which includes performing image correction and signal-to-noise ratio enhancement on a grayscale image; performing contrast stretching on the grayscale image after image correction and signal-to-noise ratio enhancement; wherein the image correction includes: Perform flat-field correction on the grayscale image; and perform dead pixel correction on the grayscale image after the flat-field correction; the signal-to-noise ratio improvement includes: removing temporal noise on the grayscale image.
本公开还提供了一种图像处理装置,包括:校正单元,用于灰度图进行图像校正;降噪单元,用于对灰度图进行信噪比提升;以及拉伸单元,用于对所述图像校正及信噪比提升后的灰度图进行对比度拉伸;其中,所述校正单元包括:平场校正单元,用于对灰度图进行平场校正;坏点校正单元,用于对平场校正后的灰度图进行坏点校正;所述降噪单元包括:时域降噪单元,用于对灰度图进行时域噪声的去除。The present disclosure also provides an image processing device, including: a correction unit for performing image correction on a grayscale image; a noise reduction unit for improving a signal-to-noise ratio of the grayscale image; and a stretching unit for performing image correction on the grayscale image. The grayscale image after image correction and signal-to-noise ratio enhancement is contrast stretched; wherein, the correction unit includes: a flat field correction unit for performing flat field correction on the grayscale image; a dead pixel correction unit for performing flat field correction on the grayscale image; The flat-field corrected gray image is corrected for dead pixels; the noise reduction unit includes: a time-domain noise reduction unit for removing time-domain noise on the gray image.
本公开还提供了一种成像设备,所述成像设备包括图像传感器以及图像处理装置,所述图像传感器连接至所述图像处理装置,所述图像处理装置包括:校正单元,用于灰度图进行图像校正;降噪单元,用于对灰度图进行信噪比提升;以及拉伸单元,用于对所述图像校正及信噪比提升后的灰度图进行对比度拉伸;其中,所述校正单元包括:平场校正单元,用于对灰度图进行平场校正;坏点校正单元,用于对平场校正后的灰度图进行坏点校正;所述降噪单元包括:时域降噪单元,用于对灰度图进行时域噪声的去除。The present disclosure also provides an imaging device. The imaging device includes an image sensor and an image processing device. The image sensor is connected to the image processing device. The image processing device includes: a correction unit for grayscale image processing. Image correction; a noise reduction unit for improving the signal-to-noise ratio of the grayscale image; and a stretching unit for performing contrast stretching on the grayscale image after the image correction and the signal-to-noise ratio increase; wherein, the The correction unit includes: a flat-field correction unit, which is used to perform flat-field correction on a grayscale image; a dead pixel correction unit, which is used to perform dead-point correction on a grayscale image after flat-field correction; and the noise reduction unit includes: time domain The noise reduction unit is used to remove the time domain noise of the gray image.
本公开还提供了一种可移动载体,包括:机身和成像设备,所述成像设备安装于所述机身,所述成像设备包括图像传感器以及图像处理装置,所述图像传感器连接至所述图像处理装置,所述图像处理装置包括:校正单元,用于灰度图进行图像校正;降噪单元,用于对灰度图进行信噪比提升;以及拉伸单元,用于对所述图像校正及信噪比提升后的灰度图进行对比度拉伸;其中,所述校正单元包括:平场校正单元,用于对灰度图进行平场校正;坏点校正单元,用于对平场校正后的灰度图进行坏点校正;所述降噪单元包括:时域降噪单元,用于对灰度图进行时域噪声的去除。The present disclosure also provides a movable carrier, including: a body and an imaging device, the imaging device is installed in the body, the imaging device includes an image sensor and an image processing device, the image sensor is connected to the An image processing device, the image processing device comprising: a correction unit for performing image correction on the grayscale image; a noise reduction unit for improving the signal-to-noise ratio of the grayscale image; and a stretching unit for performing image correction on the image The grayscale image after the correction and signal-to-noise ratio increase is contrast-stretched; wherein, the correction unit includes: a flat field correction unit for performing flat field correction on the grayscale image; a dead pixel correction unit for performing flat field correction on the grayscale image; The corrected gray image is corrected for dead pixels; the noise reduction unit includes: a time domain noise reduction unit for removing time domain noise on the gray image.
本公开图像处理方法及装置,解决了图像传感器输出的灰度图存在的坏点、明显固定模式噪声和随机噪声、对比度低、信噪比低、图像细节少等问题,从而得到了瑕疵更少、对比度更高、细节更多、画质更好的图像。The image processing method and device of the present disclosure solve the problems of dead pixels, obvious fixed pattern noise and random noise, low contrast, low signal-to-noise ratio, and few image details in the grayscale image output by the image sensor, resulting in fewer defects , Images with higher contrast, more detail, and better quality.
附图说明Description of the drawings
附图是用来提供对本公开的进一步理解,并且构成说明书的一部分,与下面的具体实施方式一起用于解释本公开,但并不构成对本公开的限制。在附图中:The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure, but do not constitute a limitation to the present disclosure. In the attached picture:
图1为本公开实施例图像处理方法的流程图。FIG. 1 is a flowchart of an image processing method according to an embodiment of the disclosure.
图2a为本公开实施例图像校正方法的流程图。Fig. 2a is a flowchart of an image correction method according to an embodiment of the disclosure.
图2b为本公开实施例信噪比提升方法的流程图。Fig. 2b is a flowchart of a method for improving a signal-to-noise ratio according to an embodiment of the disclosure.
图2c为本公开实施例对比度及细节增强的流程图。FIG. 2c is a flowchart of contrast and detail enhancement according to an embodiment of the disclosure.
图3为本公开图像处理方法一具体实施方式的流程图。FIG. 3 is a flowchart of a specific implementation of the image processing method of the present disclosure.
图4a本公开一实施例图像处理装置的示意图。Fig. 4a is a schematic diagram of an image processing device according to an embodiment of the present disclosure.
图4b为本公开另一实施例图像处理装置的示意图。Fig. 4b is a schematic diagram of an image processing apparatus according to another embodiment of the present disclosure.
图5为本公开实施例成像设备的结构示意图。FIG. 5 is a schematic structural diagram of an imaging device according to an embodiment of the disclosure.
图6为本公开实施例可移动载体的结构示意图。FIG. 6 is a schematic structural diagram of a movable carrier according to an embodiment of the disclosure.
具体实施方式Detailed ways
本公开提供了一种图像处理方法、装置、成像设备及可移动载体。本公开的图像处理方法、装置、成像设备及可移动载体能够克服现有的图像处理装置的出图存在坏点明显、固定模式噪声、随机横条纹、对比度低、细节不突出等问题,提供高画质的图像输出。The present disclosure provides an image processing method, device, imaging equipment and movable carrier. The image processing method, device, imaging device, and movable carrier of the present disclosure can overcome the problems of obvious dead pixels, fixed pattern noise, random horizontal stripes, low contrast, and inconspicuous details in the output of the existing image processing device, and provide high High-quality image output.
下面将结合实施例和实施例中的附图,对本公开技术方案进行清楚、 完整的描述。显然,所描述的实施例仅仅是本公开一部分实施例,而不是全部的实施例。基于本公开中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本公开保护的范围。The technical solutions of the present disclosure will be clearly and completely described below in conjunction with the embodiments and the drawings in the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present disclosure.
在本公开第一个示意性实施例中,提供了一种图像处理方法。本实施例所述图像处理方法的处理对象以红外传感器输出的灰度图为例,其中,红外传感器可以被配置为基于红外辐射(例如,具有700nm和1mm之间的波长)来检测和形成图像。In the first exemplary embodiment of the present disclosure, an image processing method is provided. The processing object of the image processing method in this embodiment is an example of a grayscale image output by an infrared sensor, where the infrared sensor can be configured to detect and form an image based on infrared radiation (for example, having a wavelength between 700 nm and 1 mm) .
本实施例中,图像处理方法可以先进行红外传感器的动态范围校正。红外传感器的动态范围可以理解为红外图像中最明亮处与最黑暗处的亮度之比。在进行图像处理时可以先将接收到的红外传感器的直出数据进行存储,进行红外传感器的动态范围检查,并针对动态范围检查结果进行红外传感器的动态范围校正,再对动态范围校正后的灰度图进行图像处理。In this embodiment, the image processing method may first perform the dynamic range correction of the infrared sensor. The dynamic range of an infrared sensor can be understood as the ratio of the brightness of the brightest part to the darkest part of the infrared image. When performing image processing, you can first store the received direct output data of the infrared sensor, check the dynamic range of the infrared sensor, and perform the dynamic range correction of the infrared sensor according to the dynamic range check result, and then correct the gray after the dynamic range correction. Image processing is performed on the degree map.
以下对本实施例图像处理方法进行详细说明。The image processing method of this embodiment will be described in detail below.
图1为本公开实施例图像处理方法的流程图。如图1所示,所述图像处理方法包括:FIG. 1 is a flowchart of an image processing method according to an embodiment of the disclosure. As shown in Figure 1, the image processing method includes:
步骤S1,对灰度图进行图像校正及信噪比提升。Step S1: Perform image correction and signal-to-noise ratio improvement on the grayscale image.
步骤S2,对图像校正及信噪比提升后的灰度图进行对比度拉伸。Step S2: Perform contrast stretching on the grayscale image after image correction and signal-to-noise ratio enhancement.
其中,所述步骤S1的图像校正和信噪比提升都在步骤S2对灰度图进行对比度拉伸之前执行,由此,使得对比度拉伸过程能够更准确地处理有效的图像信息,降低图像中的噪声干扰,因而输出的图像画面更加干净。其中,所述步骤S1中对灰度图进行图像校正步骤以及信噪比提升步骤的顺序可以调换。Wherein, the image correction and signal-to-noise ratio improvement in step S1 are performed before the contrast stretching of the grayscale image in step S2, so that the contrast stretching process can more accurately process the effective image information and reduce the image content. The noise interference, so the output image picture is cleaner. Wherein, the order of the image correction step and the signal-to-noise ratio improvement step on the gray image in the step S1 can be exchanged.
作为一个实施方式,对于红外传感器输出的灰度图,可以先进行图像校正,例如对图像传感器的感光单元的响应曲线的一致性进行校正或对图像传感器的坏点进行校正,在校正后的灰度图的基础上,再进行信噪比提升,例如去除灰度图中的时域噪声。通过上述先图像校正后降噪的图像处理方式,可以使得后续图像处理过程得到更加准确的图像信息。As an embodiment, for the grayscale output of the infrared sensor, image correction may be performed first, for example, to correct the consistency of the response curve of the photosensitive unit of the image sensor or to correct the dead pixels of the image sensor. On the basis of the degree map, the signal-to-noise ratio is improved, such as removing the time-domain noise in the grayscale image. Through the above-mentioned image processing method of first image correction and then noise reduction, it is possible to obtain more accurate image information in the subsequent image processing process.
作为另一种实施方式,对于红外传感器输出的灰度图,还可以先进行信噪比提升,例如去除灰度图中的时域噪声,并在信噪比提升后的灰度图的基础上,再进行图像校正,例如对图像传感器的感光单元的响应曲线的 一致性进行校正或对图像传感器的坏点进行校正。通过上述先降噪后图像校正的图像处理方式,可以使图像保留更多的边缘特性,保持图像细节。As another embodiment, for the grayscale image output by the infrared sensor, the signal-to-noise ratio can be improved first, for example, to remove the time-domain noise in the grayscale image, and on the basis of the grayscale image after the signal-to-noise ratio is improved , And then perform image correction, such as correcting the consistency of the response curve of the photosensitive unit of the image sensor or correcting the dead pixels of the image sensor. Through the above-mentioned image processing method of noise reduction first and image correction, the image can retain more edge characteristics and maintain image details.
图2a为本公开实施例图像校正方法的流程图。如图2a所示,所述步骤S1中,对灰度图进行图像校正包括:Fig. 2a is a flowchart of an image correction method according to an embodiment of the disclosure. As shown in Figure 2a, in the step S1, performing image correction on the grayscale image includes:
步骤S101,对所述灰度图进行平场校正。Step S101: Perform flat field correction on the grayscale image.
由于图像传感器的感光单元的响应不是严格一致,通过平场校正对图像传感器的响应曲线的一致性进行校正,能够改变图像传感器每个感光单元的响应曲线的斜率及偏移,从而用以排除响应曲线不一致的干扰。Since the response of the photosensitive unit of the image sensor is not strictly consistent, the uniformity of the response curve of the image sensor can be corrected by flat-field correction, which can change the slope and offset of the response curve of each photosensitive unit of the image sensor, so as to eliminate the response. Interference with inconsistent curves.
本实施例中,在对所述灰度图进行平场校正时,利用位于镜头和红外传感器之间的快门捕获用于平场校正的场景图像,对红外传感器进行校正。具体地,通过将快门闭合时采集的图像帧作为参考灰度图,将该参考灰度图存入存储器(例如DDR存储器),进行多帧平均后向后输出,得到用于逐像素偏置校正的平场帧,利用该平场帧进行红外传感器的平场校正。In this embodiment, when performing flat-field correction on the grayscale image, a shutter located between the lens and the infrared sensor is used to capture a scene image for flat-field correction, and the infrared sensor is corrected. Specifically, by taking the image frame collected when the shutter is closed as a reference grayscale image, storing the reference grayscale image in a memory (such as a DDR memory), performing multi-frame averaging and then outputting backwards to obtain a pixel-by-pixel offset correction Use the flat field frame to perform flat field correction of the infrared sensor.
需要说明的是,在其他实施例中,还可以不使用快门完成平场校正。例如,可通过对红外传感器捕获的多个图像帧进行累加来获得模糊的图像帧,或可通过有意地使红外传感器的光学元件或者其他部件散焦来获得模糊的图像帧,通过处理模糊的图像帧,以确定要应用于捕获的图像帧的合适的校正项。或者,在适当的触发事件和/或条件被检测到可以表明充当快门(例如,虚拟快门)的物体或场景存在时,可以启动虚拟快门校正程序,以产生校正项来进行平场校正等等。可以理解的是,所述不使用快门完成平场校正并并不以上述方法为限。It should be noted that in other embodiments, the flat field correction may also be completed without using the shutter. For example, a blurred image frame can be obtained by accumulating multiple image frames captured by the infrared sensor, or a blurred image frame can be obtained by deliberately defocusing the optical elements or other parts of the infrared sensor, and by processing the blurred image Frame to determine the appropriate correction term to be applied to the captured image frame. Alternatively, when an appropriate trigger event and/or condition is detected, which can indicate the existence of an object or scene acting as a shutter (for example, a virtual shutter), a virtual shutter correction program can be started to generate correction items for flat-field correction and so on. It can be understood that the completion of flat-field correction without using a shutter is not limited to the above-mentioned method.
步骤S102,对平场校正后的灰度图进行坏点校正。Step S102: Perform dead pixel correction on the gray image after flat field correction.
由于制造工艺、运输环节、使用寿命等因素的影响,图像传感器均存在一定数量的坏的像素,该些像素的亮度值不能反映所拍摄的图像,通常称之为坏点。在图像处理的初期,尤其是平场校正后进行坏点矫正,能够有效防止坏点扩散。Due to the influence of factors such as manufacturing process, transportation links, and service life, image sensors have a certain number of bad pixels, and the brightness values of these pixels cannot reflect the captured images, which are usually called dead pixels. In the early stage of image processing, especially after flat-field correction, dead pixel correction can effectively prevent the spread of dead pixels.
坏点按出现概率分成两种,即静态坏点和动态坏点。针对上述两种类型的坏点,所述步骤S102中,对平场校正后的灰度图进行坏点校正包括:According to the probability of occurrence, dead pixels are divided into two types, namely static dead pixels and dynamic dead pixels. Regarding the above two types of dead pixels, in step S102, performing dead pixel correction on the flat-field corrected grayscale image includes:
S1021,对所述图像传感器的静态坏点对应的像素进行校正。S1021: Correct the pixels corresponding to the static dead pixels of the image sensor.
静态坏点校正又包括静态亮点校正和静态暗点校正。其中,静态亮点 与静态暗点都是提前标定好的,在提前进行静态坏点标定时,可以将相机镜头遮黑,拍摄特定曝光参数的包括图像传感器全部有效像素的直出照片,然后通过软件分析所有像素,找出其中亮度比周围同类像素的平均值高出一定阈值的像素的坐标,将该些像素标记为静态亮点,以及亮度比周围同类像素的平均值低出一定阈值的像素的坐标,将这些像素标记为静态暗点,并将静态亮点与静态暗点的坐标存下来。Static dead point correction includes static bright point correction and static dark point correction. Among them, the static bright spot and the static dark spot are calibrated in advance. When the static dead spot is calibrated in advance, the camera lens can be blacked out, and the straight-out photo of the specific exposure parameters including all the effective pixels of the image sensor can be taken, and then the software can be used Analyze all pixels, find out the coordinates of the pixels whose brightness is higher than the average value of the surrounding similar pixels by a certain threshold, mark these pixels as static bright spots, and the coordinates of the pixels whose brightness is lower than the average value of the surrounding similar pixels by a certain threshold , Mark these pixels as static dark spots, and save the coordinates of the static bright spots and static dark spots.
在红外传感器正常拍照和录像时,可以根据标定好的静态坏点坐标进行校正。具体地,在图像处理过程中,被标记为静态坏点的像素的值不会被采用,而是用其周围同类像素的值经过一定计算而得出的值来代替被标记为亮点的像素的值。When the infrared sensor takes pictures and videos normally, it can be calibrated according to the calibrated static dead point coordinates. Specifically, in the image processing process, the value of the pixel marked as a static dead pixel will not be used, but the value obtained by a certain calculation of the value of the same pixel around it will replace the value of the pixel marked as a bright spot. value.
S1022,检测所述图像传感器的动态坏点,并对所述动态坏点对应的像素进行校正。S1022: Detect dynamic dead pixels of the image sensor, and correct pixels corresponding to the dynamic dead pixels.
动态坏点校正是在正常拍照或录像时根据在线检测出来的坏点进行的。进行动态坏点校正时,针对灰度图的一帧,分析比较每个像素同周围同类像素的差值,如果周围像素值较一致,而该像素值差别较大,则把该像素标记为坏点,对于此照片或视频帧,该像素的值不会被采用,而是用其周围同类像素的值经过一定计算而得出的值代替被标记为坏点的像素的值。由于动态坏点校正是在对正常拍摄的灰度图进行计算,查找坏点位置的条件不是理想条件,所以动态坏点校正很可能漏抓坏点或误抓一些噪声点,该像素的位置也不会被存下来。动态坏点校正能同时探测亮点和暗点。The dynamic dead pixel correction is carried out based on the dead pixels detected online during normal photographing or video recording. When performing dynamic bad pixel correction, analyze and compare the difference between each pixel and the surrounding pixels of the same type for one frame of the grayscale. If the surrounding pixel values are more consistent, but the pixel value is quite different, the pixel is marked as bad Point, for this photo or video frame, the value of the pixel will not be used, but the value of the same pixel around it after a certain calculation to replace the value of the pixel marked as a bad point. Because the dynamic dead pixel correction is calculating the grayscale image that is taken normally, the conditions for finding the location of the dead pixel are not ideal conditions, so the dynamic dead pixel correction is likely to miss the dead pixels or catch some noise points by mistake, and the position of the pixel is also Will not be saved. Dynamic dead point correction can detect bright and dark spots at the same time.
在一些实施例中,所述步骤S1中的图像校正过程中,所述步骤101之后,所述步骤102之前,还包括:In some embodiments, in the image correction process in the step S1, after the step 101 and before the step 102, the method further includes:
S101’,对平场校正后的灰度图进行非线性校正。S101', non-linear correction is performed on the gray image after flat field correction.
红外传感器的直出图像中通常会带有随机条纹,随机条纹的出现是由于红外传感器焦平面的非均匀性产生的,由于红外传感器生产工艺的限制,生成过程中无法避免其非均匀性的产生,因此,在工程应用过程中可以通过非线性校正提升红外图像输出质量。其中,所述非线性校正一般包括基于标定的非线性校正以及基于场景的非线性校正。The straight-out image of the infrared sensor usually has random stripes. The appearance of random stripes is caused by the non-uniformity of the infrared sensor's focal plane. Due to the limitation of the infrared sensor production process, the non-uniformity of the generation process cannot be avoided. Therefore, the quality of infrared image output can be improved through nonlinear correction in the engineering application process. Wherein, the non-linear correction generally includes calibration-based non-linear correction and scene-based non-linear correction.
具体地,基于标定的非线性校正可以根据提前标定好的红外传感器逐 像素的响应率差异,进一步进行像素级响应率校正,同时,校正灰度图像素级的偏置,最终向后一级输出整个图像的响应率和偏置保持一致的灰度图。其中,像素级的响应率是指单位辐射功率产生的输出信号电压。基于标定的非线性校正的典型算法一般采用两点校正和单点校正。基于场景的非线性校正不需要进行提前标定获取参考源,其典型算法一般包括高通滤波、恒定统计法、代数校正法、卡尔曼滤波法及神经网络算法等。Specifically, the non-linear correction based on calibration can further perform pixel-level responsivity correction based on the pixel-by-pixel responsivity difference of the infrared sensor calibrated in advance, and at the same time, correct the pixel-level offset of the grayscale image, and finally output to the next level. The responsivity and bias of the entire image are kept consistent with the grayscale image. Among them, the pixel-level responsivity refers to the output signal voltage generated per unit radiation power. Typical algorithms for non-linear correction based on calibration generally use two-point correction and single-point correction. Scene-based nonlinear correction does not need to be calibrated in advance to obtain reference sources. Typical algorithms generally include high-pass filtering, constant statistics, algebraic correction, Kalman filtering, and neural network algorithms.
通过对红外图像进行非线性校正,可以有效去除图像中的不规则条纹,得到更加清晰的输出图像,提升红外传感器的成像质量。By performing nonlinear correction on the infrared image, irregular stripes in the image can be effectively removed, a clearer output image can be obtained, and the imaging quality of the infrared sensor can be improved.
所述步骤S1中,除了对灰度图进行图像校正的步骤,还包括对灰度图进行信噪比提升的步骤。图2b为本公开实施例信噪比提升方法的流程图。如图2b所示,对灰度图进行信噪比提升包括:In the step S1, in addition to the step of performing image correction on the gray image, it also includes the step of improving the signal-to-noise ratio of the gray image. Fig. 2b is a flowchart of a method for improving a signal-to-noise ratio according to an embodiment of the disclosure. As shown in Figure 2b, the improvement of the signal-to-noise ratio of the grayscale image includes:
S103,根据红外传感器的时域噪声特性,进行时域噪声的去除,包括时域随机单点噪声、时域随机行噪声或时域随机列噪声中的至少一种。具体地,可以采用存储器缓存去噪前后的图像帧,利用两帧之间的相似性和差异性进行滤波,提升信噪比。S103: Perform time-domain noise removal according to the time-domain noise characteristics of the infrared sensor, including at least one of time-domain random single-point noise, time-domain random row noise, or time-domain random column noise. Specifically, a memory can be used to buffer the image frames before and after denoising, and the similarity and difference between the two frames can be used for filtering to improve the signal-to-noise ratio.
一般的,时域降噪采用多图像平均法,由于红外传感器引入的随机噪声在时间上表现为零均值的加性噪声,所以采用多图象平均法能有效去除噪声。时域降噪过程中,通过对多帧图像进行分析和运算,能够防止图像中的运动残留。Generally, the time-domain noise reduction adopts the multi-image averaging method. Since the random noise introduced by the infrared sensor appears as zero-mean additive noise in time, the multi-image averaging method can effectively remove the noise. In the time-domain noise reduction process, by analyzing and calculating multiple frames of images, it is possible to prevent the residual motion in the image.
在一些实施例中,所述对灰度图进行降噪还包括:In some embodiments, the denoising the grayscale image further includes:
S104,去除灰度图中的固定模式噪声。S104: Remove fixed pattern noise in the grayscale image.
红外传感器通常具有的灵敏度的偏差、电路的增益及偏移等装置固有的偏差,因此红外直出图像一般会存在固定模式噪声干扰。其中,所述固定模式噪声包括列固定模式噪声和行固定模式噪声。示例性的,行固定模式噪声可以表现为周期性横条纹,其是由于模拟累加器电路中存在寄生电阻和电容,电路失配会导致输出图像在扫描方向亮度不均匀,且呈周期性衰减导致;而列固定模式噪声表现为明暗变化的竖条纹,其产生原因是传感器列并行读出电路的系统结构由于工艺偏差容易出现列与列之间的失配,从而导致输出图像在与TDI扫描方向垂直的方向)亮度不均匀。Infrared sensors usually have inherent deviations such as sensitivity deviations, circuit gains and offsets, so the infrared direct image generally has fixed pattern noise interference. Wherein, the fixed pattern noise includes column fixed pattern noise and row fixed pattern noise. Exemplarily, the line-fixed mode noise can appear as periodic horizontal stripes, which is due to the parasitic resistance and capacitance in the analog accumulator circuit. Circuit mismatch will cause uneven brightness of the output image in the scanning direction and periodic attenuation. ; And the column fixed pattern noise appears as vertical stripes with light and dark changes. The reason is that the system structure of the sensor column parallel readout circuit is prone to mismatch between columns due to process deviations, which causes the output image to be in the scanning direction of the TDI The vertical direction) the brightness is not uniform.
作为一种实施方式,所述步骤S104去除灰度图中的固定模式噪声可 以在步骤S103进行时域降噪之后进行,从而图像经过时域降噪后去除了运动残留,有利于后续准确找出图像的固定模式噪声进行去除。作为另一种实施方式,所述步骤S104去除灰度图中的固定模式噪声还可以在步骤S103进行时域降噪之前进行。As an implementation manner, the step S104 to remove the fixed pattern noise in the grayscale image may be performed after the time domain noise reduction is performed in step S103, so that the image is removed from the motion residue after the time domain noise reduction, which is beneficial to the subsequent accurate finding The fixed pattern noise of the image is removed. As another implementation manner, the step S104 to remove the fixed pattern noise in the grayscale image may also be performed before the time domain noise reduction is performed in the step S103.
S105,去除灰度图中的空域随机噪声。S105, removing random noise in the spatial domain in the grayscale image.
在进行时域降噪之后,由于图像中局部区域运动情况不尽相同,经过时域降噪后各区域噪声消减的程度也不尽相同,而运动区域通常比静止区域有更大的噪声残余,因此可以进一步使用空域降噪来提升图像的质量。与时域降噪过程利用多帧图像在时间上的相关性进行降噪不同的是,空域降噪是针对单帧图像进行采样,利用图像内的空间相关性进行降噪,即利用当前像素和邻域之间的相似性和差异性,进行滤波提升信噪比。After the time domain noise reduction is performed, since the motion of the local area in the image is not the same, the degree of noise reduction in each area after the time domain noise reduction is also different, and the moving area usually has a larger noise residue than the static area. Therefore, spatial noise reduction can be further used to improve image quality. Different from the temporal noise reduction process that uses the temporal correlation of multiple frames of images to reduce noise, spatial noise reduction is to sample a single frame image and use the spatial correlation within the image to reduce noise, that is, use the current pixel and The similarity and difference between the neighborhoods are filtered to improve the signal-to-noise ratio.
空域降噪可采用低通滤波。低通滤波法是一种在频域上对图像信号进行处理的方法。在分析图像信号的频率特性时,一幅图像的高频率分量部分包括图像的随机噪声、图像的边缘和跃阶部分,而背景区域属于图像的低频率分量部分,通过低通滤波法滤除图像信号的高频部分可以达到去除随机噪声的目的,但是,由于图像的边缘部分和跃阶部分也处于高频区域,在低通滤波后该部分信号也被滤掉了,从而导致图像边缘和跳跃部分出现模糊。Low-pass filtering can be used for spatial noise reduction. Low-pass filtering is a method of processing image signals in the frequency domain. When analyzing the frequency characteristics of an image signal, the high-frequency component of an image includes random noise of the image, the edge and step part of the image, while the background area belongs to the low-frequency component of the image, and the image is filtered by low-pass filtering. The high-frequency part of the signal can achieve the purpose of removing random noise. However, since the edge part and step part of the image are also in the high-frequency area, this part of the signal is also filtered out after low-pass filtering, resulting in image edges and jumps Part of it appears blurred.
可以理解的是,由于空域降噪的降噪过程会损失更多的细节,因此,如果需要获取红外传感器的测温数据,需要采用进行空域降噪之前的数据,例如去除灰度图中的固定模式噪声后的数据(特别是用于绝对温度测量的测温数据)。It is understandable that the noise reduction process of spatial noise reduction will lose more details. Therefore, if you need to obtain the temperature measurement data of the infrared sensor, you need to use the data before the spatial noise reduction, such as removing the fixed in the grayscale image. Data after mode noise (especially temperature measurement data for absolute temperature measurement).
为缓解空域降噪采用低通滤波产生的模糊,通常使用带边界保留的滤波器,该滤波器主要思想是根据噪声方差设定一个域值,仅使用落在阈值内的邻域点做低通滤波,当边界跳跃幅度大于该阈值时,不会出现模糊,而对跳跃幅度小于该域值的边界同样会造成模糊。使用边界保留滤波进行降噪需要准确的估计噪声的方差才能达到较好的降噪效果,但是,无论是采用低通滤波降噪法还是采用带边界保留的降噪法,空域降噪都不可避免的会对图像造成一定的模糊,噪声越大,被噪声淹没的纹理也越多,降噪后图像的模糊也越严重,仅依赖空域相关性难以得到满意效果,因此,空 域降噪需要与时域降噪进行配合,利用时域的相关性使用多帧的信息相互补偿,从而达到更优的降噪效果。In order to alleviate the blur caused by low-pass filtering in spatial noise reduction, a filter with boundary preservation is usually used. The main idea of this filter is to set a threshold according to the noise variance, and only use the neighborhood points that fall within the threshold for low-pass Filtering, when the boundary jump amplitude is greater than the threshold, there will be no blur, and the boundary with the jump amplitude less than the threshold will also be blurred. The use of boundary preservation filtering for noise reduction requires accurate estimation of the variance of the noise to achieve a better noise reduction effect. However, whether it is a low-pass filter noise reduction method or a noise reduction method with boundary preservation, spatial noise reduction is inevitable The larger the noise, the more the texture is submerged by the noise, and the more serious the blur of the image after noise reduction. It is difficult to obtain satisfactory results only by relying on spatial correlation. Therefore, spatial noise reduction needs to be time-dependent. Cooperate with domain noise reduction, and use the correlation of the time domain to use the information of multiple frames to compensate each other, so as to achieve a better noise reduction effect.
图2c为本公开实施例对比度及细节增强的流程图。如图2c所示,所述步骤S1对图像校正及信噪比提升降噪后,还包括:FIG. 2c is a flowchart of contrast and detail enhancement according to an embodiment of the disclosure. As shown in Figure 2c, after image correction and signal-to-noise ratio increase and noise reduction are performed in step S1, it further includes:
S1’,对图像校正及信噪比提升降噪后灰度图进行空域的频率分离;S1', perform spatial frequency separation on the gray image after image correction and signal-to-noise ratio enhancement and noise reduction;
S2’,对拉伸后的灰度图进行空域的频率合成,并利用所述频率分离后的灰度图对拉伸后的灰度图中的中高频分量进行增强。S2', performing spatial frequency synthesis on the stretched grayscale image, and using the frequency-separated grayscale image to enhance the medium and high frequency components in the stretched grayscale image.
所述步骤S1’在所述步骤S1对图像进行校正及信噪比提升之后执行,通过进行空域的频率分离,将表示图像的边缘和跃阶部分的中高频率分量与表示图像背景区域的低频率分量进行分离,为后级对比度拉伸和细节增强做准备,降低噪声,提升细节;在执行所述步骤S1’之后执行步骤S2,对图像进行对比度拉伸,对图像进行对比度拉伸后,执行所述步骤S2’,对拉伸后的灰度图进行空域的频率合成;同时,可以采用所述步骤S1’获取的空域频率分离后的灰度图对拉伸后的灰度图中的中高频分量进行增强,通过补偿频率分离后的灰度图来提升细节,从而输出对比度和细节都增强后的红外灰度图。The step S1' is executed after the image is corrected and the signal-to-noise ratio is improved in the step S1. The frequency separation in the spatial domain is performed to separate the middle and high frequency components representing the edges and step parts of the image from the low frequencies representing the background area of the image. The components are separated to prepare for the subsequent contrast stretching and detail enhancement to reduce noise and enhance details; after performing the step S1', perform step S2 to perform contrast stretching on the image, and after performing contrast stretching on the image, perform In the step S2', the spatial frequency synthesis is performed on the stretched grayscale image; at the same time, the spatial frequency separated grayscale image obtained in the step S1' can be used to compare the middle and high frequencies in the stretched grayscale image. The frequency component is enhanced, and the details are enhanced by compensating the frequency-separated gray image, so as to output an infrared gray image with enhanced contrast and details.
由于在对比度拉伸之前进行了空域的频率分离,将图像中的高频部分与低频部分进行了区分,仅对图像的低频部分进行对比度拉伸,从而避免了图像中高频部分包含的随机噪声同时被拉伸,造成图像的模糊;进一步的,通过频率合成将图像的高频部分与对比度拉伸后的图像进行叠加,从而能够采用图像中高频部分包含的边缘和跃阶部分细节,对对比度拉伸过程中产生的细节损失进行补偿。由此,保证在图像对比度增强的同时,实现了图像细节的增强。Since the frequency separation in the spatial domain is carried out before the contrast stretching, the high-frequency part and the low-frequency part in the image are distinguished, and only the low-frequency part of the image is contrast stretched, thus avoiding the random noise contained in the high-frequency part of the image. The image is stretched, causing the image to be blurred; further, the high-frequency part of the image is superimposed with the contrast-stretched image through frequency synthesis, so that the edges and step details contained in the high-frequency part of the image can be used to reduce the contrast. Compensation for the loss of detail in the process of stretching. As a result, it is ensured that the image details are enhanced while the image contrast is enhanced.
所述步骤S2中,对图像校正及信噪比提升后的灰度图进行对比度拉伸包括:In the step S2, performing contrast stretching on the gray image after image correction and signal-to-noise ratio enhancement includes:
S201,执行第一级对比度拉伸,包括对频率分离后的灰度图中的每个像素的灰度值进行第一灰度直方图统计,并根据统计结果对像素做第一灰度变换得到对比度拉伸后的像素灰度值;S201. Perform the first-level contrast stretching, including performing first gray-level histogram statistics on the gray-level value of each pixel in the gray-level image after frequency separation, and performing the first gray-level transformation on the pixels according to the statistical results. Pixel gray value after contrast stretching;
S202,执行第二级对比度拉伸,包括对第一灰度变换后的灰度图中的每个像素的灰度值进行第二灰度直方图统计,并根据统计结果对像素做第 二灰度变换得到对比度拉伸后的像素灰度值,其中,所述第二灰度变换不同于所述第一灰度变换。S202. Perform a second-level contrast stretching, including performing second gray-scale histogram statistics on the gray value of each pixel in the gray-scale image after the first gray-scale transformation, and perform second gray-scale statistics on the pixels according to the statistical results. The degree transformation obtains the pixel gray value after the contrast stretching, wherein the second gray scale transformation is different from the first gray scale transformation.
具体地,所述第一灰度变换为线性灰度变换或非线性灰度变换;所述第二灰度变换也可以为线性灰度变换或非线性灰度变换。示例性地,所述第一灰度变换或第二灰度变换均为分段线性变换,在进行第一灰度变换时,对图像的第一有用数据的对比度进行增强;在进行第二灰度变换时,对图像的第二有用数据的对比度进行增强。通过上述方法,使得图像的灰度值可以更好地在直方图上分布,达到图像增强效果。Specifically, the first grayscale transformation is a linear grayscale transformation or a nonlinear grayscale transformation; the second grayscale transformation may also be a linear grayscale transformation or a nonlinear grayscale transformation. Exemplarily, the first gray-scale transformation or the second gray-scale transformation are both piecewise linear transformations. When the first gray-scale transformation is performed, the contrast of the first useful data of the image is enhanced; and the second gray-scale transformation is performed. During the degree transformation, the contrast of the second useful data of the image is enhanced. Through the above method, the gray value of the image can be better distributed on the histogram to achieve the image enhancement effect.
可以理解的是,上述对比度拉伸以两级对比度拉伸为例,在其他实施例中,还可以根据需要进行对比度增强的灰度图的具体情况,执行两级以上的对比度拉伸步骤,实现多级对比度拉伸。It is understandable that the above-mentioned contrast stretching takes two-level contrast stretching as an example. In other embodiments, it is also possible to perform more than two-level contrast stretching steps according to the specific conditions of the grayscale image that needs to be contrast-enhanced. Multi-level contrast stretching.
优选地,所述步骤S2中,所述对图像校正及信噪比提升后的灰度图进行对比度拉伸还包括:Preferably, in the step S2, the performing contrast stretching on the gray image after the image correction and the signal-to-noise ratio increase further includes:
S200,在所述第一灰度直方图统计之前,对频率分离后的灰度图的灰度值进行线性变换,进行初步的线性拉伸,为后续第一级对比度拉伸及第二级对比度拉伸做准备。示例性的,可以通过线性拉伸将原本所有像素灰度值仅处于某一部分灰度区域的图像进行灰度变换,拉伸至0-255的整个灰度空间,从而图像的对比度能够得到大幅度增强。S200, before the statistics of the first grayscale histogram, linearly transform the grayscale values of the frequency-separated grayscale images, and perform preliminary linear stretching, which is the subsequent first-level contrast stretching and second-level contrast Get ready for stretching. Exemplarily, an image whose gray value of all pixels is originally only in a certain part of the grayscale area can be grayscale transformed by linear stretching, and stretched to the entire grayscale space of 0-255, so that the contrast of the image can be greatly improved. Enhanced.
可以理解的是,上述初步线性拉伸、第一级对比度拉伸及第二级对比度拉伸分别负责不同程度的拉伸,示例性的,线性拉伸最先执行,用于实现全局对比度拉伸,第一级对比度拉伸及第二级对比度拉伸用于实现灵活的两级拉伸,得到较好的对比度。根据不同的图像处理需要,三个拉伸过程可以合并,也可以使用其中一个、两个或多个。It is understandable that the above-mentioned preliminary linear stretching, first-level contrast stretching, and second-level contrast stretching are respectively responsible for different degrees of stretching. Illustratively, linear stretching is performed first to achieve global contrast stretching. , The first-level contrast stretching and the second-level contrast stretching are used to achieve flexible two-level stretching to obtain better contrast. According to different image processing needs, the three stretching processes can be combined, or one, two or more of them can be used.
在一些实施例中,所述图像处理方法还可以包括:In some embodiments, the image processing method may further include:
S3,针对频率合成后实现了图像增强的灰度图,进行场景分析。例如进行场景分析获得室内、室外、黑体、树林、海边等场景分析结果,并将场景分析结果反馈到前面的图像校正、降噪和/或拉伸过程,对图像校正、降噪和/或拉伸过程的参数进行调整,使得整个系统通过场景分析的反馈构成一个反馈系统,能自适应地针对不同场景进行合适的校正、去噪、对比度增强和细节增强。示例性的,所述场景分析过程得到的场景分析结果为 树林,则获取树林对应的场景特性,根据该场景特性反馈至降噪及对比度和细节增强过程,调整上述过程的降噪或图像增强参数,从而进一步提高输出图像的质量。其中,树林对应的场景特性可以预存储在程序中,或保存在云端服务器。S3: Perform scene analysis on the grayscale image that achieves image enhancement after frequency synthesis. For example, scene analysis is performed to obtain indoor, outdoor, black body, woods, seaside and other scene analysis results, and the scene analysis results are fed back to the previous image correction, noise reduction and/or stretching process to correct image correction, noise reduction and/or stretch The parameters of the stretching process are adjusted so that the entire system forms a feedback system through the feedback of scene analysis, which can adaptively perform appropriate correction, denoising, contrast enhancement and detail enhancement for different scenes. Exemplarily, if the scene analysis result obtained by the scene analysis process is a forest, the scene characteristics corresponding to the forest are obtained, and the noise reduction and contrast and detail enhancement processes are fed back according to the scene characteristics, and the noise reduction or image enhancement parameters of the aforementioned process are adjusted , Thereby further improving the quality of the output image. Among them, the scene characteristics corresponding to the forest can be pre-stored in the program or stored in the cloud server.
其中,场景分析在图像频率合成之后进行,由于频率合成后的图像经过了对比度增强和细节增强,因此,其图像所对应的数据量小的同时,所包含的信息更加丰富,将其包含的信息进行反馈后,能够更好地实现对于降噪、对比度增强和细节增强过程中所用参数的调整。Among them, the scene analysis is carried out after the image frequency synthesis. The image after frequency synthesis has undergone contrast enhancement and detail enhancement. Therefore, the amount of data corresponding to the image is small, and the information contained is richer. After feedback, the parameters used in the process of noise reduction, contrast enhancement and detail enhancement can be better realized.
此外,所述图像处理方法还可以包括:In addition, the image processing method may further include:
S4,对频率合成后实现了图像增强的灰度图,进行伪彩映射。将灰度图映射为YUV色彩图,一方面凸显温度分布信息,一方面凸显物体的细节;其中,伪彩映射也在图像频率合成之后进行,也是由于频率合成后的图像经过了对比度增强和细节增强,从而所包含的信息更加丰富,因此经过伪彩映射的图也含有最丰富的细节和更好的对比度,有利于实现高画质的图像输出。S4: Perform pseudo-color mapping on the gray image that has achieved image enhancement after frequency synthesis. Mapping the grayscale image to the YUV color map, on the one hand highlights the temperature distribution information, on the other hand highlights the details of the object; among them, the pseudo-color mapping is also performed after the image frequency synthesis, also because the image after the frequency synthesis has undergone contrast enhancement and details Enhanced, so that the information contained is richer, so the pseudo-color mapped image also contains the richest details and better contrast, which is conducive to achieving high-quality image output.
需要说明的是,当同时进行场景分析时,所述场景分析反馈单元的输入数据应当采用伪彩映射之前的数据,避免场景分析处理的信息量过大。It should be noted that when scene analysis is performed at the same time, the input data of the scene analysis feedback unit should use the data before the pseudo-color mapping, so as to avoid excessive amount of information processed by the scene analysis.
具体地,所述YUV色彩图为YUV444类型的色彩图时,还可以将YUV444的色彩图转码为YUV422或420的色彩图,向后输出,便于后续的编码,节约存储空间。Specifically, when the YUV color map is a color map of YUV444 type, the color map of YUV444 can also be transcoded into a color map of YUV422 or 420, and output backward, which is convenient for subsequent encoding and saves storage space.
图3为本公开实施例图像处理方法的一实施方式的流程图。通过本实施例的图像处理方法,解决了红外传感器输出的灰度图存在的坏点、明显固定模式噪声和随机噪声、对比度低、信噪比低、图像细节少等问题,从而得到了瑕疵更少、对比度更高、细节更多、画质更好的红外图像。FIG. 3 is a flowchart of an implementation manner of an image processing method according to an embodiment of the disclosure. Through the image processing method of this embodiment, the problems of dead pixels, obvious fixed pattern noise and random noise, low contrast, low signal-to-noise ratio, and few image details in the grayscale image output by the infrared sensor are solved, thereby obtaining more flaws. Infrared images with less, higher contrast, more detail, and better picture quality.
在本公开第二个示意性实施例中,提供了一种图像处理装置。本实施例所述图像处理装置的处理对象以红外传感器输出的灰度图为例。In a second exemplary embodiment of the present disclosure, an image processing device is provided. The processing object of the image processing device in this embodiment is a grayscale image output by an infrared sensor as an example.
所述图像处理装置可以包括信号接收及控制单元。信号接收及控制单元用于接收红外传感器的直出数据,并对红外传感器进行控制,以及将红外直出图像帧存入存储器中,例如DDR存储器。在进行图像处理之前,信号接收及控制单元可以先进行红外传感器的动态范围校正,改善红外传 感器动态范围不足的问题,再对动态范围校正后的灰度图进行图像处理。The image processing device may include a signal receiving and control unit. The signal receiving and control unit is used to receive the direct output data of the infrared sensor, control the infrared sensor, and store the infrared direct output image frame in a memory, such as a DDR memory. Before image processing, the signal receiving and control unit can first perform dynamic range correction of the infrared sensor to improve the problem of insufficient dynamic range of the infrared sensor, and then perform image processing on the grayscale image after the dynamic range correction.
以下对本实施例图像处理装置进行详细说明。The image processing apparatus of this embodiment will be described in detail below.
图4a-图4b为本公开实施例图像处理装置的示意图。如图4a-图4b所示,所述图像处理装置包括图像校正单元、降噪单元及拉伸单元。其中,图像校正单元用于对灰度图进行图像校正;降噪单元用于对灰度图进行信噪比提升;拉伸单元用于对图像校正及信噪比提升后的灰度图进行对比度拉伸。4a-4b are schematic diagrams of an image processing device according to an embodiment of the disclosure. As shown in FIGS. 4a-4b, the image processing device includes an image correction unit, a noise reduction unit, and a stretching unit. Among them, the image correction unit is used to correct the gray image; the noise reduction unit is used to improve the signal-to-noise ratio of the gray image; the stretching unit is used to contrast the gray image after the image correction and the signal-to-noise ratio increase Stretch.
图4a-图4b所示为图像处理装置两种不同的实现方式。其中,所述图像校正单元与降噪单元的顺序可以的调换,即对灰度图进行图像校正以及信噪比提升的顺序可以调换。所述图像校正单元的图像校正和降噪单元的信噪比提升都在拉伸单元对灰度图进行对比度拉伸之前执行,由此,使得对比度拉伸过程能够更准确地处理有效的图像信息,降低图像中的噪声干扰,因而输出的图像画面更加干净。Figures 4a-4b show two different implementations of the image processing device. Wherein, the order of the image correction unit and the noise reduction unit can be exchanged, that is, the order of performing image correction and signal-to-noise ratio improvement on the grayscale image can be exchanged. The image correction of the image correction unit and the improvement of the signal-to-noise ratio of the noise reduction unit are performed before the stretching unit performs contrast stretching on the grayscale image, thereby enabling the contrast stretching process to process effective image information more accurately , Reduce the noise interference in the image, so the output image picture is cleaner.
如图4a所示,作为一种实施方式,对于红外传感器输出的灰度图,可以先采用图像校正单元进行图像校正,例如对图像传感器的感光单元的响应曲线的一致性进行校正或对图像传感器的坏点进行校正,在校正后的灰度图的基础上,再采用降噪单元进行信噪比提升,例如去除灰度图中的时域噪声。通过上述先图像校正后降噪的图像处理方式,可以使得后续图像处理过程得到更加准确的图像信息。As shown in Figure 4a, as an implementation manner, for the grayscale output of the infrared sensor, an image correction unit can be used to perform image correction, for example, to correct the consistency of the response curve of the photosensitive unit of the image sensor or to correct the image sensor. The dead pixels are corrected, and on the basis of the corrected gray image, the noise reduction unit is used to improve the signal-to-noise ratio, for example, to remove the time-domain noise in the gray image. Through the above-mentioned image processing method of first image correction and then noise reduction, it is possible to obtain more accurate image information in the subsequent image processing process.
如图4b所示,作为另一种实施方式,对于红外传感器输出的灰度图,还可以先采用降噪单元进行信噪比提升,例如去除灰度图中的时域噪声,并在降噪后的灰度图的基础上,再采用图像校正单元进行图像校正,例如对图像传感器的感光单元的响应曲线的一致性进行校正或对图像传感器的坏点进行校正。通过上述先降噪后图像校正的图像处理方式,可以使图像保留更多的边缘特性,保持图像细节。As shown in Figure 4b, as another implementation manner, for the grayscale image output by the infrared sensor, the noise reduction unit can also be used to improve the signal-to-noise ratio, such as removing the time domain noise in the grayscale image, and reducing the noise On the basis of the latter grayscale image, the image correction unit is then used to perform image correction, such as correcting the consistency of the response curve of the photosensitive unit of the image sensor or correcting the dead pixels of the image sensor. Through the above-mentioned image processing method of noise reduction first and image correction, the image can retain more edge characteristics and maintain image details.
具体而言,所述图像校正单元进一步包括平场校正单元及坏点校正单元。Specifically, the image correction unit further includes a flat field correction unit and a dead pixel correction unit.
其中,平场校正单元用于对灰度图进行平场校正。平场校正单元对图像传感器的响应曲线的一致性进行校正,其能够改变每个感光单元的响应曲线的斜率及偏移,从而用以排除响应曲线不一致的干扰。Among them, the flat field correction unit is used to perform flat field correction on the grayscale image. The flat field correction unit corrects the consistency of the response curve of the image sensor, which can change the slope and offset of the response curve of each photosensitive unit, so as to eliminate the interference of the inconsistent response curve.
本实施例中,平场校正单元在对所述灰度图进行平场校正时,利用快门位于红外传感器的前面时捕获基本上均匀的场景图像,对红外传感器进行校正。具体地,通过将快门闭合时采集的图像帧作为参考灰度图,以该参考灰度图作为基本上均匀的场景图像,将该图像帧存入DDR,进行多帧平均后向后输出,得到用于逐像素偏置校正的平场帧,利用该平场帧进行红外传感器的平场校正。需要说明的是,在其他实施例中,平场校正单元还可以不使用快门完成平场校正。In this embodiment, when the flat field correction unit performs flat field correction on the grayscale image, it uses the shutter to capture a substantially uniform scene image when the shutter is in front of the infrared sensor to correct the infrared sensor. Specifically, by taking the image frame collected when the shutter is closed as a reference grayscale image, using the reference grayscale image as a substantially uniform scene image, storing the image frame in the DDR, performing multi-frame averaging, and outputting backwards to obtain A flat field frame used for pixel-by-pixel offset correction, and the flat field frame is used to perform flat field correction of the infrared sensor. It should be noted that in other embodiments, the flat field correction unit may also complete flat field correction without using a shutter.
坏点校正单元用于对平场校正后的灰度图进行坏点校正。坏点校正单元用于检测出图像传感器中像素的亮度值不能反映所拍摄的图像的坏点,并做相应的处理。The dead pixel correction unit is used to perform dead pixel correction on the gray image after flat field correction. The dead pixel correction unit is used to detect that the brightness value of the pixels in the image sensor cannot reflect the dead pixels of the captured image, and perform corresponding processing.
坏点按出现概率分成两种,即静态坏点和动态坏点:静态坏点是在一定条件下(如曝光时间足够长、图像传感器达到一定温度)必定会出现的坏点;动态坏点是即使达到一定条件也是概率性出现的坏点(即该坏点在这次拍照出现,下次拍照可能正常)。According to the probability of occurrence, dead pixels are divided into two types, namely static dead pixels and dynamic dead pixels: static dead pixels are those that must appear under certain conditions (such as the exposure time is long enough and the image sensor reaches a certain temperature); dynamic dead pixels are Even if a certain condition is met, it is a probability of occurrence of dead pixels (that is, the dead pixels appear in this photo, and may be normal next time).
针对上述两种类型的坏点,所述坏点校正单元还包括静态坏点校正子单元与动态坏点校正子单元。For the above two types of dead pixels, the dead pixel correction unit further includes a static dead pixel correction sub-unit and a dynamic dead pixel correction sub-unit.
其中,静态坏点校正子单元用于对所述图像传感器的静态坏点对应的像素进行校正。静态坏点校正是在理想的测试条件下来分析坏点位置,坏点的坐标位置是固定的。其中,静态亮点与静态暗点都是提前标定好的,例如,在非线性校正单元进行响应率校正时,完成静态坏点的标记。在红外传感器正常拍照和录像时,可以根据标定好的静态坏点坐标进行校正。具体地,在图像处理过程中,被标记为静态坏点的像素的值不会被采用,而是用其周围同类像素的值经过一定计算而得出的值来代替被标记为亮点的像素的值。Wherein, the static dead pixel correction subunit is used to correct the pixels corresponding to the static dead pixels of the image sensor. Static dead pixel correction is to analyze the position of the dead pixel under ideal test conditions, and the coordinate position of the dead pixel is fixed. Among them, the static bright spot and the static dark spot are calibrated in advance, for example, when the non-linear correction unit performs the responsivity correction, the marking of the static dead spot is completed. When the infrared sensor takes pictures and videos normally, it can be calibrated according to the calibrated static dead point coordinates. Specifically, in the image processing process, the value of the pixel marked as a static dead pixel will not be used, but the value obtained by a certain calculation of the value of the same pixel around it will replace the value of the pixel marked as a bright spot. value.
动态坏点校正子单元用于检测所述图像传感器的动态坏点,并对所述动态坏点对应的像素进行校正。动态坏点校正是在正常拍照或录像时根据在线检测出来的坏点进行的。进行动态坏点校正时,针对灰度图的一帧,分析比较每个像素同周围同类像素的差值,如果周围像素值较一致,而该像素值差别较大,则把该像素标记为坏点,对于此照片或视频帧,该像素的值不会被采用,而是用其周围同类像素的值经过一定计算而得出的值代 替被标记为坏点的像素的值。由于动态坏点校正是在对正常拍摄的灰度图进行计算,查找坏点位置的条件不是理想条件,所以动态坏点校正很可能漏抓坏点或误抓一些噪声点,该像素的位置也不会被存下来。动态坏点校正能同时探测亮点和暗点。The dynamic dead pixel correction subunit is used to detect the dynamic dead pixels of the image sensor and correct the pixels corresponding to the dynamic dead pixels. The dynamic dead pixel correction is carried out based on the dead pixels detected online during normal photographing or video recording. When performing dynamic bad pixel correction, analyze and compare the difference between each pixel and the surrounding pixels of the same type for one frame of the grayscale. If the surrounding pixel values are more consistent, but the pixel value is quite different, the pixel is marked as bad Point, for this photo or video frame, the value of the pixel will not be used, but the value of the same pixel around it after a certain calculation to replace the value of the pixel marked as bad. Because the dynamic dead pixel correction is calculating the grayscale image that is taken normally, the conditions for finding the location of the dead pixel are not ideal conditions, so the dynamic dead pixel correction is likely to miss the dead pixels or catch some noise points by mistake, and the position of the pixel is also Will not be saved. Dynamic dead point correction can detect bright and dark spots at the same time.
在一些实施例中,图像校正单元还包括非线性校正单元,用于对平场校正后的灰度图进行非线性校正。通过对红外图像进行非线性校正,可以有效去除图像中的不规则条纹,得到更加清晰的输出图像,提升红外传感器的成像质量。In some embodiments, the image correction unit further includes a non-linear correction unit, which is used to perform non-linear correction on the flat-field corrected grayscale image. By performing nonlinear correction on the infrared image, irregular stripes in the image can be effectively removed, a clearer output image can be obtained, and the imaging quality of the infrared sensor can be improved.
降噪单元用于对灰度图进行信噪比提升,包括时域降噪单元。时域降噪单元用于根据红外传感器的时域噪声特性,进行时域噪声的去除,包括时域随机单点噪声、时域随机行噪声或时域随机列噪声。具体地,可以采用DDR存储器缓存去噪前后的图像帧,利用两帧之间的相似性和差异性进行滤波,提升信噪比。The noise reduction unit is used to improve the signal-to-noise ratio of the grayscale image, including a time-domain noise reduction unit. The time-domain noise reduction unit is used to remove time-domain noise according to the time-domain noise characteristics of the infrared sensor, including time-domain random single-point noise, time-domain random row noise, or time-domain random column noise. Specifically, a DDR memory can be used to buffer the image frames before and after denoising, and the similarity and difference between the two frames can be used for filtering to improve the signal-to-noise ratio.
一般的,时域降噪单元采用多图像平均法,由于红外传感器引入的随机噪声在时间上表现为零均值的加性噪声,所以采用多图象平均法能有效去除噪声。时域降噪过程中,通过对多帧图像进行分析和运算,能够防止图像中的运动残留。Generally, the time-domain noise reduction unit adopts the multi-image averaging method. Since the random noise introduced by the infrared sensor appears as zero-mean additive noise in time, the multi-image averaging method can effectively remove the noise. In the time-domain noise reduction process, by analyzing and calculating multiple frames of images, it is possible to prevent the residual motion in the image.
在一些实施例中,所述降噪单元还包括固定模式噪声去除单元,用于去除灰度图中的固定模式噪声。其中,所述固定模式噪声包括列固定模式噪声和行固定模式噪声。In some embodiments, the noise reduction unit further includes a fixed pattern noise removing unit for removing fixed pattern noise in the grayscale image. Wherein, the fixed pattern noise includes column fixed pattern noise and row fixed pattern noise.
作为一种实施方式,所述固定模式噪声去除单元去除灰度图中的固定模式噪声可以在时域降噪单元进行时域降噪之后进行。作为另一种实施方式,所述固定模式噪声去除单元去除灰度图中的固定模式噪声可以在时域降噪单元进行时域降噪之前进行。作为优选地,可以先由降噪单元执行时域降噪再由固定模式噪声去除单元执行固定模式噪声去除,从而图像经过时域降噪后去除了运动残留,有利于后续准确找出图像的固定模式噪声进行去除。As an implementation manner, the fixed-pattern noise removal unit removing the fixed-pattern noise in the grayscale image may be performed after the time-domain noise reduction unit performs the time-domain noise reduction. As another implementation manner, the fixed-pattern noise removal unit removing the fixed-pattern noise in the grayscale image may be performed before the time-domain noise reduction unit performs the time-domain noise reduction. Preferably, the time-domain noise reduction may be performed by the noise reduction unit first, and then the fixed-pattern noise removal unit may perform the fixed-pattern noise removal, so that after the image has undergone time-domain noise reduction, the motion residue is removed, which is beneficial to the subsequent accurate finding of image fixation. Pattern noise is removed.
在一些实施例中,所述降噪单元还包括空域降噪单元,用于去除空域随机噪声。In some embodiments, the noise reduction unit further includes a spatial noise reduction unit for removing random noise in the spatial domain.
在时域降噪第一进行时域降噪之后,由于图像中局部区域运动情况不 尽相同,经过时域降噪后各区域噪声消减的程度也不尽相同,而运动区域通常比静止区域有更大的噪声残余,因此可以进一步使用空域降噪单元来提升图像的质量。与时域降噪对应利用多帧图像在时间上的相关性进行降噪不同的是,空域降噪单元是针对单帧图像进行采样,利用图像内的空间相关性进行降噪,即利用当前像素和邻域之间的相似性和差异性,进行滤波提升信噪比。After the temporal denoising is first performed in the temporal denoising, since the motion of the local area in the image is not the same, the degree of noise reduction in each area after the temporal denoising is not the same, and the moving area is usually more than the static area. Larger noise residue, so the spatial noise reduction unit can be further used to improve the quality of the image. Corresponding to time domain noise reduction, the difference is that the spatial denoising unit samples a single frame image and uses the spatial correlation within the image to reduce noise, that is, using the current pixel The similarity and difference between the neighborhood and the neighborhood are filtered to improve the signal-to-noise ratio.
空域降噪单元可以包括低通滤波器。通过低通滤波器滤除图像信号的高频部分可以达到去除随机噪声的目的,但是,由于图像的边缘部分和跃阶部分也处于高频区域,在低通滤波后该部分信号也被滤掉了,从而导致图像边缘和跳跃部分出现模糊。The spatial noise reduction unit may include a low-pass filter. Filtering out the high-frequency part of the image signal by a low-pass filter can achieve the purpose of removing random noise. However, since the edge part and step part of the image are also in the high-frequency area, this part of the signal is also filtered out after the low-pass filter. , Resulting in blurring of the edges and jumps of the image.
可以理解的是,由于经过空域降噪单元会损失更多的细节,因此,如果需要获取红外传感器的测温数据,需要采用进行空域降噪之前的数据。It is understandable that more details will be lost through the airspace noise reduction unit. Therefore, if you need to obtain the temperature measurement data of the infrared sensor, you need to use the data before airspace noise reduction.
为缓解空域降噪采用低通滤波产生的模糊,通常使用带边界保留的滤波器,该滤波器主要思想是根据噪声方差设定一个域值,仅使用落在阈值内的邻域点做低通滤波,当边界跳跃幅度大于该阈值时,不会出现模糊,而对跳跃幅度小于该域值的边界同样会造成模糊。使用边界保留滤波进行降噪需要准确的估计噪声的方差才能达到较好的降噪效果,但是,无论是采用低通滤波降噪法还是采用带边界保留的降噪法,空域降噪都不可避免的会对图像造成一定的模糊,噪声越大,被噪声淹没的纹理也越多,降噪后图像的模糊也越严重,仅依赖空域相关性难以得到满意效果,因此,空域降噪单元需要与时域降噪单元进行配合,利用时域的相关性使用多帧的信息相互补偿,从而达到更优的降噪效果。In order to alleviate the blur caused by low-pass filtering in spatial noise reduction, a filter with boundary preservation is usually used. The main idea of this filter is to set a threshold according to the noise variance, and only use the neighborhood points that fall within the threshold for low-pass Filtering, when the boundary jump amplitude is greater than the threshold, there will be no blur, and the boundary with the jump amplitude less than the threshold will also be blurred. The use of boundary preservation filtering for noise reduction requires accurate estimation of the variance of the noise to achieve a better noise reduction effect. However, whether it is a low-pass filter noise reduction method or a noise reduction method with boundary preservation, spatial noise reduction is inevitable The larger the noise, the more the texture is submerged by the noise, and the more serious the blur of the image after noise reduction. It is difficult to obtain satisfactory results only by relying on spatial correlation. Therefore, the spatial noise reduction unit needs to be combined with The time-domain noise reduction unit cooperates, and uses the correlation of the time domain to use the information of multiple frames to compensate each other, so as to achieve a better noise reduction effect.
进一步的,所述图像处理装置还包括频率分离单元与频率合成单元,其中,频率分离单元用于对图像校正及信噪比提升降噪后灰度图进行空域的频率分离;频率合成单元用于对拉伸后的灰度图进行空域的频率合成,并利用所述频率分离后的灰度图对拉伸后的灰度图中的中高频分量进行增强。Further, the image processing device further includes a frequency separation unit and a frequency synthesis unit, wherein the frequency separation unit is used to perform spatial frequency separation on the gray image after image correction and signal-to-noise ratio enhancement and noise reduction; the frequency synthesis unit is used for Perform spatial frequency synthesis on the stretched grayscale image, and use the frequency-separated grayscale image to enhance the middle and high frequency components in the stretched grayscale image.
具体地,所述频率分离单元在图像进行校正及信噪比提升之后执行空域的频率分离,将表示图像的边缘和跃阶部分的中高频率分量与表示图像背景区域的低频率分量进行分离,为后级对比度拉伸和细节增强做准备, 降低噪声,提升细节;之后由拉伸单元对图像进行对比度拉伸,对图像进行对比度拉伸后,频率合成单元对拉伸后的灰度图进行空域的频率合成;同时,可以采用频率分离单元获取的空域频率分离后的灰度图对拉伸后的灰度图中的中高频分量进行增强,通过补偿频率分离后的灰度图来提升细节,从而输出对比度和细节都增强后的红外灰度图。Specifically, the frequency separation unit performs frequency separation in the spatial domain after the image is corrected and the signal-to-noise ratio is improved, and separates the middle and high frequency components representing the edges and step parts of the image from the low frequency components representing the background area of the image, as Preparing for post-contrast stretching and detail enhancement to reduce noise and improve details; after that, the stretching unit performs contrast stretching on the image, and after the image is contrast stretched, the frequency synthesis unit performs spatial domain on the stretched grayscale image At the same time, the spatial frequency separated gray image obtained by the frequency separation unit can be used to enhance the medium and high frequency components in the stretched gray image, and the details can be improved by compensating the frequency separated gray image. In this way, the infrared gray image with enhanced contrast and details is output.
由于频率分离单元在对比度拉伸之前进行了空域的频率分离,将图像中的高频部分与低频部分进行了区分,拉伸单元仅对图像的低频部分进行对比度拉伸,从而避免了图像中高频部分包含的随机噪声同时被拉伸,造成图像的模糊;进一步的,通过频率合成单元将图像的高频部分与对比度拉伸后的图像进行叠加,从而能够采用图像中高频部分包含的边缘和跃阶部分细节,对对比度拉伸过程中产生的细节损失进行补偿。由此,保证在图像对比度增强的同时,实现了图像细节的增强。Since the frequency separation unit performs spatial frequency separation before the contrast stretching, it distinguishes the high frequency part from the low frequency part in the image. The stretching unit only performs contrast stretching on the low frequency part of the image, thereby avoiding the high frequency in the image. Part of the random noise contained in the image is stretched at the same time, causing the image to be blurred; further, the high-frequency part of the image is superimposed with the contrast-stretched image through the frequency synthesis unit, so that the edges and jumps contained in the high-frequency part of the image can be used. To compensate for the loss of detail in the process of contrast stretching. As a result, it is ensured that the image details are enhanced while the image contrast is enhanced.
在一些实施例中,所述拉伸单元包括第一对比度拉伸单元及第二对比度拉伸单元。其中,第一对比度拉伸单元用于执行第一级对比度拉伸,包括对频率分离后的灰度图中的每个像素的灰度值进行第一灰度直方图统计,并根据统计结果对像素做第一灰度变换得到对比度拉伸后的像素灰度值;In some embodiments, the stretching unit includes a first contrast stretching unit and a second contrast stretching unit. Wherein, the first contrast stretching unit is used to perform the first-level contrast stretching, including performing first grayscale histogram statistics on the grayscale value of each pixel in the frequency-separated grayscale image, and performing the statistics according to the statistical results. The pixel performs the first gray scale transformation to obtain the pixel gray value after contrast stretching;
第二对比度拉伸单元用于执行第二级对比度拉伸,包括对第一灰度变换后的灰度图中的每个像素的灰度值进行第二灰度直方图统计,并根据统计结果对像素做第二灰度变换得到对比度拉伸后的像素灰度值,其中,所述第二灰度变换不同于所述第一灰度变换。The second contrast stretching unit is used to perform the second-level contrast stretching, including performing second gray-scale histogram statistics on the gray value of each pixel in the gray-scale image after the first gray-scale transformation, and according to the statistical result Performing a second grayscale transformation on the pixel to obtain a pixel grayscale value after contrast stretching, wherein the second grayscale transformation is different from the first grayscale transformation.
具体地,所述第一灰度变换为线性灰度变换或非线性灰度变换;所述第二灰度变换也可以为线性灰度变换或非线性灰度变换。示例性地,所述第一灰度变换或第二灰度变换均为分段线性变换,在进行第一灰度变换时,对图像的第一有用数据的对比度进行增强;在进行第二灰度变换时,对图像的第二有用数据的对比度进行增强。通过上述装置,使得图像的灰度值可以更好地在直方图上分布,达到图像增强效果。Specifically, the first grayscale transformation is a linear grayscale transformation or a nonlinear grayscale transformation; the second grayscale transformation may also be a linear grayscale transformation or a nonlinear grayscale transformation. Exemplarily, the first gray-scale transformation or the second gray-scale transformation are both piecewise linear transformations. When the first gray-scale transformation is performed, the contrast of the first useful data of the image is enhanced; and the second gray-scale transformation is performed. During the degree transformation, the contrast of the second useful data of the image is enhanced. Through the above device, the gray value of the image can be better distributed on the histogram to achieve the image enhancement effect.
可以理解的是,上述对比度拉伸以两级对比度拉伸单元为例,在其他实施例中,还可以根据需要进行对比度增强的灰度图的具体情况,还可以包括两级以上的对比度拉伸单元,实现多级对比度拉伸。It is understandable that the above-mentioned contrast stretching takes a two-level contrast stretching unit as an example. In other embodiments, the grayscale image for contrast enhancement can also be performed according to the specific situation of the contrast enhancement, and it can also include more than two levels of contrast stretching. Unit to achieve multi-level contrast stretching.
优选地,所述拉伸单元还包括线性拉伸单元,用于在所述第一灰度直方图统计之前,对频率分离后的灰度图的灰度值进行线性变换,进行初步的线性拉伸,为后续第一级对比度拉伸及第二级对比度拉伸做准备。示例性的,可以通过线性拉伸将原本所有像素灰度值仅处于某一部分灰度区域的图像进行灰度变换,拉伸至0-255的整个灰度空间,从而图像的对比度能够得到大幅度增强。Preferably, the stretching unit further includes a linear stretching unit for performing a linear transformation on the gray value of the grayscale image after frequency separation before the statistics of the first grayscale histogram, and performing a preliminary linear stretching. Stretching to prepare for the subsequent first-level contrast stretching and second-level contrast stretching. Exemplarily, an image whose gray value of all pixels is originally only in a certain part of the grayscale area can be grayscale transformed by linear stretching, and stretched to the entire grayscale space of 0-255, so that the contrast of the image can be greatly improved. Enhanced.
可以理解的是,上述初步线性拉伸单元、第一对比度拉伸单元及第二对比度拉伸单元分别负责不同程度的拉伸,示例性的,线性拉伸单元最先执行全局对比度拉伸,第一级对比度拉伸单元及第二级对比度拉伸单元用于实现灵活的两级拉伸,得到较好的对比度。根据不同的图像处理需要,三个拉伸过程可以合并,也可以使用其中一个、两个或多个。It can be understood that the above-mentioned preliminary linear stretching unit, the first contrast stretching unit, and the second contrast stretching unit are respectively responsible for different degrees of stretching. Illustratively, the linear stretching unit first performs global contrast stretching. The first-stage contrast stretching unit and the second-stage contrast stretching unit are used to realize flexible two-stage stretching and obtain better contrast. According to different image processing needs, the three stretching processes can be combined, or one, two or more of them can be used.
在一些实施例中,所述图像处理装置还包括场景分析反馈单元。场景分析单元用于针对频率合成后实现了图像增强的灰度图进行场景分析,例如室内、室外、黑体、树林、海边等,并将场景分析结果反馈到前面的图像校正、降噪和/或拉伸过程,对图像校正、降噪和/或拉伸过程的参数进行调整,使得整个系统通过场景分析的反馈构成一个反馈系统,能自适应地针对不同场景进行合适的校正、去噪、对比度增强和细节增强。示例性的,所述场景分析过程得到的场景分析结果为树林,则获取树林对应的场景特性,根据该场景特性反馈至降噪及对比度和细节增强过程,调整上述过程的降噪或图像增强参数,从而进一步提高输出图像的质量。其中,树林对应的场景特性可以预存储在程序中,或保存在云端服务器,场景分析单元能够执行所述场景特性的调用。In some embodiments, the image processing device further includes a scene analysis feedback unit. The scene analysis unit is used to perform scene analysis on the grayscale images that have realized image enhancement after frequency synthesis, such as indoor, outdoor, black body, woods, seaside, etc., and feed back the results of the scene analysis to the previous image correction, noise reduction and/or In the stretching process, the parameters of the image correction, noise reduction and/or stretching process are adjusted, so that the entire system forms a feedback system through the feedback of scene analysis, which can adaptively perform appropriate correction, denoising, and contrast for different scenes. Enhancement and detail enhancement. Exemplarily, if the scene analysis result obtained by the scene analysis process is a forest, the scene characteristics corresponding to the forest are obtained, and the noise reduction and contrast and detail enhancement processes are fed back according to the scene characteristics, and the noise reduction or image enhancement parameters of the aforementioned process are adjusted , Thereby further improving the quality of the output image. Wherein, the scene characteristics corresponding to the forest can be pre-stored in the program or stored in the cloud server, and the scene analysis unit can execute the call of the scene characteristics.
其中,场景分析反馈单元的场景分析在图像频率合成之后进行,由于频率合成后的图像经过了对比度增强和细节增强,因此,其所包含的信息更加丰富,将其包含的信息进行反馈后,能够更好地实现对于降噪、对比度增强和细节增强过程中所用参数的调整。Among them, the scene analysis of the scene analysis feedback unit is performed after the image frequency synthesis. Since the frequency synthesized image has undergone contrast enhancement and detail enhancement, the information contained in it is richer. After the information contained in it is fed back, it can be Better realize the adjustment of the parameters used in the process of noise reduction, contrast enhancement and detail enhancement.
此外,所述图像处理装置还包括伪彩映射单元,用于对于频率合成后实现了图像增强的灰度图映射为YUV色彩图。由此,一方面凸显温度分布信息,一方面凸显物体的细节;其中,伪彩映射也在图像频率合成之后进行,也是由于频率合成后的图像经过了对比度增强和细节增强,从而所 包含的信息更加丰富,因此经过伪彩映射的图也含有最丰富的细节和更好的对比度,有利于实现高画质的图像输出。In addition, the image processing device further includes a pseudo-color mapping unit for mapping the grayscale image that has achieved image enhancement after frequency synthesis into a YUV color image. Therefore, on the one hand, the temperature distribution information is highlighted, on the other hand, the details of the object are highlighted; among them, the pseudo-color mapping is also performed after the image frequency synthesis, also because the image after frequency synthesis has undergone contrast enhancement and detail enhancement, so that the information contained Richer, so the pseudo-color mapping image also contains the richest details and better contrast, which is conducive to achieving high-quality image output.
需要说明的是,当同时设置有场景分析反馈单元时,所述场景分析反馈单元的输入数据应当采用伪彩映射之前的数据,避免场景分析处理的信息量过大。It should be noted that when a scene analysis feedback unit is provided at the same time, the input data of the scene analysis feedback unit should adopt the data before the pseudo-color mapping, so as to avoid the excessive amount of information processed by the scene analysis.
具体地,所述YUV色彩图为YUV444类型的色彩图时,还可以采用转码单元将YUV444的色彩图转码为YUV422或420的色彩图,向后输出,便于后续的编码,节约存储空间。Specifically, when the YUV color map is a color map of YUV444 type, a transcoding unit may also be used to transcode the color map of YUV444 into a color map of YUV422 or 420, and output it backward, which is convenient for subsequent encoding and saves storage space.
所述图像处理装置可包括具有数据处理能力的任何类型的数据处理装置,例如但不限于CPU、DSP、FPGA、CPLD等。The image processing device may include any type of data processing device with data processing capabilities, such as but not limited to CPU, DSP, FPGA, CPLD, etc.
通过本实施例的图像处理装置,解决了红外传感器输出的灰度图存在的坏点、明显固定模式噪声和随机噪声、对比度低、信噪比低、图像细节少等问题,从而得到了瑕疵更少、对比度更高、细节更多、画质更好的红外图像。Through the image processing device of this embodiment, the problems of dead pixels, obvious fixed pattern noise and random noise, low contrast, low signal-to-noise ratio, and less image details in the grayscale image output by the infrared sensor are solved, thereby obtaining more flaws. Infrared images with less, higher contrast, more detail, and better picture quality.
本公开第三个示意性实施例提供了一种图像处理装置,包括读存储介质及一个或多个处理器。其中,可读存储介质用于存储可执行指令;一个或多个处理器用于执行所述可执行指令,以执行如第一实施例所述的图像处理方法。A third exemplary embodiment of the present disclosure provides an image processing device, including a reading storage medium and one or more processors. The readable storage medium is used to store executable instructions; one or more processors are used to execute the executable instructions to execute the image processing method described in the first embodiment.
本公开第四个示意性实施例提供了一种成像设备。图5为本公开实施例成像设备的结构示意图。如图5所示,所述成像设备包括图像传感器及图像处理装置,所述图像传感器连接至所述图像处理装置,所述图像处理装置采用如第二实施例所述的图像处理装置。The fourth exemplary embodiment of the present disclosure provides an imaging device. FIG. 5 is a schematic structural diagram of an imaging device according to an embodiment of the disclosure. As shown in FIG. 5, the imaging device includes an image sensor and an image processing device, the image sensor is connected to the image processing device, and the image processing device adopts the image processing device described in the second embodiment.
本公开第五个实施例提供了一种可移动载体。图6为本公开实施例可移动载体的结构示意图。如图6所示,所述可移动载体包括:机身和成像设备,所述成像设备安装于所述机身,所述成像设备包括图像传感器以及图像处理装置,所述图像传感器连接至所述图像处理装置,所述图像处理装置采用如第二实施例所述的图像处理装置。其中,所述可移动载体可以为无人机、无人车、无人船等。The fifth embodiment of the present disclosure provides a movable carrier. FIG. 6 is a schematic structural diagram of a movable carrier according to an embodiment of the disclosure. As shown in FIG. 6, the movable carrier includes: a body and an imaging device, the imaging device is installed in the body, the imaging device includes an image sensor and an image processing device, the image sensor is connected to the An image processing device, the image processing device adopts the image processing device described in the second embodiment. Wherein, the movable carrier may be an unmanned aerial vehicle, an unmanned vehicle, an unmanned ship, and the like.
本公开第六个实施例提供了一种计算机可读存储介质,所述计算机可读存储介质存储有计算机程序,所述计算机程序被处理器执行时使所述处 理器实现如第一实施例所述的图像处理方法。The sixth embodiment of the present disclosure provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor realizes the same as described in the first embodiment. The image processing method described.
本领域技术人员可以清楚地了解到,为描述的方便和简洁,仅以上述各功能模块的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能模块完成,即将装置的内部结构划分成不同的功能模块,以完成以上描述的全部或者部分功能。上述描述的装置的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。Those skilled in the art can clearly understand that for the convenience and conciseness of the description, only the division of the above-mentioned functional modules is used as an example. In practical applications, the above-mentioned functions can be allocated by different functional modules as required, that is, the device The internal structure is divided into different functional modules to complete all or part of the functions described above. For the specific working process of the device described above, reference may be made to the corresponding process in the foregoing method embodiment, which is not repeated here.
最后应说明的是:以上各实施例仅用以说明本公开的技术方案,而非对其限制;尽管参照前述各实施例对本公开进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分或者全部技术特征进行等同替换;在不冲突的情况下,本公开实施例中的特征可以任意组合;而这些修改或者替换,并不使相应技术方案的本质脱离本公开各实施例技术方案的范围。Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, not to limit it; although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions recorded in the foregoing embodiments can still be modified, or some or all of the technical features can be equivalently replaced; in the case of no conflict, the features in the embodiments of the present disclosure can be combined arbitrarily; and these modifications or replacements It does not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present disclosure.

Claims (75)

  1. 一种图像处理方法,其特征在于,包括:An image processing method, characterized in that it comprises:
    对灰度图进行图像校正及信噪比提升;Perform image correction and signal-to-noise ratio improvement on grayscale images;
    对图像校正及信噪比提升后的灰度图进行对比度拉伸;其中,Perform contrast stretching on the grayscale image after image correction and signal-to-noise ratio enhancement; among them,
    所述图像校正包括:The image correction includes:
    对灰度图进行平场校正;以及Perform flat-field correction on grayscale images; and
    对平场校正后的灰度图进行坏点校正;Perform dead pixel correction on the gray image after flat field correction;
    所述信噪比提升包括:The signal-to-noise ratio improvement includes:
    对灰度图进行时域噪声的去除。Removal of time-domain noise on grayscale images.
  2. 如权利要求1所述的图像处理方法,其特征在于,所述图像处理方法还包括:5. The image processing method of claim 1, wherein the image processing method further comprises:
    对图像校正及信噪比提升后的灰度图进行空域的频率分离,并将频率分离后的灰度图的低频部分进行拉伸;Perform spatial frequency separation on the grayscale image after image correction and signal-to-noise ratio enhancement, and stretch the low-frequency part of the grayscale image after frequency separation;
    对拉伸后的灰度图进行空域的频率合成。Perform spatial frequency synthesis on the stretched grayscale image.
  3. 如权利要求2所述的图像处理方法,其特征在于,所述图像处理方法还包括:3. The image processing method of claim 2, wherein the image processing method further comprises:
    利用所述频率分离后的灰度图对拉伸后的灰度图中的中高频分量进行增强。The frequency-separated grayscale image is used to enhance the medium and high frequency components in the stretched grayscale image.
  4. 如权利要求2所述的图像处理方法,其特征在于,所述对图像校正及信噪比提升后的灰度图进行对比度拉伸包括:3. The image processing method according to claim 2, wherein said performing contrast stretching on the gray image after image correction and signal-to-noise ratio increase comprises:
    对频率分离后的灰度图中的每个像素的灰度值进行第一灰度直方图统计,并根据统计结果对像素做第一灰度变换得到对比度拉伸后的像素灰度值;Perform a first grayscale histogram statistics on the grayscale value of each pixel in the grayscale image after frequency separation, and perform the first grayscale transformation on the pixel according to the statistical result to obtain the pixel grayscale value after contrast stretching;
    对第一灰度变换后的灰度图中的每个像素的灰度值进行第二灰度直方图统计,并根据统计结果对像素做第二灰度变换得到对比度拉伸后的像素灰度值,其中,所述第二灰度变换不同于所述第一灰度变换。Perform a second grayscale histogram statistics on the grayscale value of each pixel in the grayscale image after the first grayscale transformation, and perform the second grayscale transformation on the pixels according to the statistical results to obtain the pixel grayscale after contrast stretching Value, wherein the second grayscale transformation is different from the first grayscale transformation.
  5. 如权利要求4所述的图像处理方法,其特征在于,所述第一灰度变换为线性灰度变换或非线性灰度变换;和/或,所述第二灰度变换为线性灰度变换或非线性灰度变换。The image processing method according to claim 4, wherein the first grayscale transformation is a linear grayscale transformation or a nonlinear grayscale transformation; and/or the second grayscale transformation is a linear grayscale transformation Or nonlinear gray scale transformation.
  6. 如权利要求4所述的图像处理方法,其特征在于,所述对图像校 正及信噪比提升后的灰度图进行对比度拉伸还包括:The image processing method according to claim 4, wherein said performing contrast stretching on the gray image after image correction and signal-to-noise ratio increase further comprises:
    在所述第一灰度直方图统计之前,对频率分离后的灰度图的灰度值进行线性变换。Before the statistics of the first grayscale histogram, the grayscale value of the grayscale image after frequency separation is linearly transformed.
  7. 如权利要求1所述的图像处理方法,其特征在于,所述对所述灰度图进行平场校正包括:利用参考灰度图对所述图像传感器采集的灰度图进行平场校正,所述参考灰度图为所述图像传感器在快门关闭时采集的灰度图。The image processing method of claim 1, wherein the flat-field correction of the grayscale image comprises: using a reference grayscale image to perform flat-field correction on the grayscale image collected by the image sensor, and The reference grayscale image is a grayscale image collected by the image sensor when the shutter is closed.
  8. 如权利要求1所述的图像处理方法,其特征在于,所述对平场校正后的灰度图进行坏点校正包括:8. The image processing method according to claim 1, wherein said performing dead pixel correction on the gray image after flat field correction comprises:
    对图像传感器的静态坏点对应的像素进行校正;和/或Correct the pixels corresponding to the static dead pixels of the image sensor; and/or
    检测所述图像传感器的动态坏点,并对所述动态坏点对应的像素进行校正。The dynamic dead pixels of the image sensor are detected, and the pixels corresponding to the dynamic dead pixels are corrected.
  9. 如权利要求1所述的图像处理方法,其特征在于,所述信噪比提升还包括:8. The image processing method of claim 1, wherein the signal-to-noise ratio improvement further comprises:
    去除时域噪声后,再去除时域降噪后的灰度图中的固定模式噪声;After removing the time domain noise, remove the fixed pattern noise in the grayscale image after the time domain noise reduction;
    或者;or;
    去除灰度图中的固定模式噪声,再对去除了固定模式噪声的灰度图进行时域噪声的去除。Remove the fixed pattern noise in the grayscale image, and then remove the time domain noise on the grayscale image from which the fixed pattern noise has been removed.
  10. 如权利要求9所述的图像处理方法,其特征在于,所述信噪比提升还包括:9. The image processing method of claim 9, wherein the signal-to-noise ratio improvement further comprises:
    对去除了时域噪声以及固定模式噪声后的灰度图进行空域随机噪声的去除。After removing the time domain noise and the fixed pattern noise, the gray image is removed from the spatial domain random noise.
  11. 如权利要求1所述的图像处理方法,其特征在于,所述信噪比提升还包括:8. The image processing method of claim 1, wherein the signal-to-noise ratio improvement further comprises:
    去除时域噪声后,去除时域降噪后的灰度图中的空域随机噪声。After removing the time domain noise, the spatial random noise in the grayscale image after the time domain noise reduction is removed.
  12. 如权利要求1所述的图像处理方法,其特征在于,所述时域噪声包括:时域随机单点噪声、时域随机行噪声、时域随机列噪声中的至少一种。8. The image processing method according to claim 1, wherein the time domain noise comprises at least one of time domain random single point noise, time domain random row noise, and time domain random column noise.
  13. 如权利要求1所述的图像处理方法,其特征在于,所述图像校正还包括:8. The image processing method of claim 1, wherein the image correction further comprises:
    在所述坏点校正之前,对平场校正后的灰度图进行非线性校正。Before the dead pixel correction, non-linear correction is performed on the gray image after the flat field correction.
  14. 如权利要求1所述的图像处理方法,其特征在于,还包括:8. The image processing method of claim 1, further comprising:
    对对比度拉伸后的灰度图进行场景信息分析,并将分析结果反馈至所述图像校正、信噪比提升、对比度拉伸至少其中之一,针对不同场景自适应地调整所述图像校正、信噪比提升、对比度拉伸至少其中之一的参数。Perform scene information analysis on the grayscale image after contrast stretching, and feed back the analysis result to at least one of the image correction, signal-to-noise ratio improvement, and contrast stretching, and adaptively adjust the image correction, S/N ratio improvement, contrast stretching at least one of the parameters.
  15. 如权利要求1所述的图像处理方法,其特征在于,还包括:8. The image processing method of claim 1, further comprising:
    将对比度拉伸后的灰度图映射为YUV色彩图。The grayscale image after contrast stretching is mapped to the YUV color image.
  16. 如权利要求15所述的图像处理方法,其特征在于,还包括:15. The image processing method of claim 15, further comprising:
    当伪彩映射的输出为YUV444色彩图时,将YUV444色彩图转码为YUV422或YUV420的色彩图。When the output of the pseudo-color mapping is the YUV444 color map, the YUV444 color map is transcoded into the YUV422 or YUV420 color map.
  17. 如权利要求1所述的图像处理方法,其特征在于,还包括:8. The image processing method of claim 1, further comprising:
    在进行所述平场校正之前,接收图像传感器采集的灰度图,并对所述图像传感器进行控制,以及对所述图像传感器的动态范围进行检查及校正。Before performing the flat field correction, receiving the grayscale image collected by the image sensor, controlling the image sensor, and checking and correcting the dynamic range of the image sensor.
  18. 如权利要求1或10所述的图像处理方法,其特征在于,还包括:8. The image processing method of claim 1 or 10, further comprising:
    根据去除时域噪声或固定模式噪声后的灰度图确定绝对温度。Determine the absolute temperature according to the gray image after removing the time domain noise or fixed pattern noise.
  19. 一种图像处理装置,其特征在于,包括:An image processing device, characterized in that it comprises:
    校正单元,用于灰度图进行图像校正;Correction unit for image correction of grayscale images;
    降噪单元,用于对灰度图进行信噪比提升;以及The noise reduction unit is used to improve the signal-to-noise ratio of the grayscale image; and
    拉伸单元,用于对所述图像校正及信噪比提升后的灰度图进行对比度拉伸;The stretching unit is used to perform contrast stretching on the grayscale image after the image correction and the signal-to-noise ratio increase;
    其中,所述校正单元包括:Wherein, the correction unit includes:
    平场校正单元,用于对灰度图进行平场校正;Flat-field correction unit for flat-field correction of grayscale images;
    坏点校正单元,用于对平场校正后的灰度图进行坏点校正;The dead pixel correction unit is used to perform dead pixel correction on the gray image after flat field correction;
    所述降噪单元包括:The noise reduction unit includes:
    时域降噪单元,用于对灰度图进行时域噪声的去除。The time domain noise reduction unit is used to remove the time domain noise of the gray image.
  20. 如权利要求19所述的图像处理装置,其特征在于,所述图像处理装置还包括:19. The image processing device of claim 19, wherein the image processing device further comprises:
    频率分离单元,用于对所述图像校正及信噪比提升后的灰度图进行空域的频率分离,并将频率分离后的灰度图的低频部分输出至拉伸单元;The frequency separation unit is used to perform spatial frequency separation on the gray image after the image correction and the signal-to-noise ratio increase, and output the low frequency part of the gray image after the frequency separation to the stretching unit;
    频率合成单元,用于对拉伸单元输出的灰度图进行空域的频率合成。The frequency synthesis unit is used to perform spatial frequency synthesis on the gray image output by the stretching unit.
  21. 如权利要求20所述的图像处理装置,其特征在于,所述频率合成单元还用于接收所述频率分离后的灰度图,利用所述频率分离后的灰度图对拉伸单元输出的灰度图中的中高频分量进行增强。The image processing device according to claim 20, wherein the frequency synthesis unit is further configured to receive the frequency-separated grayscale image, and use the frequency-separated grayscale image to output to the stretching unit The middle and high frequency components in the grayscale image are enhanced.
  22. 如权利要求20所述的图像处理装置,其特征在于,所述拉伸单元包括:22. The image processing device according to claim 20, wherein the stretching unit comprises:
    第一对比度拉伸单元,所述第一对比度拉伸单元用于对频率分离后的灰度图中的每个像素的灰度值进行第一灰度直方图统计,并根据统计结果对像素做第一灰度变换得到对比度拉伸后的像素灰度值;The first contrast stretching unit, the first contrast stretching unit is used to perform the first grayscale histogram statistics on the grayscale value of each pixel in the grayscale image after frequency separation, and perform the statistics on the pixels according to the statistical results The first gray scale transformation obtains the pixel gray value after contrast stretching;
    第二对比度拉伸单元,用于对第一对比度拉伸单元输出的灰度图中的每个像素的灰度值进行第二灰度直方图统计,并根据统计结果对像素做第二灰度变换得到对比度拉伸后的像素灰度值,其中,所述第二灰度变换不同于所述第一灰度变换。The second contrast stretching unit is used to perform a second grayscale histogram statistics on the grayscale value of each pixel in the grayscale image output by the first contrast stretching unit, and perform a second grayscale on the pixel according to the statistical result The pixel gray value after the contrast stretching is obtained by transformation, wherein the second gray scale transformation is different from the first gray scale transformation.
  23. 如权利要求22所述的图像处理装置,其特征在于,所述第一灰度变换为线性灰度变换或非线性灰度变换;和/或所述第二灰度变换均为线性灰度变换或非线性灰度变换。The image processing device according to claim 22, wherein the first grayscale transformation is a linear grayscale transformation or a nonlinear grayscale transformation; and/or the second grayscale transformation is a linear grayscale transformation Or nonlinear gray scale transformation.
  24. 如权利要求22所述的图像处理装置,其特征在于,所述拉伸单元还包括:22. The image processing device according to claim 22, wherein the stretching unit further comprises:
    线性拉伸单元,用于对频率分离后的灰度图的灰度值进行线性变换;Linear stretching unit, used to linearly transform the gray value of the gray image after frequency separation;
    所述第一对比度拉伸单元还用于对经线性变换的每个像素的灰度值进行第一灰度直方图统计。The first contrast stretching unit is also used to perform first gray-scale histogram statistics on the gray-scale value of each pixel that has been linearly transformed.
  25. 如权利要求19所述的图像处理装置,其特征在于,所述平场校正单元用于利用参考灰度图对所述图像传感器采集的灰度图进行平场校正,所述参考灰度图为所述图像传感器在所述快门关闭时采集的灰度图。The image processing device according to claim 19, wherein the flat field correction unit is configured to perform flat field correction on the gray image collected by the image sensor by using a reference gray image, and the reference gray image is A grayscale image collected by the image sensor when the shutter is closed.
  26. 如权利要求19所述的图像处理装置,其特征在于,所述坏点校正单元包括:19. The image processing device of claim 19, wherein the dead pixel correction unit comprises:
    静态坏点校正子单元,用于对所述图像传感器的静态坏点对应的像素进行校正;和/或The static dead pixel correction subunit is used to correct the pixels corresponding to the static dead pixels of the image sensor; and/or
    动态坏点校正子单元,用于检测所述图像传感器的动态坏点,并对所述动态坏点对应的像素进行校正。The dynamic dead pixel correction subunit is used to detect the dynamic dead pixels of the image sensor and correct the pixels corresponding to the dynamic dead pixels.
  27. 如权利要求19所述的图像处理装置,其特征在于,所述降噪单 元还包括:The image processing device according to claim 19, wherein the noise reduction unit further comprises:
    固定模式噪声去除单元,用于去除坏点校正单元或时域降噪单元输出的灰度图中固定模式噪声。The fixed pattern noise removal unit is used to remove fixed pattern noise in the grayscale output from the bad pixel correction unit or the time domain noise reduction unit.
  28. 如权利要求27所述的图像处理装置,其特征在于,所述降噪单元还包括:The image processing device according to claim 27, wherein the noise reduction unit further comprises:
    空域降噪单元,用于对去除了时域噪声以及固定模式噪声后的灰度图进行空域随机噪声的去除。The spatial noise reduction unit is used to remove the spatial random noise of the gray image after removing the time domain noise and the fixed pattern noise.
  29. 如权利要求19所述的图像处理装置,其特征在于,所述降噪单元还包括:19. The image processing device of claim 19, wherein the noise reduction unit further comprises:
    空域降噪单元,用于对时域降噪单元输出的灰度图进行空域随机噪声的去除。The spatial noise reduction unit is used to remove the spatial random noise of the gray image output by the time domain noise reduction unit.
  30. 如权利要求19所述的图像处理装置,其特征在于,所述时域降噪单元用于去除时域随机单点噪声、时域随机行噪声、时域随机列噪声中的至少一种。The image processing device according to claim 19, wherein the time-domain noise reduction unit is used to remove at least one of time-domain random single point noise, time-domain random row noise, and time-domain random column noise.
  31. 如权利要求19所述的图像处理装置,其特征在于,所述校正单元还包括:19. The image processing device according to claim 19, wherein the correction unit further comprises:
    非线性校正单元,平场校正后的灰度图经所述非线性校正单元进行非线性校正后,由所述坏点校正单元进行坏点校正。A non-linear correction unit, after the flat-field corrected gray image is non-linearly corrected by the non-linear correction unit, the bad pixel correction unit performs bad pixel correction.
  32. 如权利要求19所述的图像处理装置,其特征在于,还包括:19. The image processing device of claim 19, further comprising:
    场景分析反馈单元,用于对所述对比度拉伸后的灰度图进行场景信息分析,并将分析结果反馈至图像校正单元、降噪单元和/或拉伸单元至少其中之一,针对不同场景自适应地调整所述图像校正单元、降噪单元、拉伸单元至少其中之一的参数。The scene analysis feedback unit is used to analyze the scene information of the gray scale image after the contrast stretching, and feed back the analysis result to at least one of the image correction unit, the noise reduction unit and/or the stretching unit, for different scenes The parameters of at least one of the image correction unit, noise reduction unit, and stretching unit are adaptively adjusted.
  33. 如权利要求19所述的图像处理装置,其特征在于,还包括:19. The image processing device of claim 19, further comprising:
    伪彩映射单元,用于将对比度拉伸后的灰度图映射为YUV色彩图。The pseudo-color mapping unit is used to map the grayscale image after the contrast stretching to the YUV color image.
  34. 如权利要求33所述的图像处理装置,其特征在于,还包括:The image processing device according to claim 33, further comprising:
    转码单元,用于在伪彩映射单元输出为YUV444色彩图时,将YUV444色彩图转码为YUV422或YUV420的色彩图。The transcoding unit is used to transcode the YUV444 color map into a YUV422 or YUV420 color map when the pseudo color mapping unit outputs a YUV444 color map.
  35. 如权利要求19所述的图像处理装置,其特征在于,还包括:19. The image processing device of claim 19, further comprising:
    信号接收及控制单元,用于接收并保存所述图像传感器采集的灰度图, 并对所述图像传感器进行控制,以及对所述图像传感器的动态范围进行检查及校正。The signal receiving and control unit is used to receive and save the grayscale image collected by the image sensor, to control the image sensor, and to check and correct the dynamic range of the image sensor.
  36. 如权利要求19或27所述的图像处理装置,其特征在于,还包括:The image processing device according to claim 19 or 27, further comprising:
    测温单元,用于根据所述时域降噪单元或固定模式噪声去除单元输出的灰度图确定绝对温度。The temperature measurement unit is used to determine the absolute temperature according to the grayscale image output by the time domain noise reduction unit or the fixed pattern noise removal unit.
  37. 一种图像处理装置,其特征在于,包括:An image processing device, characterized in that it comprises:
    可读存储介质,用于存储可执行指令;A readable storage medium for storing executable instructions;
    一个或多个处理器,用于执行所述可执行指令,以执行根据权利要求1-18任一项所述的图像处理方法。One or more processors, configured to execute the executable instructions to execute the image processing method according to any one of claims 1-18.
  38. 一种成像设备,其特征在于,包括:图像传感器以及图像处理装置,所述图像传感器连接至所述图像处理装置,所述图像处理装置包括:An imaging device, characterized by comprising: an image sensor and an image processing device, the image sensor is connected to the image processing device, and the image processing device includes:
    校正单元,用于灰度图进行图像校正;Correction unit for image correction of grayscale images;
    降噪单元,用于对灰度图进行信噪比提升;以及The noise reduction unit is used to improve the signal-to-noise ratio of the grayscale image; and
    拉伸单元,用于对所述图像校正及信噪比提升后的灰度图进行对比度拉伸;The stretching unit is used to perform contrast stretching on the grayscale image after the image correction and the signal-to-noise ratio increase;
    其中,所述校正单元包括:Wherein, the correction unit includes:
    平场校正单元,用于对灰度图进行平场校正;Flat-field correction unit for flat-field correction of grayscale images;
    坏点校正单元,用于对平场校正后的灰度图进行坏点校正;The dead pixel correction unit is used to perform dead pixel correction on the gray image after flat field correction;
    所述降噪单元包括:The noise reduction unit includes:
    时域降噪单元,用于对灰度图进行时域噪声的去除。The time domain noise reduction unit is used to remove the time domain noise of the gray image.
  39. 如权利要求38所述的成像设备,其特征在于,所述图像处理装置还包括:The imaging device according to claim 38, wherein the image processing device further comprises:
    频率分离单元,用于对所述图像校正及信噪比提升后的灰度图进行空域的频率分离,并将频率分离后的灰度图输出至拉伸单元;The frequency separation unit is used to perform spatial frequency separation on the grayscale image after image correction and signal-to-noise ratio enhancement, and output the frequency-separated grayscale image to the stretching unit;
    频率合成单元,用于对拉伸单元输出的灰度图进行空域的频率合成。The frequency synthesis unit is used to perform spatial frequency synthesis on the gray image output by the stretching unit.
  40. 如权利要求39所述的成像设备,其特征在于,所述频率合成单元还用于接收所述频率分离后的灰度图,利用所述频率分离后的灰度图对拉伸单元输出的灰度图中的中高频分量进行增强。The imaging device of claim 39, wherein the frequency synthesis unit is further configured to receive the frequency-separated grayscale image, and use the frequency-separated grayscale image to compare the grayscale output from the stretching unit The mid- and high-frequency components in the degree diagram are enhanced.
  41. 如权利要求39所述的成像设备,其特征在于,所述拉伸单元包括:The imaging device according to claim 39, wherein the stretching unit comprises:
    第一对比度拉伸单元,所述第一对比度拉伸单元用于对频率分离后的灰度图中的每个像素的灰度值进行第一灰度直方图统计,并根据统计结果对像素做第一灰度变换得到对比度拉伸后的像素灰度值;The first contrast stretching unit, the first contrast stretching unit is used to perform the first grayscale histogram statistics on the grayscale value of each pixel in the grayscale image after frequency separation, and perform the statistics on the pixels according to the statistical results The first gray scale transformation obtains the pixel gray value after contrast stretching;
    第二对比度拉伸单元,用于对第一对比度拉伸单元输出的灰度图中的每个像素的灰度值进行第二灰度直方图统计,并根据统计结果对像素做第二灰度变换得到对比度拉伸后的像素灰度值,其中,所述第二灰度变换不同于所述第一灰度变换。The second contrast stretching unit is used to perform a second grayscale histogram statistics on the grayscale value of each pixel in the grayscale image output by the first contrast stretching unit, and perform a second grayscale on the pixel according to the statistical result The pixel gray value after the contrast stretching is obtained by transformation, wherein the second gray scale transformation is different from the first gray scale transformation.
  42. 如权利要求41所述的成像设备,其特征在于,所述第一灰度变换为线性灰度变换或非线性灰度变换;和/或所述第二灰度变换均为线性灰度变换或非线性灰度变换。The imaging device according to claim 41, wherein the first gray-scale transformation is a linear gray-scale transformation or a nonlinear gray-scale transformation; and/or the second gray-scale transformation is a linear gray-scale transformation or Non-linear gray scale transformation.
  43. 如权利要求41所述的成像设备,其特征在于,所述拉伸单元还包括:41. The imaging device according to claim 41, wherein the stretching unit further comprises:
    线性拉伸单元,用于对频率分离后的灰度图的灰度值进行线性变换;Linear stretching unit, used to linearly transform the gray value of the gray image after frequency separation;
    所述第一对比度拉伸单元还用于对经线性变换的每个像素的灰度值进行第一灰度直方图统计。The first contrast stretching unit is also used to perform first gray-scale histogram statistics on the gray-scale value of each pixel that has been linearly transformed.
  44. 如权利要求38所述的成像设备,其特征在于,所述平场校正单元用于利用参考灰度图对所述图像传感器采集的灰度图进行平场校正,所述参考灰度图为所述图像传感器在所述快门关闭时采集的灰度图。The imaging device according to claim 38, wherein the flat-field correction unit is configured to perform flat-field correction on the gray-scale image collected by the image sensor by using a reference gray-scale image, and the reference gray-scale image is The grayscale image collected by the image sensor when the shutter is closed.
  45. 如权利要求38所述的成像设备,其特征在于,所述坏点校正单元包括:The imaging device according to claim 38, wherein the dead pixel correction unit comprises:
    静态坏点校正子单元,用于对所述图像传感器的静态坏点对应的像素进行校正;和/或The static dead pixel correction subunit is used to correct the pixels corresponding to the static dead pixels of the image sensor; and/or
    动态坏点校正子单元,用于检测所述图像传感器的动态坏点,并对所述动态坏点对应的像素进行校正。The dynamic dead pixel correction subunit is used to detect the dynamic dead pixels of the image sensor and correct the pixels corresponding to the dynamic dead pixels.
  46. 如权利要求38所述的成像设备,其特征在于,所述降噪单元还包括:The imaging device of claim 38, wherein the noise reduction unit further comprises:
    固定模式噪声去除单元,用于去除坏点校正单元或时域降噪单元输出的灰度图中固定模式噪声。The fixed pattern noise removal unit is used to remove fixed pattern noise in the grayscale output from the bad pixel correction unit or the time domain noise reduction unit.
  47. 如权利要求46所述的成像设备,其特征在于,所述降噪单元还包括:The imaging device of claim 46, wherein the noise reduction unit further comprises:
    空域降噪单元,用于对去除了时域噪声以及固定模式噪声后的灰度图进行空域随机噪声的去除。The spatial noise reduction unit is used to remove the spatial random noise of the gray image after removing the time domain noise and the fixed pattern noise.
  48. 如权利要求38所述的成像设备,其特征在于,所述降噪单元还包括:The imaging device of claim 38, wherein the noise reduction unit further comprises:
    空域降噪单元,用于对时域降噪单元输出的灰度图进行空域随机噪声的去除。The spatial noise reduction unit is used to remove the spatial random noise of the gray image output by the time domain noise reduction unit.
  49. 如权利要求38所述的成像设备,其特征在于,所述时域降噪单元用于去除时域随机单点噪声、时域随机行噪声、时域随机列噪声中的至少一种。The imaging device according to claim 38, wherein the time-domain noise reduction unit is used to remove at least one of time-domain random single-point noise, time-domain random row noise, and time-domain random column noise.
  50. 如权利要求38所述的成像设备,其特征在于,所述校正单元还包括:The imaging device according to claim 38, wherein the correction unit further comprises:
    非线性校正单元,平场校正后的灰度图经所述非线性校正单元进行非线性校正后,由所述坏点校正单元进行坏点校正。A non-linear correction unit, after the flat-field corrected gray image is non-linearly corrected by the non-linear correction unit, the bad pixel correction unit performs bad pixel correction.
  51. 如权利要求38所述的成像设备,其特征在于,还包括:The imaging device of claim 38, further comprising:
    场景分析反馈单元,用于对所述对比度拉伸后的灰度图进行场景信息分析,并将分析结果反馈至图像校正单元、降噪单元、拉伸单元至少其中之一,针对不同场景自适应地调整所述图像校正单元、降噪单元、拉伸单元至少其中之一的参数。The scene analysis feedback unit is used to analyze the scene information of the gray image after the contrast stretching, and feed back the analysis result to at least one of the image correction unit, the noise reduction unit, and the stretching unit, and is adaptive to different scenes The parameters of at least one of the image correction unit, noise reduction unit, and stretching unit are adjusted.
  52. 如权利要求38所述的成像设备,其特征在于,还包括:The imaging device of claim 38, further comprising:
    伪彩映射单元,用于将对比度拉伸后的灰度图映射为YUV色彩图。The pseudo-color mapping unit is used to map the grayscale image after the contrast stretching to the YUV color image.
  53. 如权利要求52所述的成像设备,其特征在于,还包括:52. The imaging device of claim 52, further comprising:
    转码单元,用于在伪彩映射单元输出为YUV444色彩图时,将YUV444色彩图转码为YUV422或YUV420的色彩图。The transcoding unit is used to transcode the YUV444 color map into a YUV422 or YUV420 color map when the pseudo color mapping unit outputs a YUV444 color map.
  54. 如权利要求38所述的成像设备,其特征在于,还包括:The imaging device of claim 38, further comprising:
    信号接收及控制单元,用于接收并保存所述图像传感器采集的灰度图,并对所述图像传感器进行控制,以及对所述图像传感器的动态范围进行检查及校正。The signal receiving and control unit is used to receive and save the grayscale image collected by the image sensor, control the image sensor, and check and correct the dynamic range of the image sensor.
  55. 如权利要求38或46所述的成像设备,其特征在于,还包括:The imaging device according to claim 38 or 46, further comprising:
    测温单元,用于根据所述时域降噪单元或固定模式噪声去除单元输出的灰度图确定绝对温度。The temperature measurement unit is used to determine the absolute temperature according to the grayscale image output by the time domain noise reduction unit or the fixed pattern noise removal unit.
  56. 一种可移动载体,其特征在于,包括:机身和成像设备,所述成像设备安装于所述机身,所述成像设备包括图像传感器以及图像处理装置,所述图像传感器连接至所述图像处理装置,所述图像处理装置包括:A movable carrier, comprising: a body and an imaging device, the imaging device is installed in the body, the imaging device includes an image sensor and an image processing device, the image sensor is connected to the image A processing device, the image processing device includes:
    校正单元,用于灰度图进行图像校正;Correction unit for image correction of grayscale images;
    降噪单元,用于对灰度图进行信噪比提升;以及The noise reduction unit is used to improve the signal-to-noise ratio of the grayscale image; and
    拉伸单元,用于对所述图像校正及信噪比提升后的灰度图进行对比度拉伸;The stretching unit is used to perform contrast stretching on the grayscale image after the image correction and the signal-to-noise ratio increase;
    其中,所述校正单元包括:Wherein, the correction unit includes:
    平场校正单元,用于对灰度图进行平场校正;Flat-field correction unit for flat-field correction of grayscale images;
    坏点校正单元,用于对平场校正后的灰度图进行坏点校正;The dead pixel correction unit is used to perform dead pixel correction on the gray image after flat field correction;
    所述降噪单元包括:The noise reduction unit includes:
    时域降噪单元,用于对灰度图进行时域噪声的去除。The time domain noise reduction unit is used to remove the time domain noise of the gray image.
  57. 如权利要求56所述的可移动载体,其特征在于,所述图像处理装置还包括:The movable carrier according to claim 56, wherein the image processing device further comprises:
    频率分离单元,用于对所述图像校正及信噪比提升后的灰度图进行空域的频率分离,并将频率分离后的灰度图输出至拉伸单元;The frequency separation unit is used to perform spatial frequency separation on the grayscale image after image correction and signal-to-noise ratio enhancement, and output the frequency-separated grayscale image to the stretching unit;
    频率合成单元,用于对拉伸单元输出的灰度图进行空域的频率合成。The frequency synthesis unit is used to perform spatial frequency synthesis on the gray image output by the stretching unit.
  58. 如权利要求57所述的可移动载体,其特征在于,所述频率合成单元还用于接收所述频率分离后的灰度图,利用所述频率分离后的灰度图对拉伸单元输出的灰度图中的中高频分量进行增强。The removable carrier according to claim 57, wherein the frequency synthesis unit is further configured to receive the frequency-separated grayscale image, and use the frequency-separated grayscale image to output to the stretching unit The middle and high frequency components in the grayscale image are enhanced.
  59. 如权利要求57所述的可移动载体,其特征在于,所述拉伸单元包括:The movable carrier according to claim 57, wherein the stretching unit comprises:
    第一对比度拉伸单元,所述第一对比度拉伸单元用于对频率分离后的灰度图中的每个像素的灰度值进行第一灰度直方图统计,并根据统计结果对像素做第一灰度变换得到对比度拉伸后的像素灰度值;The first contrast stretching unit, the first contrast stretching unit is used to perform the first grayscale histogram statistics on the grayscale value of each pixel in the grayscale image after frequency separation, and perform the statistics on the pixels according to the statistical results The first gray scale transformation obtains the pixel gray value after contrast stretching;
    第二对比度拉伸单元,用于对第一对比度拉伸单元输出的灰度图中的每个像素的灰度值进行第二灰度直方图统计,并根据统计结果对像素做第二灰度变换得到对比度拉伸后的像素灰度值,其中,所述第二灰度变换不同于所述第一灰度变换。The second contrast stretching unit is used to perform a second grayscale histogram statistics on the grayscale value of each pixel in the grayscale image output by the first contrast stretching unit, and perform a second grayscale on the pixel according to the statistical result The pixel gray value after the contrast stretching is obtained by transformation, wherein the second gray scale transformation is different from the first gray scale transformation.
  60. 如权利要求59所述的可移动载体,其特征在于,所述第一灰度 变换为线性灰度变换或非线性灰度变换;和/或所述第二灰度变换均为线性灰度变换或非线性灰度变换。The removable carrier according to claim 59, wherein the first grayscale transformation is a linear grayscale transformation or a nonlinear grayscale transformation; and/or the second grayscale transformation is a linear grayscale transformation Or non-linear gray scale transformation.
  61. 如权利要求59所述的可移动载体,其特征在于,所述拉伸单元还包括:The movable carrier according to claim 59, wherein the stretching unit further comprises:
    线性拉伸单元,用于对频率分离后的灰度图的灰度值进行线性变换;Linear stretching unit, used to linearly transform the gray value of the gray image after frequency separation;
    所述第一对比度拉伸单元还用于对经线性变换的每个像素的灰度值进行第一灰度直方图统计。The first contrast stretching unit is also used to perform first gray-scale histogram statistics on the gray-scale value of each pixel that has been linearly transformed.
  62. 如权利要求56所述的可移动载体,其特征在于,所述平场校正单元用于利用参考灰度图对所述图像传感器采集的灰度图进行平场校正,所述参考灰度图为所述图像传感器在所述快门关闭时采集的灰度图。The movable carrier according to claim 56, wherein the flat field correction unit is configured to use a reference gray image to perform flat field correction on the gray image collected by the image sensor, and the reference gray image is A grayscale image collected by the image sensor when the shutter is closed.
  63. 如权利要求56所述的可移动载体,其特征在于,所述坏点校正单元包括:The removable carrier according to claim 56, wherein the dead pixel correction unit comprises:
    静态坏点校正子单元,用于对所述图像传感器的静态坏点对应的像素进行校正;和/或The static dead pixel correction subunit is used to correct the pixels corresponding to the static dead pixels of the image sensor; and/or
    动态坏点校正子单元,用于检测所述图像传感器的动态坏点,并对所述动态坏点对应的像素进行校正。The dynamic dead pixel correction subunit is used to detect the dynamic dead pixels of the image sensor and correct the pixels corresponding to the dynamic dead pixels.
  64. 如权利要求56所述的可移动载体,其特征在于,所述降噪单元还包括:The movable carrier according to claim 56, wherein the noise reduction unit further comprises:
    固定模式噪声去除单元,用于去除坏点校正单元或时域降噪单元输出的灰度图中固定模式噪声。The fixed pattern noise removal unit is used to remove fixed pattern noise in the grayscale output from the bad pixel correction unit or the time domain noise reduction unit.
  65. 如权利要求64所述的可移动载体,其特征在于,所述降噪单元还包括:The movable carrier according to claim 64, wherein the noise reduction unit further comprises:
    空域降噪单元,用于对去除了时域噪声以及固定模式噪声后的灰度图进行空域随机噪声的去除。The spatial noise reduction unit is used to remove the spatial random noise of the gray image after removing the time domain noise and the fixed pattern noise.
  66. 如权利要求56所述的可移动载体,其特征在于,所述降噪单元还包括:The movable carrier according to claim 56, wherein the noise reduction unit further comprises:
    空域降噪单元,用于对时域降噪单元输出的灰度图进行空域随机噪声的去除。The spatial noise reduction unit is used to remove the spatial random noise of the gray image output by the time domain noise reduction unit.
  67. 如权利要求56所述的可移动载体,其特征在于,所述时域降噪单元用于去除时域随机单点噪声、时域随机行噪声、时域随机列噪声中的 至少一种。The movable carrier according to claim 56, wherein the time domain noise reduction unit is used to remove at least one of time domain random single point noise, time domain random row noise, and time domain random column noise.
  68. 如权利要求56所述的可移动载体,其特征在于,所述校正单元还包括:The movable carrier according to claim 56, wherein the correction unit further comprises:
    非线性校正单元,平场校正后的灰度图经所述非线性校正单元进行非线性校正后,由所述坏点校正单元进行坏点校正。A non-linear correction unit, after the flat-field corrected gray image is non-linearly corrected by the non-linear correction unit, the bad pixel correction unit performs bad pixel correction.
  69. 如权利要求56所述的可移动载体,其特征在于,还包括:The movable carrier according to claim 56, characterized in that it further comprises:
    场景分析反馈单元,用于对所述对比度拉伸后的灰度图进行场景信息分析,并将分析结果反馈至图像校正单元、降噪单元、拉伸单元至少其中之一,针对不同场景自适应地调整所述图像校正单元、降噪单元、拉伸单元至少其中之一的参数。The scene analysis feedback unit is used to analyze the scene information of the gray image after the contrast stretching, and feed back the analysis result to at least one of the image correction unit, the noise reduction unit, and the stretching unit, and is adaptive to different scenes The parameters of at least one of the image correction unit, noise reduction unit, and stretching unit are adjusted.
  70. 如权利要求56所述的可移动载体,其特征在于,还包括:The movable carrier according to claim 56, characterized in that it further comprises:
    伪彩映射单元,用于将对比度拉伸后的灰度图映射为YUV色彩图。The pseudo-color mapping unit is used to map the grayscale image after the contrast stretching to the YUV color image.
  71. 如权利要求70所述的可移动载体,其特征在于,还包括:The movable carrier according to claim 70, further comprising:
    转码单元,用于在伪彩映射单元输出为YUV444色彩图时,将YUV444色彩图转码为YUV422或YUV420的色彩图。The transcoding unit is used to transcode the YUV444 color map into a YUV422 or YUV420 color map when the pseudo color mapping unit outputs a YUV444 color map.
  72. 如权利要求56所述的可移动载体,其特征在于,还包括:The movable carrier according to claim 56, characterized in that it further comprises:
    信号接收及控制单元,用于接收并保存所述图像传感器采集的灰度图,并对所述图像传感器进行控制,以及对所述图像传感器的动态范围进行检查及校正。The signal receiving and control unit is used to receive and save the grayscale image collected by the image sensor, to control the image sensor, and to check and correct the dynamic range of the image sensor.
  73. 如权利要求56或64所述的可移动载体,其特征在于,还包括:The movable carrier according to claim 56 or 64, further comprising:
    测温单元,用于根据所述时域降噪单元或固定模式噪声去除单元输出的灰度图确定绝对温度。The temperature measurement unit is used to determine the absolute temperature according to the grayscale image output by the time domain noise reduction unit or the fixed pattern noise removal unit.
  74. 如权利要求56所述的可移动载体,其特征在于,所述可移动载体包括:无人机、无人车、无人船的至少一种。The movable carrier according to claim 56, wherein the movable carrier includes at least one of an unmanned aerial vehicle, an unmanned vehicle, and an unmanned ship.
  75. 一种计算机可读存储介质,其特征在于,所述计算机可读存储介质存储有计算机程序,所述计算机程序被处理器执行时使所述处理器实现如权利要求1-18任一项所述的图像处理方法。A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements any one of claims 1-18 Image processing method.
PCT/CN2019/127865 2019-12-24 2019-12-24 Image processing method and apparatus, imaging device, and movable carrier WO2021127972A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201980050132.6A CN112544064A (en) 2019-12-24 2019-12-24 Image processing method and device, imaging equipment and movable carrier
PCT/CN2019/127865 WO2021127972A1 (en) 2019-12-24 2019-12-24 Image processing method and apparatus, imaging device, and movable carrier

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/127865 WO2021127972A1 (en) 2019-12-24 2019-12-24 Image processing method and apparatus, imaging device, and movable carrier

Publications (1)

Publication Number Publication Date
WO2021127972A1 true WO2021127972A1 (en) 2021-07-01

Family

ID=75013446

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/127865 WO2021127972A1 (en) 2019-12-24 2019-12-24 Image processing method and apparatus, imaging device, and movable carrier

Country Status (2)

Country Link
CN (1) CN112544064A (en)
WO (1) WO2021127972A1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116128750B (en) * 2022-12-21 2024-02-09 浙江黑卡电气有限公司 Fire-fighting enhancement imaging method, device, computer equipment and storage medium
CN117437151B (en) * 2023-12-21 2024-03-08 成都市晶林科技有限公司 Pseudo-color mapping method for noise suppression

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104574330A (en) * 2013-10-18 2015-04-29 中国航天科工集团第三研究院第八三五八研究所 Digital detail enhancing method capable of coping with HDR (high dynamic range) infrared images
CN105787883A (en) * 2014-12-19 2016-07-20 南京理工大学 Wide dynamic image device and algorithm
CN106454216A (en) * 2016-11-02 2017-02-22 南京理工大学 Night driving system based on uncooled infrared and low-light-level fusion
CN109348216A (en) * 2018-11-28 2019-02-15 凌云光技术集团有限责任公司 A kind of combination treatment method of bad point detection peace field calibration
US20190306413A1 (en) * 2018-03-29 2019-10-03 Realtek Semiconductor Corporation Image processing device and image processing method

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5148989B2 (en) * 2007-12-27 2013-02-20 イーストマン コダック カンパニー Imaging device
CN103875235B (en) * 2011-06-10 2018-10-12 菲力尔系统公司 Nonuniformity Correction for infreared imaging device
CN103077547A (en) * 2012-11-22 2013-05-01 中国科学院自动化研究所 CT (computerized tomography) on-line reconstruction and real-time visualization method based on CUDA (compute unified device architecture)
CN103217397A (en) * 2013-01-23 2013-07-24 山西省电力公司晋城供电分公司 SF6 gas detection method based on infrared image processing
CN105827999B (en) * 2016-05-04 2019-01-22 吴礼刚 Infrared image acquisition and processing system based on FPGA
CN106254794A (en) * 2016-08-16 2016-12-21 成都市和平科技有限责任公司 A kind of infrared video transmission system and method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104574330A (en) * 2013-10-18 2015-04-29 中国航天科工集团第三研究院第八三五八研究所 Digital detail enhancing method capable of coping with HDR (high dynamic range) infrared images
CN105787883A (en) * 2014-12-19 2016-07-20 南京理工大学 Wide dynamic image device and algorithm
CN106454216A (en) * 2016-11-02 2017-02-22 南京理工大学 Night driving system based on uncooled infrared and low-light-level fusion
US20190306413A1 (en) * 2018-03-29 2019-10-03 Realtek Semiconductor Corporation Image processing device and image processing method
CN109348216A (en) * 2018-11-28 2019-02-15 凌云光技术集团有限责任公司 A kind of combination treatment method of bad point detection peace field calibration

Also Published As

Publication number Publication date
CN112544064A (en) 2021-03-23

Similar Documents

Publication Publication Date Title
US11218630B2 (en) Global tone mapping
US10007967B2 (en) Temporal and spatial video noise reduction
US8941761B2 (en) Information processing apparatus and information processing method for blur correction
US9124811B2 (en) Apparatus and method for processing image by wide dynamic range process
US9544505B2 (en) Image processing apparatus for synthesizing images based on a plurality of exposure time periods and image processing method thereof
Kim et al. A novel approach for denoising and enhancement of extremely low-light video
KR20110032344A (en) High dynamic range imaging apparatus and method
JP6312487B2 (en) Image processing apparatus, control method therefor, and program
KR20090078583A (en) Method and system for processing for low light level image
US8780215B2 (en) Apparatus and method for processing an image to correct image distortion caused by a hand shake
KR102106537B1 (en) Method for generating a High Dynamic Range image, device thereof, and system thereof
WO2019022812A9 (en) Multiplexed high dynamic range images
KR102127306B1 (en) Movement detecting apparatus and movement detecting method
US9055223B2 (en) Digital image stabilization method and imaging device using the same
CN105841821B (en) The Nonuniformity Correction devices and methods therefor without baffle based on calibration
US7889942B2 (en) Dynamic range compensation-dependent noise reduction
US9589339B2 (en) Image processing apparatus and control method therefor
WO2019104047A1 (en) Global tone mapping
US11272146B1 (en) Content adaptive lens shading correction method and apparatus
KR101874738B1 (en) Apparatus and method for generating hdr image from ldr image through image processing
WO2021127972A1 (en) Image processing method and apparatus, imaging device, and movable carrier
JP5861924B2 (en) Imaging device
US20230120604A1 (en) System and Method for Maximizing Inference Accuracy using Recaptured Datasets
JP2011100204A (en) Image processor, image processing method, image processing program, imaging apparatus, and electronic device
KR101778461B1 (en) A method of reducing noise of low light environment and an apparatus therefor

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19957607

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19957607

Country of ref document: EP

Kind code of ref document: A1