CN114187196A - Self-adaptive multi-integral time infrared image sequence optimization method - Google Patents
Self-adaptive multi-integral time infrared image sequence optimization method Download PDFInfo
- Publication number
- CN114187196A CN114187196A CN202111439718.7A CN202111439718A CN114187196A CN 114187196 A CN114187196 A CN 114187196A CN 202111439718 A CN202111439718 A CN 202111439718A CN 114187196 A CN114187196 A CN 114187196A
- Authority
- CN
- China
- Prior art keywords
- image
- infrared
- sequence
- optimal
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000005457 optimization Methods 0.000 title abstract description 9
- 238000011156 evaluation Methods 0.000 claims abstract description 29
- 230000011218 segmentation Effects 0.000 claims abstract description 23
- 230000006870 function Effects 0.000 claims abstract description 19
- 238000003384 imaging method Methods 0.000 claims abstract description 18
- 238000003331 infrared imaging Methods 0.000 claims abstract description 11
- 238000012216 screening Methods 0.000 claims abstract description 8
- 230000005855 radiation Effects 0.000 claims description 32
- 238000013441 quality evaluation Methods 0.000 claims description 22
- 230000004927 fusion Effects 0.000 claims description 17
- 230000010354 integration Effects 0.000 claims description 13
- 230000000694 effects Effects 0.000 claims description 8
- 230000004044 response Effects 0.000 claims description 8
- 230000003044 adaptive effect Effects 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000012163 sequencing technique Methods 0.000 claims description 4
- 230000005587 bubbling Effects 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 2
- 230000015572 biosynthetic process Effects 0.000 claims 1
- 238000003786 synthesis reaction Methods 0.000 claims 1
- 238000001931 thermography Methods 0.000 description 4
- 238000011160 research Methods 0.000 description 3
- 238000010187 selection method Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000016776 visual perception Effects 0.000 description 2
- 201000004569 Blindness Diseases 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000010977 jade Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000004445 quantitative analysis Methods 0.000 description 1
- 238000005057 refrigeration Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/94—Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/741—Circuitry for compensating brightness variation in the scene by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Radiation Pyrometers (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a self-adaptive multi-integral time infrared image sequence optimization method, and belongs to the technical field of high-dynamic infrared imaging. The implementation method of the invention comprises the following steps: dividing the sequence image based on the region growing points, respectively searching optimal functions of a normal-temperature target and a strong-radiation target for the divided image, respectively obtaining local optimal images from the input sequence image according to the optimal functions of the normal-temperature target and the strong-radiation target, keeping image details as much as possible, enabling the normal-temperature target to be imaged clearly by adopting a high-integration-time exposure mode, and enabling the strong-radiation target to be imaged clearly by adopting a low-integration-time exposure mode; screening out a global optimal image from the input sequence image based on the gray information evaluation index, and eliminating the discontinuity of a local optimal image selected based on a region growing point segmentation method; the local optimal image and the global optimal image are fused to obtain the infrared high dynamic range HDR image, the infrared high dynamic range HDR image is subjected to enhanced display, and the imaging quality in a high dynamic scene is improved.
Description
Technical Field
The invention relates to a self-adaptive multi-integral time infrared image sequence optimization method, and belongs to the technical field of high-dynamic infrared imaging.
Background
In an actual Infrared application scene, when strong radiation targets such as sun, flame or interfering bomb and the like exist together with a normal temperature target, the radiation brightness span is large, and for an existing 14-bit A/D refrigeration Infrared Focal Plane (IRFPA) thermal imaging system, even if the nonlinear S effect of the response of a detector is not considered, the equivalent dynamic range is only about 84dB and is far smaller than the radiation difference of a natural scene with radiation interference. No matter how the imaging parameters of the system are adjusted, an underexposure phenomenon or an overexposure phenomenon is caused, all details of a scene cannot be captured at one time, which causes very adverse effects on a plurality of target detection and identification tasks, and a High-Dynamic Range (HDR) imaging method needs to be adopted to adapt to effective imaging of a full-radiation scene.
Currently, methods adopted by HDR infrared thermal imaging include: 1) based on pixel a/D technology. The method designs on-chip pixel level AD on each pixel respectively, realizes HDR imaging with high bit width and low noise, is an advanced IRFPA design method, but the detector process is complex, the development difficulty is high, the cost at the present stage is also high, and the NETD is low, so that the scene dynamic range obtained by adopting 18-bit AD is still difficult to meet the requirement of an actual HDR scene. 2) A superframe-based variable integration time imaging technique. The method adopts periodic cycle variable integration time ultrahigh frame frequency imaging on the basis of the existing IRFPA, expands the Low-Dynamic Range (LDR) images with different integration times into HDR thermal imaging by fusing the Low-Dynamic Range (LDR) images with different integration times, completely adopts a digital program control mode, does not need to increase a mechanical structure, and has higher imaging frame frequency, so that for infrared signals with lower visible light deflection, an infrared objective lens generally adopts a larger relative aperture to pursue the detection of long distance and weak targets, and for scenes with strong radiation interference, the HDR imaging is realized by adopting a variable integration time method.
Unlike typical visible light images, infrared thermal images are typically low in contrast, noisy, unclear in detail, and relatively concentrated in gray scale distribution. The variable integration time method has the following problems: 1) in the previous research, the selection of the fused image of the HDR infrared thermal image has certain blindness, and usually, according to prior knowledge, the change of the response value and the exposure degree are roughly judged by combining the LDR image obtained by the acquisition software, and two to three LDR images are selected from the image sequence for fusion, or a certain weight calculation method is used to fuse all the acquired image sequences. But inaccuracies in the manual selection, false positives, noise and gray non-uniformity due to blind spots all affect the LDR image selection from "optimal". 2) Due to the limitation of real-time performance, when an imaging device shoots a scene, the imaging device is difficult to adjust to parameters with moderate image exposure, and a learner expects to adjust the exposure parameters of the next frame of image according to the current scene frame information. Some researches try to compare the current image full-pixel gray level average value with a preset optimal gray level average value, the method is simple in processing flow, the image quality can be optimized to a certain degree, but the local gray level difference is ignored, and the overall effect is poor. The self-adaptive exposure method based on the gray level histogram (fixed block theory, fuzzy logic calculation weight theory, pixel saturation threshold value theory, scene region segmentation theory and the like) developed subsequently and the method for adjusting the integration time by utilizing other image information (such as image evaluation indexes of Sobel operator extraction edge information, information entropy, gradient difference and the like) research also provide an integration time self-adaptive adjustment approach for evaluating the front-end LDR output image sequence, and the methods have the problems of weak self-adaptability or single evaluation index and unsatisfactory imaging.
In summary, it is necessary to develop a method capable of adaptively selecting a multi-integral infrared image sequence to improve the quality of a high dynamic fusion image. How to define the image selection standard, solve the exposure quality problem caused by strong radiation of a high dynamic scene, and carry out validity verification on the fusion image effect is a key problem worthy of being solved.
Disclosure of Invention
Aiming at the problem that the dynamic range in a high dynamic scene cannot be effectively and comprehensively captured, the invention aims to provide a self-adaptive multi-integration time infrared image sequence preference method, aiming at the response requirement of the high dynamic range in the high dynamic scene, inputting a sequence image, segmenting the sequence image based on a region growing point, respectively searching optimal functions of a normal temperature target and a strong radiation target for the segmented image, respectively obtaining local optimal images from the input sequence image according to the optimal functions of the normal temperature target and the strong radiation target, wherein the local optimal images comprise a local optimal image (high exposure image) of the normal temperature target and a local optimal image (low exposure image) of the strong radiation target, so as to keep the image details as much as possible, and the local optimal image of the normal temperature target is called as the high exposure image by adopting a high integration time exposure mode, the strong radiation target can be imaged clearly by adopting a low integral time exposure mode, so that the local optimal image of the strong radiation target is called a low exposure image; in addition, a global optimal image is comprehensively screened from the input sequence images based on the gray information evaluation index, and the discontinuity of a local optimal image selected based on a region growing point segmentation method is eliminated; and the local optimal image and the global optimal image are fused to obtain the infrared high dynamic range HDR image, and the infrared high dynamic range HDR image is subjected to enhanced display, namely, the dynamic range of infrared imaging is effectively expanded under the condition that the dynamic range of an infrared imaging device is limited, and the imaging quality in a high dynamic scene is improved.
The purpose of the invention is realized by the following technical scheme.
The invention discloses a self-adaptive multi-integral time infrared image sequence preference method, which comprises the following steps:
and acquiring infrared sequence images under different integration times aiming at the same high dynamic response scene.
Inputting the collected sequence image to a region growing point segmentation module, segmenting the sequence image based on a region growing point method, respectively searching optimal functions of a normal-temperature target and a strong-radiation target for the segmented image, respectively obtaining local optimal images from the input sequence image according to the optimal functions of the normal-temperature target and the strong-radiation target, wherein the local optimal images comprise a local optimal image (high-exposure image) of the normal-temperature target and a local optimal image (low-exposure image) of the strong-radiation target, so that the image details are kept as much as possible, the normal-temperature target can be clearly imaged by adopting a high-integration time exposure mode, the local optimal image of the normal-temperature target is called as the high-exposure image, the strong-radiation target can be clearly imaged by adopting a low-integration time exposure mode, and the local optimal image of the strong-radiation target is called as the low-exposure image. And parameters for segmentation in the sequence image based on the region growing points are changed, the gray scale multiple relation between the normal temperature region and the strong radiation region is calculated through a gray scale histogram, and the gray scale multiple relation is set as a threshold segmentation value of the current scene.
And inputting the acquired sequence images to a global image selection module based on a gray information evaluation index, and comprehensively and quantitatively screening global optimal images containing more information from the input sequence based on the gray information evaluation index to eliminate the discontinuity of local optimal images selected by a region growing point segmentation method.
The infrared high dynamic range HDR image is obtained by fusing the normal-temperature target local optimal image, the strong-radiation target local optimal image and the global optimal image, self-adaptive infrared image sequence preferred selection is realized, the fusion effect error of artificially and empirically selected images is reduced, namely the dynamic range of infrared imaging is effectively expanded under the condition that the dynamic range of an infrared imaging device is limited, and the imaging quality under a high dynamic response scene is improved.
Parameters for segmentation in the segmentation sequence image are changed based on the region growing points, and the gray scale multiple relation between the normal temperature region and the strong radiation region is calculated through a gray scale histogram and is set as a segmentation threshold value mu of the current scene. Preferably, the threshold parameter μ for segmentation is obtained according to equation (1) for different high dynamic range infrared scenes:
wherein M is the number of histogram intervals; n is the number of pixels of each interval; x and y are respectively the positions corresponding to the maximum N value in the intervals (1, M/2) and (M/2, M); edges is a one-dimensional tuple.
The image sequence input based on region growing point segmentation is specifically realized by the following steps: presetting initial seed points, calculating the relation between the gray level of each pixel in the sequence image and the seed points, if the gray level of the pixel is more than the gray level of the seed points which is multiplied by mu, dividing the pixel into a strong radiation area, and marking the strong radiation area by using '1'; if the gray scale of the pixel is less than μ times the gray scale of the seed point, the pixel is divided into normal temperature regions and marked with "0", so as to realize the target division based on the gray scale threshold μ, which is shown in formula (2).
Respectively searching the optimal functions of the normal temperature target and the strong radiation target for the segmented image, and obtaining a local optimal image from the input sequence image according to the optimal functions of the normal temperature target and the strong radiation target, wherein the specific implementation method comprises the following steps: and (3) respectively searching the segmented images for the optimal functions of the normal-temperature target and the strong-radiation target according to the formula (3), namely respectively obtaining the gray average values of two areas of the multi-integral sequence image, and respectively calculating the image closest to the medium gray level according to the bubbling sequence.
Wherein, IxIs the x-th frame image of the input;
is the gray level average value of the calculated area; w is the bit width of the original data; range is the image region L to be calculatedrangeAnd Hrange。
And obtaining local optimal images, namely a local optimal image (high exposure image) of the normal-temperature target and a local optimal image (low exposure image) of the strong-radiation target from the input sequence images according to an optimal function formula (3) of the normal-temperature target and the strong-radiation target.
Preferably, a globally optimal image containing more information is screened quantitatively from the input sequence comprehensively on the basis of the gray information evaluation index, and the specific implementation method is as follows:
establishing an index system for evaluating the image quality of an input image sequence, wherein the index system comprises an information entropy H (X) image quality evaluation index and a gradient difference SobelImage quality evaluation index, gray level mean value MeanIImage quality evaluation index and gray mean square error StdIAnd (4) image quality evaluation indexes.
And quantitatively screening the input sequence comprehensively based on the gray information evaluation index.
For the information entropy h (x) image quality evaluation index: the image is an m × n pixel matrix, and each pixel corresponds to a gray scale between 0 and 255. The probability of each gray level is defined as Pi:
wherein alpha isiIs the sum of the pixels for each gray level. The information entropy is represented as:
wherein w is the system bit width.
For the gradient difference SobelImage quality evaluation index:
wherein, IxAnd IyFor the original image respectively at hx，hyThe convolved image. Sobel operator hx，hyComprises the following steps:
for the mean value M of the gray scaleeanIImage quality evaluation index calculationThe method comprises the following steps:
mean square error for gray scale StdIThe image quality evaluation index calculation mode is as follows:
the calculation formula of the comprehensive evaluation measurement standard is
Mulrank＝SUM(Hrank,Srank,Mrank,Stdrank) (10)
Wherein M isulrankThe image comprehensive ranking is calculated for the sum of the ranking of each evaluation index; hrankRanking the information entropy H; srankAs image gradient difference SobelRank of (2); mrankAs mean value M of image gray scaleeanIRank of (2); stdrankIs the mean square error S of the imagetdIThe rank name of (c).
The entropy H (X) and the gradient difference S of the image quality evaluation index information are calculated according to the formula (10)obelMean value of gray scale MeanIMean square error of gray scale StdIAnd carrying out comprehensive sequencing, and selecting the image with the highest comprehensive ranking to realize that each evaluation index is in a relatively optimal state on the whole, namely quantitatively screening out the globally optimal image containing more information from the input sequence.
Preferably, the fused infrared high dynamic range HDR image is subjected to enhanced display based on a detail enhancement cascade algorithm.
Has the advantages that:
1. the invention discloses a self-adaptive multi-integration time infrared image sequence optimization method, which comprises the steps of inputting sequence images aiming at high dynamic response requirements in a high dynamic scene, segmenting the sequence images based on region growing points, respectively searching optimal functions of a normal temperature target and a strong radiation target for the segmented images, and obtaining local optimal images from the input sequence images according to the optimal functions of the normal temperature target and the strong radiation target, wherein the local optimal images comprise a normal temperature target local optimal image (high exposure image) and a strong radiation target local optimal image (low exposure image), and the detail of the images can be kept as much as possible. In order to enable the normal-temperature target to be imaged clearly, a high-integration-time exposure mode is required, so that the local optimal image of the normal-temperature target is called a high-exposure image; in order to enable the strong radiation target to be imaged clearly, a low-integration-time exposure mode is required, so that the local optimal image of the strong radiation target is called a low-exposure image.
2. The invention discloses a self-adaptive multi-integral time infrared image sequence optimization method, which is characterized in that a global image selection module based on gray level information evaluation indexes evaluates information entropy and gradient difference S of image quality evaluation indexes according to an established global image comprehensive evaluation measurement standardobelMean value of gray scale MeanIMean square error of gray scale StdIAnd carrying out comprehensive sequencing, selecting the image with the highest comprehensive ranking, converting infrared multi-integral time image selection from qualitative judgment to quantitative analysis, and selecting a global image with the best overall imaging quality from an infrared image sequence, namely quantitatively screening a global optimal image containing more information from an input sequence, so that the overall scene has continuity.
3. The invention discloses a self-adaptive multi-integral time infrared image sequence optimization method, which is used for self-adaptively selecting a segmentation threshold according to different high dynamic range infrared scenes.
4. The invention discloses a self-adaptive multi-integral time infrared image sequence optimization method, which realizes the optimal selection of self-adaptive infrared image sequence selection by fusing the normal-temperature target local optimal image, the strong-radiation target local optimal image and the global optimal image, reduces the fusion effect error of manually and empirically selected images, obtains an infrared High Dynamic Range (HDR) image, effectively expands the dynamic range of infrared imaging under the condition that the dynamic range of an infrared imaging device is limited, and improves the imaging quality under a high dynamic range scene.
5. According to the self-adaptive multi-integral time infrared image sequence preference method disclosed by the invention, a quantitative standardized process can provide a guidance method for variable integral infrared imaging, self-adaptive infrared image sequence preference selection is realized, the fusion effect error of manually and empirically selected images is reduced, the dynamic range of high-dynamic-range infrared images is further expanded, and the imaging quality of a thermal infrared imager is improved.
Drawings
Fig. 1 is a flowchart of an infrared image multi-integration time adaptive selection method based on gray scale information evaluation and region growing point gray scale segmentation.
Fig. 2 shows an experimental medium wave infrared thermal imaging system, fig. 2(a) shows a refrigerated medium wave infrared imager ImageIR8355, and fig. 2(b) shows a refrigerated medium wave thermal imager Jade.
Fig. 3 is a set of scene graph gray distribution conditions. Fig. 3(a) shows the infrared image gray scale distribution of the scene at 100 clock cycles, fig. 3(b) shows the infrared image gray scale distribution of the scene at 5000 clock cycles, and fig. 3(c) shows the infrared image gray scale distribution of the scene at 30000 clock cycles.
Fig. 4 is a sequence of images obtained based on segmentation of the growing point region. Fig. 4(a) shows the scene image division situation in fig. 3 at 100 clock cycles, fig. 4(b) shows the scene image division situation in fig. 3 at 500 clock cycles, fig. 4(c) shows the scene image division situation in fig. 3 at 1000 clock cycles, fig. 4(d) shows the scene image division situation in fig. 3 at 2500 clock cycles, fig. 4(e) shows the scene image division situation in fig. 3 at 5000 clock cycles, fig. 4(f) shows the scene image division situation in fig. 3 at 10000 clock cycles, fig. 4(g) shows the scene image division situation in fig. 3 at 15000 clock cycles, and fig. 4(h) shows the scene image division situation in fig. 3 at 30000 clock cycles.
Fig. 5 is an LDR image of scene 1 and its HDR fused image. Fig. 5(a) is an infrared enhanced image of scene 1 at 50 clock cycles, fig. 5(b) is an infrared enhanced image of scene 1 at 1050 clock cycles, and fig. 5(c) is an infrared enhanced image of scene 1 at 1150 clock cycles.
Fig. 6 is an LDR image of scene 2 and its HDR fused image. Fig. 6(a) is an infrared enhanced image of scene 2 at 400 clock cycles, fig. 6(b) is an infrared enhanced image of scene 2 at 900 clock cycles, and fig. 6(c) is an infrared enhanced image of scene 2 at 1050 clock cycles.
Fig. 7 is an LDR image of scene 3 and its HDR fused image. Fig. 7(a) is an infrared-enhanced image of scene 3 at 500 clock cycles, fig. 7(b) is an infrared-enhanced image of scene 3 at 1500 clock cycles, and fig. 7(c) is an infrared-enhanced image of scene 3 at 30000 clock cycles.
Fig. 8 is an LDR image of scene 4 and its HDR fused image. Fig. 8(a) is an infrared-enhanced image of scene 4 at 1500 clock cycles, and fig. 8(b) is an infrared-enhanced image of scene 4 at 16000 clock cycles. Since the local optimal image and the global optimal image of the normal-temperature target screened out in the scene are consistent, fig. 8(c) is fig. 8 (b).
Fig. 5 to 8(a), (b), and (c) are respectively a normal-temperature target local optimal image, a strong-radiation target local optimal image, and a global optimal image screened out based on the method of the present invention in the current scene.
Fig. 5 to 8 (d) are HDR fusion maps obtained by human experience, and the selection method is: and judging the exposure condition of the image by calculating the relation between the average gray level value and the medium gray level of the image, and selecting three images close to the medium gray level for fusion.
Fig. 5 to 8 (e) show HDR fusion maps obtained by image evaluation indexes, and the selection method is: and calculating the information entropy, the gradient difference, the gray mean value and the gray mean square error of each image in the sequence, and respectively selecting the optimal images corresponding to the four indexes for fusion.
Fig. 5-8 (f) are HDR fusion diagrams implemented by the method of the present invention.
Fig. 9 shows index evaluation results of the HDR images obtained by fusing the four sets of scenes shown in fig. 5 to 8.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings in combination with specific steps.
As shown in fig. 1, the adaptive multi-integration time infrared image sequence optimization method disclosed in this embodiment includes the following detailed steps:
the method comprises the following steps: the thermal imager in fig. 2 was used for image acquisition. And adjusting the integration time of the thermal infrared imager, collecting scene image sequences under different integration times, inputting images to software, and initializing parameters. Setting an initial threshold value mu to be 1.5, and taking a pixel point in a normal temperature area as a seed. Traversing the infrared image sequence, and performing the second step to the fifth step.
Step two: the threshold μ of the current scene is calculated. To input an image IxThe gray level of (2) is divided into equal-value intervals edges, and the number N of pixels whose gray level values fall in each interval is calculated. Taking the middle gray level as a gray dividing line, calculating the maximum value of the counting N forward, and determining the gray value corresponding to the maximum value as the gray value L of the normal temperature area of the frame imagegray(ii) a Calculating the maximum value of the count N, and determining the gray value H corresponding to the value as the gray value of the strong radiation area of the frame imagegray. The threshold μ can be obtained according to equation (1).
Step three: and performing region segmentation. The pixel value I (I, j) and the gray value I (I) of the seed point are compared0,j0) When the pixel value is less than the gray value of the mu times of the seed point, the pixel is classified into the seed area and is marked with '0' as the normal temperature point, otherwise, the pixel is marked with '1' as the high temperature point. The image is divided into two parts according to the formula (2), namely a normal temperature region LrangeAnd a region of intense radiation HrangeAs shown in fig. 4.
Step four: and optimizing the objective function. And acquiring the gray average value of the high-temperature area of the multi-integral sequence image, and calculating the image closest to the medium gray level according to the bubbling sequence to be used as the local optimal image (low-exposure image) of the strong radiation target. The normal temperature target local optimum image (high exposure image) is selected in the same manner.
Step five: and calculating four groups of image evaluation indexes and carrying out comprehensive evaluation. For the information entropy indexes, sorting the information entropy indexes from big to small; for the image gradient indexes, sorting the image gradient indexes from large to small according to absolute values; for the gray level mean index, sorting the gray level mean index according to the difference value from the middle gray level to the large gray level; and for the gray level mean square error index, sorting the indexes from large to small. And finally, obtaining the comprehensive ranking of all image evaluation indexes, and obtaining an image with the first comprehensive ranking as a global optimal image.
Step six: and (4) image fusion and enhanced display. And carrying out gray level nonlinear fusion and enhancement treatment on the local optimal image of the strong radiation target, the local optimal image of the normal temperature target and the global optimal image to form an 8-bit displayed HDR image.
Step seven: the HDR image obtained by the present invention was compared with other methods (d and e in fig. 5 to 8) involved in the present invention, and the image was evaluated by subjective evaluation and objective evaluation to verify the effectiveness of the present invention.
From the subjective analysis of fig. 5 to 8, the HDR image realized by the present embodiment can be clearly imaged in both the strong radiation and normal temperature regions, and has no distortion.
Because there is no accepted HDR infrared fusion image quality evaluation index at present, in order to evaluate the effectiveness of the self-adaptive multi-integral time infrared image sequence preference method disclosed in the embodiment, the invention also provides an objective evaluation index for evaluating the infrared image quality in a high dynamic range after fusion, and the image is objectively evaluated from three aspects of noise level, fidelity and visual perception quality by means of roughness (rho), fusion visual information fidelity (VIFF) and natural image quality evaluation index (NIQE). According to the objective analysis, the HDR image obtained by the method has low roughness (rho), namely low noise; the visual information fidelity (VIFF) is high, namely the fused image retains the visual information of the source image accurately; the natural image quality evaluation index (NIQE) is high, namely the visual perception quality of the image is good, and the image can be well matched with a natural scene.
The above detailed description is intended to illustrate the objects, aspects and advantages of the present invention, and it should be understood that the above detailed description is only exemplary of the present invention and is not intended to limit the scope of the present invention, and any modifications, equivalents, improvements and the like within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (6)
1. A self-adaptive multi-integral time infrared image sequence preference method is characterized by comprising the following steps: comprises the following steps of (a) carrying out,
acquiring infrared sequence images under different integration times aiming at the same high dynamic response scene;
inputting the collected sequence image to a region growing point segmentation module, segmenting the sequence image based on a region growing point method, respectively searching optimal functions of a normal-temperature target and a strong-radiation target for the segmented image, respectively obtaining local optimal images from the input sequence image according to the optimal functions of the normal-temperature target and the strong-radiation target, wherein the local optimal images comprise the local optimal image of the normal-temperature target and the local optimal image of the strong-radiation target, so as to keep image details as much as possible, the normal-temperature target can be clearly imaged by adopting a high-integration time exposure mode, so that the local optimal image of the normal-temperature target is called a high-exposure image, the strong-radiation target can be clearly imaged by adopting a low-integration time exposure mode, and the local optimal image of the strong-radiation target is called a low-exposure image; parameters for segmentation in the segmentation sequence image based on the region growing points are changed, the gray scale multiple relation between the normal temperature region and the strong radiation region is calculated through a gray scale histogram, and the gray scale multiple relation is set as a threshold segmentation value of the current scene;
inputting the collected sequence images to a global image selection module based on a gray information evaluation index, quantitatively screening global optimal images containing more information from the input sequence comprehensively based on the gray information evaluation index, and eliminating discontinuity of local optimal images selected based on a region growing point segmentation method;
the high dynamic range HDR infrared thermal image is obtained by fusing the normal-temperature target local optimal image, the strong-radiation target local optimal image and the global optimal image, self-adaptive infrared image sequence preference selection is realized, the fusion effect error of manually and empirically selected images is reduced, namely the dynamic range of the infrared imaging device is limited, the dynamic range of the infrared imaging is effectively expanded, and the imaging quality in a high dynamic response scene is improved.
2. The adaptive multi-integration time-infrared image sequence preference method of claim 1, characterized by: for different high dynamic range infrared scenes, a threshold parameter μ for segmentation is obtained according to equation (1):
wherein M is the number of histogram intervals; n is the number of pixels of each interval; x and y are respectively the positions corresponding to the maximum N value in the intervals (1, M/2) and (M/2, M); edges is a one-dimensional tuple.
3. The adaptive multi-integration time-infrared image sequence preference method of claim 2, characterized in that: dividing an input image sequence based on region growing points, wherein the specific implementation method comprises the steps of presetting initial seed points, calculating the relation between the gray level of each pixel in a sequence image and the seed points, if the gray level of the pixel is more than mu times of the gray level of the seed points, dividing the pixel into strong radiation regions and marking the regions with '1'; if the gray scale of the pixel is less than mu times of the gray scale value of the seed point, dividing the pixel into a normal temperature area, and marking the normal temperature area by using a '0' mark, so as to realize the target segmentation based on the gray scale threshold value mu, wherein the representation condition is shown as a formula (2);
4. an adaptive multi-integration time-infrared image sequence preference method as defined in claim 3, wherein: respectively searching optimal functions of a normal temperature target and a strong radiation target for the segmented image, and obtaining a local optimal image from the input sequence image according to the optimal functions of the normal temperature target and the strong radiation target, wherein the specific realization method is that the optimal functions of the normal temperature target and the strong radiation target are respectively searched for the segmented image according to the formula (3), namely, the gray average values of two areas of the multi-integral sequence image are respectively obtained, and the image closest to the medium gray level is respectively calculated according to bubbling sequencing;
wherein, IxIs the x-th frame image of the input;
is the gray level average value of the calculated area; w is the bit width of the original data; range is the image region L to be calculatedrangeAnd Hrange；
And obtaining local optimal images, namely the local optimal image of the normal-temperature target and the local optimal image of the strong-radiation target from the input sequence images according to the optimal function formula (3) of the normal-temperature target and the strong-radiation target.
5. The adaptive multi-integration time-infrared image sequence preference method of claim 4, wherein: the overall optimal image containing more information is screened out quantitatively from the input sequence comprehensively based on the gray information evaluation index, the specific realization method is as follows,
establishing an index system for evaluating the image quality of an input image sequence, wherein the index system comprises an information entropy H (X) image quality evaluation index and a gradient difference SobelImage quality evaluation index, gray level mean value MeanIImage quality evaluation index and gray mean square error StdIAn image quality evaluation index;
quantitatively screening from the input sequence based on gray information evaluation index synthesis;
for the information entropy h (x) image quality evaluation index: the image is an mxn pixel matrix, and each pixel corresponds to a gray scale between 0 and 255; the probability of each gray level is defined as Pi:
wherein alpha isiIs the sum of the pixels for each gray level; the information entropy is represented as:
wherein w is a system bit width;
for the gradient difference SobelImage quality evaluation index:
wherein, IxAnd IyRespectively corresponding to h for the original imagex，hyA convolved image; sobel operator hx，hyComprises the following steps:
for the mean value M of the gray scaleeanIThe image quality evaluation index calculation mode is as follows:
mean square error for gray scale StdIThe image quality evaluation index calculation mode is as follows:
the calculation formula of the comprehensive evaluation measurement standard is
Mulrank＝SUM(Hrank,Srank,Mrank,Stdrank) (10)
Wherein M isulrankThe image comprehensive ranking is calculated for the sum of the ranking of each evaluation index; hrankRanking the information entropy H; srankAs image gradient difference SobelRank of (2); mrankAs mean value M of image gray scaleeanIRank of (2); stdrankIs the mean square error S of the imagetdIRank of (2);
the entropy H (X) and the gradient difference S of the image quality evaluation index information are calculated according to the formula (10)obelMean value of gray scale MeanIMean square error of gray scale StdIAnd carrying out comprehensive sequencing, and selecting the image with the highest comprehensive ranking to realize that each evaluation index is in a relatively optimal state on the whole, namely quantitatively screening out the globally optimal image containing more information from the input sequence.
6. The adaptive multi-integration time-infrared image sequence preference method of claim 5, characterized in that: and performing enhanced display on the fused infrared high dynamic range HDR image based on a detail enhanced cascade algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111439718.7A CN114187196B (en) | 2021-11-30 | 2021-11-30 | Self-adaptive multi-integration time infrared image sequence optimizing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111439718.7A CN114187196B (en) | 2021-11-30 | 2021-11-30 | Self-adaptive multi-integration time infrared image sequence optimizing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114187196A true CN114187196A (en) | 2022-03-15 |
| CN114187196B CN114187196B (en) | 2024-06-14 |
Family
ID=80602975
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111439718.7A Active CN114187196B (en) | 2021-11-30 | 2021-11-30 | Self-adaptive multi-integration time infrared image sequence optimizing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114187196B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100053376A1 (en) * | 2008-09-01 | 2010-03-04 | Canon Kabushiki Kaisha | Image processing apparatus and method |
| WO2011093994A1 (en) * | 2010-01-27 | 2011-08-04 | Thomson Licensing | High dynamic range (hdr) image synthesis with user input |
| CN110035239A (en) * | 2019-05-21 | 2019-07-19 | 北京理工大学 | One kind being based on the more time of integration infrared image fusion methods of gray scale-gradient optimizing |
| CN110889806A (en) * | 2019-11-19 | 2020-03-17 | 常州工学院 | Adaptive gain image enhancement method based on fractional order multi-scale entropy fusion |
| CN110910323A (en) * | 2019-11-19 | 2020-03-24 | 常州工学院 | Underwater image enhancement method based on self-adaptive fractional order multi-scale entropy fusion |
| CN113674186A (en) * | 2021-08-02 | 2021-11-19 | 中国科学院长春光学精密机械与物理研究所 | Image synthesis method and device based on self-adaptive adjustment factor |
-
2021
- 2021-11-30 CN CN202111439718.7A patent/CN114187196B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100053376A1 (en) * | 2008-09-01 | 2010-03-04 | Canon Kabushiki Kaisha | Image processing apparatus and method |
| WO2011093994A1 (en) * | 2010-01-27 | 2011-08-04 | Thomson Licensing | High dynamic range (hdr) image synthesis with user input |
| CN110035239A (en) * | 2019-05-21 | 2019-07-19 | 北京理工大学 | One kind being based on the more time of integration infrared image fusion methods of gray scale-gradient optimizing |
| CN110889806A (en) * | 2019-11-19 | 2020-03-17 | 常州工学院 | Adaptive gain image enhancement method based on fractional order multi-scale entropy fusion |
| CN110910323A (en) * | 2019-11-19 | 2020-03-24 | 常州工学院 | Underwater image enhancement method based on self-adaptive fractional order multi-scale entropy fusion |
| CN113674186A (en) * | 2021-08-02 | 2021-11-19 | 中国科学院长春光学精密机械与物理研究所 | Image synthesis method and device based on self-adaptive adjustment factor |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114187196B (en) | 2024-06-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108492262B (en) | No-ghost high-dynamic-range imaging method based on gradient structure similarity | |
| CN101739672B (en) | A kind of histogram equalizing method based on sub-regional interpolation and device | |
| CN111292264A (en) | Image high dynamic range reconstruction method based on deep learning | |
| CN110163807B (en) | Low-illumination image enhancement method based on expected bright channel | |
| CN110346116B (en) | Scene illumination calculation method based on image acquisition | |
| WO2019056549A1 (en) | Image enhancement method, and image processing device | |
| Moriwaki et al. | Hybrid loss for learning single-image-based HDR reconstruction | |
| CN107370910A (en) | Minimum surround based on optimal exposure exposes set acquisition methods | |
| CN113177438A (en) | Image processing method, apparatus and storage medium | |
| CN113432723B (en) | Image processing method, system and computer system for weakening stray radiation | |
| CN113628169B (en) | Infrared image automatic focusing evaluation method, system and medium based on pseudo color | |
| Purohit et al. | Enhancing the surveillance detection range of image sensors using HDR techniques | |
| CN111881924B (en) | Dark-light vehicle illumination identification method combining illumination invariance and short-exposure illumination enhancement | |
| CN114187196B (en) | Self-adaptive multi-integration time infrared image sequence optimizing method | |
| Venkatesh et al. | Image Enhancement and Implementation of CLAHE Algorithm and Bilinear Interpolation | |
| CN108564534A (en) | A kind of picture contrast method of adjustment based on retrieval | |
| CN113781368B (en) | Infrared imaging device based on local information entropy | |
| Teutsch et al. | An evaluation of objective image quality assessment for thermal infrared video tone mapping | |
| US20220114736A1 (en) | Method and system for motion segmentation | |
| CN114998173A (en) | High dynamic range imaging method for space environment based on local area brightness adjustment | |
| Wang et al. | Raw infrared image enhancement via an inverted framework based on infrared basic prior | |
| EP3054668B1 (en) | Method and device for processing a video stream | |
| TW202217741A (en) | Image processing method | |
| Soma et al. | Fast and memory efficient de-hazing technique for real-time computer vision applications | |
| CN112348771B (en) | Imaging consistency evaluation method based on wavelet transformation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |