CN113538314B - Four-band common-optical-axis photoelectric imaging platform and image fusion processing method thereof - Google Patents
Four-band common-optical-axis photoelectric imaging platform and image fusion processing method thereof Download PDFInfo
- Publication number
- CN113538314B CN113538314B CN202110837286.9A CN202110837286A CN113538314B CN 113538314 B CN113538314 B CN 113538314B CN 202110837286 A CN202110837286 A CN 202110837286A CN 113538314 B CN113538314 B CN 113538314B
- Authority
- CN
- China
- Prior art keywords
- image
- wave infrared
- band
- long
- fusion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 64
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000007499 fusion processing Methods 0.000 title claims abstract description 24
- 230000004927 fusion Effects 0.000 claims abstract description 109
- 238000012545 processing Methods 0.000 claims abstract description 56
- 238000009529 body temperature measurement Methods 0.000 claims abstract description 53
- 238000001514 detection method Methods 0.000 claims abstract description 15
- 238000003331 infrared imaging Methods 0.000 claims description 75
- 230000000875 corresponding effect Effects 0.000 claims description 68
- 230000003287 optical effect Effects 0.000 claims description 52
- 230000005855 radiation Effects 0.000 claims description 24
- 238000003333 near-infrared imaging Methods 0.000 claims description 23
- 230000003595 spectral effect Effects 0.000 claims description 13
- 238000012937 correction Methods 0.000 claims description 9
- 238000007781 pre-processing Methods 0.000 claims description 9
- 230000004044 response Effects 0.000 claims description 8
- 238000002834 transmittance Methods 0.000 claims description 8
- 230000005540 biological transmission Effects 0.000 claims description 6
- 230000002708 enhancing effect Effects 0.000 claims description 6
- 230000000007 visual effect Effects 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 3
- 230000002596 correlated effect Effects 0.000 claims description 3
- 230000005693 optoelectronics Effects 0.000 claims description 2
- 238000012546 transfer Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000009977 dual effect Effects 0.000 description 3
- 238000003672 processing method Methods 0.000 description 3
- 238000004861 thermometry Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000002349 favourable effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 229910000530 Gallium indium arsenide Inorganic materials 0.000 description 1
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004297 night vision Effects 0.000 description 1
- 238000012634 optical imaging Methods 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 238000005057 refrigeration Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Radiation Pyrometers (AREA)
- Studio Devices (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
Abstract
The invention discloses a four-band common-axis photoelectric imaging platform and an image fusion processing method thereof, belonging to the technical field of photoelectric detection and image processing. The invention provides a four-band common-axis photoelectric imaging platform and a multi-band image fusion processing method, wherein the four-band common-axis photoelectric imaging platform can be used for simultaneously acquiring four-band images in a target scene, and the four-band common-axis photoelectric imaging platform is combined with the four-band image fusion processing method to complete multi-band image processing of four-band color fusion, three-band fusion and two-band fusion, so that complementation and enrichment of multi-band information are fully utilized, and the target detection and identification efficiency is improved. In addition, the invention also discloses a medium-wave infrared and long-wave infrared dual-band colorimetric temperature measurement method which can be applied in parallel with a multi-band image fusion processing method, and the application range of the four-band common-axis photoelectric imaging platform is expanded.
Description
Technical Field
The invention relates to a four-band common-axis photoelectric imaging platform and an image fusion processing method thereof, belonging to the technical field of photoelectric detection and image processing.
Background
With the development of the multiband imaging detector technology, the multiband fusion imaging technology has wide application prospect. Visible light + near infrared, short wave infrared, medium wave infrared and long wave infrared are currently the most commonly used photoimaging bands. The visible light and near infrared band has scene detail texture information consistent with the visual habit of human eyes; under the condition of normal temperature, the short wave infrared mainly belongs to reflection information imaging, the transmission characteristic of the short wave infrared is superior to that of visible light, and the information difference of a target scene and the visible light wave band are obviously different; the medium wave infrared and the long wave infrared mainly reflect the heat radiation information of the target scene, have obvious differences with the reflection characteristics, and have respective characteristics, so that the medium wave infrared and the long wave infrared are commonly used for temperature measurement, armored target identification and the like. Therefore, the four wave bands all contain different aspects of the characteristics of the target scene, namely, the spectrum information of the target scene is added, and the effective utilization or fusion of the information becomes an effective means for improving the reconnaissance capability of the photoelectric imaging system.
At present, on one hand, the performance of a visible light and near InfraRed imaging detector by a silicon-based CCD/CMOS is continuously improved, on the other hand, the technology of a short-wave, medium-wave and long-wave InfraRed Focal plane detector array (InfraRed Focal PLANE ARRAY, IRFPA) is also rapidly developed, and the technology of a double-color and even multi-color InfraRed Focal plane detector is put to practical use, namely the technology of a multi-band imaging sensor is already or gradually mature. However, the research on the multiband image processing method is insufficient at present, although the research on the double-band image fusion method has been developed, except for some simple weighted superposition algorithms, the natural sense color fusion algorithm based on color transfer has small calculated amount, is convenient for real-time processing, and is applied to visible light and infrared color night vision equipment; the approach of deep learning in recent years also has been developing in terms of image fusion. Because the multiband imaging detector is complex in system and high in price on the whole, and links such as multichannel registration and image preprocessing are required to be matched, effective development of a multiband image processing algorithm is limited, namely, a multiband photoelectric imaging platform also limits research and development of the processing algorithm.
Disclosure of Invention
The invention aims to provide a four-band common-axis photoelectric imaging platform and a multi-band image fusion processing method, wherein the four-band common-axis photoelectric imaging platform can be used for simultaneously acquiring four-band images in a target scene, and the four-band common-axis photoelectric imaging platform is combined with the four-band image fusion processing method to complete multi-band image processing of four-band color fusion, three-band fusion and dual-band fusion, so that the complementation and enrichment of multi-band information are fully utilized, and the target detection and identification efficiency is improved. In addition, the invention also discloses a medium-wave infrared and long-wave infrared dual-band colorimetric temperature measurement method which can be applied in parallel with a multi-band image fusion processing method, and the application range of the four-band common-axis photoelectric imaging platform is expanded.
The four wave bands refer to visible light, near infrared, short wave infrared, medium wave infrared and long wave infrared. Three bands refer to optional three bands of the four bands described above. Dual band refers to two bands of the four bands described above.
The aim of the invention is achieved by the following technical scheme.
The invention discloses a four-band common-axis photoelectric imaging platform which comprises a multiband window, a four-band common-axis optical system, a visible light and near infrared imaging assembly, a short wave infrared imaging assembly, a medium wave infrared imaging assembly, a long wave infrared imaging assembly and a digital video processing board.
The four-band common-axis optical system comprises a first beam splitter, a second beam splitter and a third beam splitter.
The digital video processing board is used for carrying out corresponding image processing according to an image processing algorithm.
The imaging assembly includes an objective lens and a cartridge.
The included angle between the first beam splitter and the cross section of the multiband window is 45 degrees, the included angle between the second beam splitter and the first beam splitter is 90 degrees, and the included angle between the third beam splitter and the first beam splitter is 0 degree, so that the four-band coaxial optical system is realized. In order to avoid the loss of the view field and ensure that the light beams passing through the beam splitters are not overlapped, the lower end point of the second beam splitter needs to be above the upper end point of the first beam splitter, and the right end point of the third beam splitter needs to be on the left side of the left end point of the first beam splitter.
The four-band common-axis photoelectric imaging platform disclosed by the invention realizes optical registration of four-band images by adjusting a four-band common-axis optical system, a visible light and near infrared imaging assembly, a short wave infrared imaging assembly, a medium wave infrared imaging assembly and a long wave infrared imaging assembly.
The method for adjusting the optical registration is realized by the following steps.
Registration step one: coarse adjustment. The visible light and near infrared imaging assembly, the short wave infrared imaging assembly, the medium wave infrared imaging assembly, the long wave infrared imaging assembly, the first beam splitter, the second beam splitter and the third beam splitter are approximately adjusted to the specified positions.
Registering: and fine tuning the long-wave infrared imaging assembly to enable the optical axis of the long-wave infrared imaging assembly to be fixed after being perpendicular to the multiband window. And (3) taking the long-wave infrared imaging assembly as a reference, and after finishing the second registration step, adjusting other optical devices through the third registration step, the fourth registration step, the fifth registration step and the sixth registration step, so as to finish the registration work of the other optical devices.
Registering step three: taking a long-wave infrared image acquired by a long-wave infrared imaging assembly as a reference, observing a short-wave infrared image acquired by the short-wave infrared imaging assembly and a gray level superposition image of the long-wave infrared image, and adjusting a z-axis rotation angle and a pitching rotation angle of the short-wave infrared imaging assembly and the first beam splitterRegistering the short wave infrared image and the long wave infrared image, namely determining the positions of the short wave infrared imaging assembly and the first beam splitter.
The gray level superposition image of the short wave infrared image and the long wave infrared image is 1/2 of the sum of the gray level values of the pixels corresponding to the short wave infrared image and the long wave infrared image.
Registering step four: observing the intermediate wave infrared image acquired by the intermediate wave infrared imaging assembly and the gray level superposition image of the long wave infrared image by taking the long wave infrared image acquired by the long wave infrared imaging assembly as a reference, and adjusting the z-axis rotation angle and the pitching rotation angle of the intermediate wave infrared imaging assembly and the second beam splitterRegistering the medium wave infrared image and the long wave infrared image, namely determining the positions of the medium wave infrared imaging component and the second beam splitter.
The gray level superposition image of the medium wave infrared image and the long wave infrared image is 1/2 of the sum of the gray level values of the pixels corresponding to the medium wave infrared image and the long wave infrared image.
Registering step five: observing visible light, near infrared images and gray level superposition images of the long-wave infrared images acquired by the visible light, near infrared imaging assembly by taking the long-wave infrared images acquired by the long-wave infrared imaging assembly as a reference, and adjusting the z-axis rotation angle and the pitching rotation angle of the visible light, near infrared imaging assembly and the third beam splitterAnd the visible light and near infrared images and the long-wave infrared images are aligned, namely the positions of the visible light and near infrared imaging assembly and the third beam splitter can be determined.
The gray level superposition image of the visible light, the near infrared image and the long wave infrared image is 1/2 of the sum of the gray level values of the pixels corresponding to the visible light, the near infrared image and the long wave infrared image.
Registering step six: and observing the visible light and near infrared images acquired by the visible light and near infrared imaging assembly, the short wave infrared images acquired by the short wave infrared imaging assembly, the medium wave infrared images acquired by the medium wave infrared imaging assembly and the gray superposition images of the long wave infrared images acquired by the long wave infrared imaging assembly to determine the complete registration of the images of the four wave bands.
The gray level superposition image of the visible light and the near infrared image, the short wave infrared image, the medium wave infrared image and the long wave infrared image is 1/4 of the sum of the gray level values of the pixels corresponding to the visible light and the near infrared image, the short wave infrared image, the medium wave infrared image and the long wave infrared image.
Incident radiation of a scene enters from a multiband window, after passing through a first beam splitter, radiation of medium-wave infrared and long-wave infrared wave bands enters a second beam splitter through transmission, and radiation of visible light, near infrared and short-wave infrared wave bands enters a third beam splitter through reflection. The middle-wave infrared band radiation is reflected by the second beam splitter and enters the middle-wave infrared component for focusing imaging, the long-wave infrared band radiation is transmitted by the second beam splitter and enters the long-wave infrared component for focusing imaging, the visible light and near-infrared band radiation is reflected by the third beam splitter and enters the visible light and near-infrared component for focusing imaging, and the short-wave infrared band radiation is transmitted by the third beam splitter and enters the short-wave infrared component for focusing imaging. And simultaneously acquiring four-band image information in the target scene through the four-band common-axis photoelectric imaging platform. And inputting the four-band image information into a digital video processing board according to application requirements. And the digital video processing board performs corresponding image processing according to the selected image processing algorithm. The four-band common-axis photoelectric imaging platform can be combined with the four-band image fusion processing method to complete multi-band image processing of four-band color fusion, three-band color fusion and double-band color fusion, the complementation and the enrichment of multi-band information are fully utilized, and the target detection and identification efficiency is improved.
The invention also discloses a multiband image fusion processing method, which comprises the following steps:
Image fusion step one: and simultaneously acquiring four-band image information in the target scene.
And a second image fusion step: preprocessing the four-band image information obtained in the first image fusion step to obtain preprocessed four-band image information. The pretreatment comprises the following steps: performing blind pixel correction on the short-wave infrared image, and performing non-uniformity correction on the medium-wave infrared image and the long-wave infrared image respectively; and respectively enhancing the short wave infrared image, the medium wave infrared image and the long wave infrared image.
And step three, image fusion: and (3) carrying out linear combination of the preprocessed four-band image information obtained in the image fusion step II in YUV space to obtain an initial color image of a corresponding band and corresponding parameters, wherein the corresponding parameters comprise a brightness channel Y i, a blue color difference component U i and a red color difference component V i of the fused image.
Wherein Vis, SWIR, MWIR and LWIR respectively represent visible light + near infrared, short wave infrared, medium wave infrared and long wave infrared images ;k1,k2,k3,k4,m1,m2,m3,m4,m5,m6,m7 and m 8 which are positive rational numbers and empirical values, and k 1+k2>k3+k4 can control the wave bands participating in fusion by adjusting the values of the 12 parameters, and correspondingly realize multi-band image processing of four-band image fusion, three-band image fusion and two-band image fusion, and fully utilize complementation and enrichment of multi-band information. U i and V i respectively correspond to blue color difference components and red color difference components, so that visible light, near infrared Vis and short wave infrared images can be reflected in a blue color difference channel, and medium wave infrared images and long wave infrared images are respectively reflected in a red color difference channel, so that a color image which is more in line with the visual characteristics of human eyes is obtained; y i is the brightness channel of the fusion image, namely the gray fusion result of the multiband image.
And step four, image fusion: and (3) substituting the corresponding parameters into the formula (2) according to the initial color image and the corresponding parameters of the corresponding wave bands obtained in the image fusion step three, namely transmitting the color of the reference image to the initial color image and the corresponding parameters, wherein the corresponding parameters comprise Y i、Ui、Vi.
Y o,Uo,Vo is each channel of YUV of the color fusion image obtained finally; σ T,Y,σT,U,σT,V and σ i,Y,σi,U,σi,V are standard deviations of the channels of the color reference image and the initial color image YUV, respectively; mu T,Y,μT,U,μT,V and mu i,Y,μi,U,μi,V are the mean of the channels of the color reference image and the initial color image YUV, respectively.
Image fusion step five: according to application requirements, the 12 parameters in the third step are adjusted to control the wave bands participating in fusion, and multi-band image processing corresponding to four-band color fusion, three-band color fusion and dual-band color fusion is realized according to the third and fourth image fusion steps, so that complementation and enrichment of multi-band information are fully utilized, and the target detection and identification efficiency is improved.
The four wave bands refer to visible light, near infrared, short wave infrared, medium wave infrared and long wave infrared. Three bands refer to optional three bands of the four bands described above. Two bands refer to two optional bands of the four bands.
Preferably, the multiband image fusion processing method is applied to the four-band common-axis photoelectric imaging platform, so that multiband image processing corresponding to four-band color fusion, three-band color fusion and double-band color fusion is realized, complementation and enrichment of multiband information are fully utilized, and target detection and recognition efficiency is improved.
Preferably, the four-band common-axis photoelectric imaging platform not only can be used for a multi-band image fusion processing method, but also can be used for multi-band image typical applications such as medium-wave infrared and long-wave infrared dual-band colorimetric temperature measurement, heterologous image registration and the like by selecting a medium-wave infrared and long-wave infrared dual-band colorimetric temperature measurement method and a heterologous image registration algorithm.
The invention also discloses a medium-wave infrared and long-wave infrared dual-band colorimetric temperature measurement method which can be used in parallel with a multi-band image fusion processing method, and the application range of the four-band common-axis photoelectric imaging platform is widened.
The medium-wave infrared and long-wave infrared dual-band colorimetric temperature measurement method comprises the following steps of:
The first temperature measurement step: and simultaneously acquiring medium-wave infrared and long-wave infrared image information in the target scene.
According to the response principle of the detector, the signal level output by the detector in two temperature measuring wave bands [ lambda min,λmax ] is shown as a formula (3).
Wherein R V (lambda) is the spectral response rate of the temperature measuring wave band detector; a is the area of the detector unit; epsilon (lambda) object spectral emissivity; d is the aperture of the optical system; f' is the focal length of the optical system; τ a (λ) is the atmospheric spectral transmittance; τ 0 (λ) is the optical system transmittance; m eb (λ, T) is planck's law; u (T) is the signal level output by the detector.
And a temperature measurement step II: and (3) preprocessing the medium-wave infrared image information and the long-wave infrared image information obtained in the temperature measurement step I to obtain preprocessed dual-band image information. The pretreatment comprises the following steps: respectively carrying out non-uniformity correction on the medium-wave infrared image and the long-wave infrared image; and respectively enhancing the medium-wave infrared image and the long-wave infrared image.
And a temperature measurement step III: because the gray value of a single pixel of the detector is positively correlated with the output signal level U (T) of the detector, the ratio of the pixel signal levels corresponding to the two detectors is equal to the ratio of the gray values of the pixels corresponding to the two detectors.
The temperature of the measured object is determined by using the ratio of the image element signal levels corresponding to the medium wave infrared and the long wave infrared obtained by the two detectors, the medium wave infrared and the long wave infrared are compared with the expression of the expression (3), the ratio result is shown as the expression (4),
U 1 (T) is the signal level output by the medium wave infrared detector; u 2 (T) is the signal level output by the long-wave infrared detector; q (T) is the ratio of the signal level output by the medium-wave infrared detector to the signal level output by the long-wave infrared detector.
The influence of the spectral emissivity epsilon (lambda) of the temperature measurement target and the spectral transmittance tau a (lambda) and tau 0 (lambda) on the temperature measurement accuracy in the transmission process can be greatly reduced by using the two-band colorimetric temperature measurement of the medium-wavelength infrared and the long-wavelength infrared, and the ratio of the two different-band signals is approximately judged to be a function related to the temperature.
And a temperature measurement step four: and (3) fitting a curve of the gray value ratio Q (T) of each corresponding pixel point of the middle-wavelength infrared image and the long-wavelength infrared image along with the change of the temperature T through blackbody calibration, and fitting an expression of a corresponding polynomial.
Temperature measurement step five: and (3) bringing the ratio Q (T) of the signal level output by the medium-wave infrared detector and the signal level output by the long-wave infrared detector obtained in the temperature measurement step III into a polynomial fitting expression in the temperature measurement step IV to obtain a corresponding temperature T, and obtaining a temperature image with the same resolution as the medium-wave infrared image and the long-wave infrared image.
The beneficial effects are that:
1. the four-band coaxial optical system coaxial optical imaging platform disclosed by the invention has the advantages that the included angle between the first beam splitter and the cross section of the multiband window is 45 degrees, the included angle between the second beam splitter and the first beam splitter is 90 degrees, and the included angle between the third beam splitter and the first beam splitter is 0 degree, so that the four-band coaxial optical system is realized. The sceneries in the view field can be accurately registered, and parallax does not exist for sceneries with different object distances after registration is completed. The common axis spectroscopic optical system is well suited to multi-channel imaging systems requiring strict registration. Meanwhile, the common-axis spectroscopic optical system can complete registration in an optical and mechanical mode without electronic registration, so that the view field and the image resolution of each channel are not lost.
2. According to the multiband image fusion processing method disclosed by the invention, based on a formula and a color transfer formula for carrying out linear combination on four-band image information in YUV space, the bands participating in fusion can be controlled by adjusting the parameter values in the formula of linear combination according to application requirements, multiband image processing corresponding to four-band color fusion, three-band color fusion and dual-band color fusion is realized, the complementation and enrichment of multiband information are fully utilized, and the target detection and recognition efficiency is improved. The color transfer is performed by using the YUV space, a large number of logarithms and exponential operations are reduced, the color space is the color space which is most favorable for real-time video processing of hardware, and the color transfer method has high reliability and robustness on the premise of ensuring the processing speed of an algorithm.
3. The invention discloses a dual-band colorimetric temperature measurement method of medium-wave infrared and long-wave infrared, which uses the signal ratio of a conventional medium-wave infrared detector and a conventional long-wave infrared detector as input, and calculates the temperature of a scene according to a polynomial fitting expression of the temperature calibrated by a black body along with the signal ratio of the medium-wave infrared detector and the long-wave infrared detector. The uncooled medium-wavelength and long-wavelength infrared focal plane detector is adopted, the work is stable, the cost is low, the medium-wavelength and long-wavelength infrared information and the temperature information of a scene can be obtained at the same time, and a special temperature measuring detector is not needed. The dual-band colorimetric temperature measurement can effectively reduce temperature measurement errors caused by attenuation in the infrared radiation propagation process due to different object emissivity. The dual-band colorimetric temperature measurement can be used for carrying out large-range and long-distance temperature measurement on a scene, and is also convenient for conveniently screening environmental targets with different temperatures in an image.
4. The four-band common-axis photoelectric imaging platform and the image processing method thereof can realize processing links such as multi-band image registration, multi-band image preprocessing, multi-band image fusion, dual-band colorimetric temperature measurement and the like, can simultaneously acquire four-band image information of a target scene, process the four-band image information, and provide an experimental platform for subsequent multi-band imaging processing algorithm research.
Drawings
FIG. 1 is a schematic diagram of a four-band common-axis optoelectronic imaging platform according to the present invention. Wherein: 1-a multi-band window; 2-four band common optical axis optical system; 3-a visible + near infrared imaging assembly; 4-a short wave infrared imaging assembly; 5-a medium wave infrared imaging assembly; 6-a long wave infrared imaging assembly; 7-digital video processing board.
Fig. 2 is a diagram of the SOLIWORKS structure of the four-band coaxial-axis photoelectric imaging platform according to the present invention.
FIG. 3 is a schematic view of a beam splitter micro-displacement adjustment bracket according to the present invention. Wherein: 8-Pitch rotation angleA fine tuning knob; 9-Pitch rotation angle/>Locking a screw; 10-z axis rotation angle rotation fine tuning knob; 11-z axis rotation angle rotation locking knob.
FIG. 4 is a flowchart of a four-band natural sense color fusion image algorithm of the invention.
Fig. 5 is a four-band image and a natural sense color fusion image thereof according to the present invention, wherein: (a) is a visible light+near infrared image, (b) is a short wave infrared image, (c) is a medium wave infrared image, (d) is a long wave infrared image, (e) is a color fusion image, and (f) is a gray fusion image.
FIG. 6 is a flow chart of a dual band colorimetric thermometry method of the present invention.
FIG. 7 is a dual band colorimetric thermometry image of the invention, wherein: (a) is a medium-wave infrared image, (b) is a long-wave infrared image, and (c) is a temperature image.
Detailed description of the preferred embodiments
For a better description of the objects and advantages of the present invention, the following description will be given with reference to the accompanying drawings and examples.
Example 1:
As shown in fig. 1 and fig. 2, the four-band coaxial-axis photoelectric imaging platform disclosed in this embodiment includes a multiband window 1, a four-band coaxial-axis optical system 2, a visible light+near infrared imaging component 3, a short wave infrared imaging component 4, a medium wave infrared imaging component 5, a long wave infrared imaging component 6, and a digital video processing board 7.
The four-band common optical axis optical system 2 comprises a first beam splitter 2.1, a second beam splitter 2.2 and a third beam splitter 2.3.
The digital video processing board 7 is used for performing corresponding image processing according to an image processing algorithm.
The imaging assembly includes an objective lens and a cartridge.
The included angle between the first beam splitter 2.1 and the cross section of the multiband window 1 is 45 degrees, the included angle between the second beam splitter 2.2 and the first beam splitter 2.1 is 90 degrees, and the included angle between the third beam splitter 2.3 and the first beam splitter 2.1 is 0 degree, so that the four-band coaxial optical system 2 is coaxial. In order to avoid the loss of the field of view, the light beams passing through the beam splitters cannot overlap, the lower end point of the second beam splitter 2.2 needs to be above the upper end point of the first beam splitter 2.1, and the right end point of the third beam splitter 2.3 needs to be on the left side of the left end point of the first beam splitter 2.1.
As shown in fig. 3, the four-band common-axis photoelectric imaging platform disclosed in the embodiment realizes optical registration of four-band images by adjusting the four-band common-axis optical system 2, the visible light+near infrared imaging assembly 3, the short-wave infrared imaging assembly 4, the medium-wave infrared imaging assembly 5 and the long-wave infrared imaging assembly 6.
The method for adjusting the optical registration is realized by the following steps.
Registration step one: coarse adjustment. The visible light + near infrared imaging module 3, the short wave infrared imaging module 4, the medium wave infrared imaging module 5, the long wave infrared imaging module 6, the first beam splitter 2.1, the second beam splitter 2.2 and the third beam splitter 2.3 are adjusted to the specified positions.
Registering: the long-wave infrared imaging assembly 6 is finely adjusted, so that the optical axis of the long-wave infrared imaging assembly 6 is fixed after being perpendicular to the multiband window 1. And (3) taking the long-wave infrared imaging assembly 6 as a reference, and after finishing the second registration step, adjusting other optical devices through the third, fourth, fifth and sixth steps to finish the registration work of the other optical devices.
Registering step three: based on the long-wave infrared image acquired by the long-wave infrared imaging assembly 6, observing the short-wave infrared image acquired by the short-wave infrared imaging assembly 4 and the gray level superposition image of the long-wave infrared image, and adjusting the z-axis rotation angle and the pitching rotation angle of the short-wave infrared imaging assembly 4 and the first beam splitter 2.1Registering the short wave infrared image with the long wave infrared image, namely determining the positions of the short wave infrared imaging assembly 4 and the first beam splitter 2.1.
The gray level superposition image of the short wave infrared image and the long wave infrared image is 1/2 of the sum of the gray level values of the pixels corresponding to the short wave infrared image and the long wave infrared image.
The z-axis rotation angle and the pitching rotation angle of the first beam splitter 2.1The structure is shown in fig. 3. The adjustable structure can be locked, and meanwhile, the registration precision and the mechanical structure strength are ensured.
Registering step four: based on the long-wave infrared image acquired by the long-wave infrared imaging assembly 6, observing the intermediate-wave infrared image acquired by the intermediate-wave infrared imaging assembly 5 and the gray level superposition image of the long-wave infrared image, and adjusting the z-axis rotation angle and the pitching rotation angle of the intermediate-wave infrared imaging assembly 5 and the second beam splitter 2.2The intermediate wave infrared image and the long wave infrared image are registered, namely the positions of the intermediate wave infrared imaging assembly 5 and the second beam splitter 2.2 are determined.
The gray level superposition image of the medium wave infrared image and the long wave infrared image is 1/2 of the sum of the gray level values of the pixels corresponding to the medium wave infrared image and the long wave infrared image.
The z-axis rotation angle and the pitching rotation angle of the second beam splitter 2.2The structure is shown in fig. 3. The adjustable structure can be locked, and meanwhile, the registration precision and the mechanical structure strength are ensured.
Registering step five: based on the long-wave infrared image acquired by the long-wave infrared imaging assembly 6, observing the visible light, the near-infrared image acquired by the visible light, the near-infrared imaging assembly 3 and the gray level superposition image of the long-wave infrared image, and adjusting the z-axis rotation angle and the pitching rotation angle of the visible light, the near-infrared imaging assembly 3 and the third beam splitter 2.3The visible light + near infrared image is aligned with the long wave infrared image, i.e. the positions of the visible light + near infrared imaging assembly 3 and the third beam splitter 2.3 are determined.
The gray level superposition image of the visible light, the near infrared image and the long wave infrared image is 1/2 of the sum of the gray level values of the pixels corresponding to the visible light, the near infrared image and the long wave infrared image.
The z-axis rotation angle and the pitching rotation angle of the third beam splitter 2.3The structure is shown in fig. 3. The adjustable structure can be locked, and meanwhile, the registration precision and the mechanical structure strength are ensured.
Registering step six: and observing the visible light and near infrared images acquired by the visible light and near infrared imaging assembly 3, the short wave infrared images acquired by the short wave infrared imaging assembly 4, the medium wave infrared images acquired by the medium wave infrared imaging assembly 5 and the gray level superposition images of the long wave infrared images acquired by the long wave infrared imaging assembly 6 to determine that the images of the four wave bands are completely registered.
The gray level superposition image of the visible light and the near infrared image, the short wave infrared image, the medium wave infrared image and the long wave infrared image is 1/4 of the sum of the gray level values of the pixels corresponding to the visible light and the near infrared image, the short wave infrared image, the medium wave infrared image and the long wave infrared image.
Scenes in the view field of the four-band imaging assembly can be accurately registered, and parallax does not exist for scenes with different object distances after registration is completed. The common axis spectroscopic optical system is well suited to multi-channel imaging systems requiring strict registration. Meanwhile, the common-axis spectroscopic optical system can complete registration in an optical and mechanical mode without electronic registration, so that the view field and the image resolution of each channel are not lost.
Incident radiation of a scene enters from the multiband window 1, after passing through the first beam splitter 2.1, radiation of middle-wave infrared and long-wave infrared wave bands enters the second beam splitter 2.2 through transmission, and radiation of visible light, near infrared and short-wave infrared wave bands enters the third beam splitter 2.3 through reflection. The middle-wave infrared band radiation is reflected by the second beam splitter 2.2 and enters the middle-wave infrared component for focusing imaging, the long-wave infrared band radiation is transmitted by the second beam splitter 2.2 and enters the long-wave infrared component for focusing imaging, the visible light and near-infrared band radiation is reflected by the third beam splitter 2.3 and enters the visible light and near-infrared component for focusing imaging, and the short-wave infrared band radiation is transmitted by the third beam splitter 2.3 and enters the short-wave infrared component for focusing imaging. And simultaneously acquiring four-band image information in the target scene through the four-band common-axis photoelectric imaging platform. The four-band image information is input to the digital video processing board 7 according to the application requirements. The digital video processing board 7 performs corresponding image processing according to the selected image processing algorithm. The four-band common-axis photoelectric imaging platform can be combined with the four-band image fusion processing method to complete multi-band image processing of four-band color fusion, three-band color fusion and double-band color fusion, the complementation and the enrichment of multi-band information are fully utilized, and the target detection and identification efficiency is improved.
The parameters of the medium-wave infrared and long-wave infrared objective lenses are 40mm in focal length, and F=1.0; the medium wave infrared and long wave infrared detector component is a non-refrigeration focal plane detector component LA6110 of the Kazak AiRui company, the pixel number is 640 multiplied by 512, the pixel spacing is 17 mu m, NETD is less than or equal to 60mK, the frame frequency is 50Hz, and the output video is a CameraLink digital video; the response wave band of the medium wave infrared detector is 3-14 mu m, and the response wave band of the long wave infrared detector is 8-14 mu m. The parameters of the visible light and near infrared objective lens are that the focal length is 12.5-75 mm, and the minimum F=1.2; the visible light detector component is a low-illumination CMOS movement P2101 of Kunshan Ruixiang micro-company, the target surface size is 1 inch, the pixel number is 1280 multiplied by 1024, the pixel size is 9.7 mu m multiplied by 9.7 mu m, the frame rate is 50Hz, the output video is a CameraLink digital video, and when the lens of F1.4 is used, clear imaging can be realized under the weak light condition of 1 multiplied by 10 -3 lx. The parameters of the short wave infrared objective lens are that the focal length is 12.5-75 mm, and the minimum F=1.2; the short wave infrared detector component is an InGaAs uncooled focal plane detector component GH-SWCL-15 of Shanxi national benefit phototechnology limited company, the pixel number is 640 multiplied by 512, the pixel spacing is 15 mu m, the frame frequency is 100Hz, and the output video is a CameraLink digital video.
The digital video image processing board adopts a high-speed digital signal processing board taking an FPGA (model Kintex-7) as a core, and has 4 paths of CameraLink digital video inputs and 2 paths of CameraLink digital video outputs. The image data which can be selectively output comprises a visible light and near infrared image sequence, a short wave infrared image sequence, a medium wave infrared image sequence, a long wave infrared image sequence, a visible light and near infrared and long wave infrared color fusion image sequence, a visible light and near infrared and long wave infrared gray scale fusion image sequence, a four-wave band image color fusion image sequence, any three-wave band image color fusion image sequence in four wave bands, any two-wave band image color fusion image sequence in four wave bands, and a medium wave infrared and long wave infrared dual-wave infrared temperature measurement image sequence.
The four-band common-axis photoelectric imaging platform and the image processing method thereof disclosed by the embodiment can realize processing links such as multi-band image registration, multi-band image preprocessing, multi-band image fusion, dual-band colorimetric temperature measurement and the like, can acquire four-band image information of a target scene at the same time, process the four-band image information, and provide an experimental platform for subsequent multi-band imaging processing algorithm research.
The four-band common-axis photoelectric imaging platform can also select a medium-wave infrared and long-wave infrared dual-band colorimetric temperature measurement algorithm and a heterogeneous image registration algorithm for multi-band image typical application such as medium-wave infrared and long-wave infrared dual-band colorimetric temperature measurement and heterogeneous image registration.
The embodiment also discloses a multiband image fusion processing method, a flow chart of which is shown in fig. 4, comprising the following steps:
Image fusion step one: and simultaneously acquiring four-band image information in the target scene.
And a second image fusion step: preprocessing the four-band image information obtained in the first image fusion step to obtain preprocessed four-band image information. The pretreatment comprises the following steps: performing blind pixel correction on the short-wave infrared image, and performing non-uniformity correction on the medium-wave infrared image and the long-wave infrared image respectively; and respectively enhancing the short wave infrared image, the medium wave infrared image and the long wave infrared image. The preprocessed images are shown in fig. 5 (a), 5 (b), 5 (c), and 5 (d).
And step three, image fusion: and (3) carrying out linear combination of the preprocessed four-band image information obtained in the image fusion step II in YUV space to obtain an initial color image of a corresponding band and corresponding parameters, wherein the corresponding parameters comprise a brightness channel Y i, a blue color difference component U i and a red color difference component V i of the fused image.
Wherein Vis, SWIR, MWIR and LWIR respectively represent visible light + near infrared, short wave infrared, medium wave infrared and long wave infrared images ;k1,k2,k3,k4,m1,m2,m3,m4,m5,m6,m7 and m 8 which are positive rational numbers and empirical values, and k 1+k2>k3+k4 can control the wave bands participating in fusion by adjusting the values of the 12 parameters, and correspondingly realize multi-band image processing of four-band image fusion, three-band image fusion and two-band image fusion, and fully utilize complementation and enrichment of multi-band information. U i and V i respectively correspond to blue color difference components and red color difference components, so that visible light, near infrared Vis and short wave infrared images can be reflected in a blue color difference channel, and medium wave infrared images and long wave infrared images are respectively reflected in a red color difference channel, so that a color image which is more in line with the visual characteristics of human eyes is obtained; y i is the brightness channel of the fusion image, namely the gray fusion result of the multiband image.
And step four, image fusion: and (3) substituting the corresponding parameters into the formula (6) according to the initial color image and the corresponding parameters of the corresponding wave bands obtained in the image fusion step three, namely transmitting the color of the reference image to the initial color image and the corresponding parameters, wherein the corresponding parameters comprise Y i、Ui、Vi.
Y o,Uo,Vo is each channel of YUV of the color fusion image obtained finally; σ T,Y,σT,U,σT,V and σ i,Y,σi,U,σi,V are standard deviations of the channels of the color reference image and the initial color image YUV, respectively; mu T,Y,μT,U,μT,V and mu i,Y,μi,U,μi,V are the mean of the channels of the color reference image and the initial color image YUV, respectively.
Image fusion step five: according to application requirements, the 12 parameters in the third step are adjusted to control the wave bands participating in fusion, and multi-band image processing corresponding to four-band color fusion, three-band color fusion and dual-band color fusion is realized according to the third and fourth image fusion steps, so that complementation and enrichment of multi-band information are fully utilized, and the target detection and identification efficiency is improved.
The four wave bands refer to visible light, near infrared, short wave infrared, medium wave infrared and long wave infrared. Three bands refer to optional three bands of the four bands described above. Two bands refer to two optional bands of the four bands.
The multiband image fusion processing method is applied to the four-band common-axis photoelectric imaging platform, achieves multiband image processing corresponding to four-band color fusion, three-band color fusion and double-band color fusion, fully utilizes complementation and enrichment of multiband information, and improves target detection and identification efficiency.
The color fusion result of the four-band image is shown in fig. 5 (e), and the gray fusion result of the four-band image is shown in fig. 5 (f).
According to the multiband image fusion processing method disclosed by the embodiment, based on a formula and a color transfer formula for linearly combining four-band image information in YUV space, the bands participating in fusion can be controlled by adjusting the parameter values in the formula of linear combination according to application requirements, multiband image processing corresponding to four-band color fusion, three-band color fusion and dual-band color fusion is realized, complementation and enrichment of multiband information are fully utilized, and target detection and recognition efficiency is improved. The color transfer is performed by using the YUV space, a large number of logarithms and exponential operations are reduced, the color space is the color space which is most favorable for real-time video processing of hardware, and the color transfer method has high reliability and robustness on the premise of ensuring the processing speed of an algorithm.
The embodiment also discloses a medium-wave infrared and long-wave infrared dual-band colorimetric temperature measurement algorithm which can be applied in parallel with a multi-band image fusion processing method, and the application range of the four-band common-axis photoelectric imaging platform is widened.
The flow chart of the method is shown in fig. 6, and the method comprises the following steps:
The first temperature measurement step: and simultaneously acquiring medium-wave infrared and long-wave infrared image information in the target scene.
According to the response principle of the detector, the signal level output by the detector in two temperature measuring wave bands [ lambda min,λmax ] is shown as a formula (7).
Wherein R V (lambda) is the spectral response rate of the temperature measuring wave band detector; a is the area of the detector unit; epsilon (lambda) object spectral emissivity; d is the aperture of the optical system; f' is the focal length of the optical system; τ a (λ) is the atmospheric spectral transmittance; τ 0 (λ) is the optical system transmittance; m eb (λ, T) is planck's law; u (T) is the signal level output by the detector.
And a temperature measurement step II: and (3) preprocessing the medium-wave infrared image information and the long-wave infrared image information obtained in the temperature measurement step I to obtain preprocessed dual-band image information. The pretreatment comprises the following steps: respectively carrying out non-uniformity correction on the medium-wave infrared image and the long-wave infrared image; and respectively enhancing the medium-wave infrared image and the long-wave infrared image. The preprocessed image is shown in fig. 7 (a) and 7 (b).
And a temperature measurement step III: because the gray value of a single pixel of the detector is positively correlated with the output signal level U (T) of the detector, the ratio of the pixel signal levels corresponding to the two detectors is equal to the ratio of the gray values of the pixels corresponding to the two detectors.
The temperature of the measured object is determined by using the ratio of the image element signal levels corresponding to the medium wave infrared and the long wave infrared obtained by the two detectors, the medium wave infrared and the long wave infrared are compared with the expression of the expression (7), the ratio result is shown as the expression (8),
U 1 (T) is the signal level output by the medium wave infrared detector; u 2 (T) is the signal level output by the long-wave infrared detector; q (T) is the ratio of the signal level output by the medium-wave infrared detector to the signal level output by the long-wave infrared detector.
The influence of the spectral emissivity epsilon (lambda) of the temperature measurement target and the spectral transmittance tau a (lambda) and tau 0 (lambda) on the temperature measurement accuracy in the transmission process can be greatly reduced by using the two-band colorimetric temperature measurement of the medium-wavelength infrared and the long-wavelength infrared, and the ratio of two different-band signals is approximately considered to be a function related to temperature.
And a temperature measurement step four: and (3) fitting a curve of the gray value ratio Q (T) of each corresponding pixel point of the middle-wavelength infrared image and the long-wavelength infrared image along with the change of the temperature T through blackbody calibration, and fitting an expression of a corresponding polynomial.
Temperature measurement step five: and (3) bringing the ratio Q (T) of the signal level output by the medium-wave infrared detector and the signal level output by the long-wave infrared detector obtained in the temperature measurement step III into a polynomial fitting expression in the temperature measurement step IV to obtain a corresponding temperature T, and obtaining a temperature image with the same resolution as the medium-wave infrared image and the long-wave infrared image.
The temperature image obtained by the dual-band colorimetric thermometry is shown in fig. 7 (c).
The embodiment discloses a dual-band colorimetric temperature measurement method of medium-wave infrared and long-wave infrared, which uses the signal ratio of a conventional medium-wave infrared detector and a conventional long-wave infrared detector as input, and calculates the temperature of a scene according to a polynomial fitting expression of the temperature calibrated by a black body along with the signal ratio of the medium-wave infrared detector and the long-wave infrared detector. The uncooled medium-wavelength and long-wavelength infrared focal plane detector is adopted, the work is stable, the cost is low, the medium-wavelength and long-wavelength infrared information and the temperature information of a scene can be obtained at the same time, and a special temperature measuring detector is not needed. The dual-band colorimetric temperature measurement can effectively reduce temperature measurement errors caused by attenuation in the infrared radiation propagation process due to different object emissivity. The dual-band colorimetric temperature measurement can be used for carrying out large-range and long-distance temperature measurement on a scene, and is also convenient for conveniently screening environmental targets with different temperatures in an image.
In summary, the above embodiments are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (4)
1. The method for adjusting the optical registration of the four-band common-optical-axis photoelectric imaging platform is characterized by comprising the following steps of: the four-band common-axis photoelectric imaging platform comprises a multiband window (1), a four-band common-axis optical system (2), a visible light and near infrared imaging component (3), a short wave infrared imaging component (4), a medium wave infrared imaging component (5), a long wave infrared imaging component (6) and a digital video processing board (7);
the four-band common-axis optical system (2) comprises a first beam splitter (2.1), a second beam splitter (2.2) and a third beam splitter (2.3);
The digital video processing board (7) is used for carrying out corresponding image processing according to an image processing algorithm;
The imaging assembly comprises an objective lens and a movement;
The included angle between the first beam splitter and the cross section of the multiband window (1) is 45 degrees, the included angle between the second beam splitter (2.2) and the first beam splitter (2.1) is 90 degrees, and the included angle between the third beam splitter (2.3) and the first beam splitter (2.1) is 0 degree, so that the four-band optical system (2) with the same optical axis is realized; in order to avoid losing the view field and ensure that the light beams passing through the beam splitters are not overlapped, the lower end point of the second beam splitter (2.2) needs to be above the upper end point of the first beam splitter (2.1), and the right end point of the third beam splitter (2.3) needs to be on the left side of the left end point of the first beam splitter (2.1);
The four-band image optical registration is realized by adjusting a four-band common optical axis optical system (2), a visible light and near infrared imaging component (3), a short wave infrared imaging component (4), a medium wave infrared imaging component (5) and a long wave infrared imaging component (6);
the method for adjusting the optical registration is realized by the following steps of;
registration step one: coarse adjustment; the visible light and near infrared imaging assembly (3), the short wave infrared imaging assembly (4), the medium wave infrared imaging assembly (5), the long wave infrared imaging assembly (6), the first beam splitter (2.1), the second beam splitter (2.2) and the third beam splitter (2.3) are approximately adjusted to a specified position;
Registering: fine-tuning the long-wave infrared imaging assembly (6) to enable the optical axis of the long-wave infrared imaging assembly (6) to be fixed after being perpendicular to the multiband window (1); after the second registration step is completed by taking the long-wave infrared imaging assembly (6) as a reference, other optical devices are adjusted through the third, fourth, fifth and sixth steps, and the registration of the other optical devices is completed;
Registering step three: taking a long-wave infrared image acquired by a long-wave infrared imaging assembly (6) as a reference, observing a short-wave infrared image acquired by a short-wave infrared imaging assembly (4) and a gray level superposition image of the long-wave infrared image, and adjusting a z-axis rotation angle and a pitching rotation angle of the short-wave infrared imaging assembly (4) and a first beam splitter (2.1) Registering the short wave infrared image and the long wave infrared image, namely determining the positions of the short wave infrared imaging assembly (4) and the first beam splitter (2.1);
The gray level superposition image of the short wave infrared image and the long wave infrared image is 1/2 of the sum of the gray level values of the pixels corresponding to the short wave infrared image and the long wave infrared image;
Registering step four: taking a long-wave infrared image acquired by a long-wave infrared imaging assembly (6) as a reference, observing a middle-wave infrared image acquired by a middle-wave infrared imaging assembly (5) and a gray level superposition image of the long-wave infrared image, and adjusting the z-axis rotation angle and the pitching rotation angle of the middle-wave infrared imaging assembly (5) and the second beam splitter (2.2) Registering the medium-wave infrared image and the long-wave infrared image, namely determining the positions of the medium-wave infrared imaging component (5) and the second beam splitting mirror (2.2);
The gray level superposition image of the medium-wave infrared image and the long-wave infrared image is 1/2 of the sum of the gray level values of the pixels corresponding to the medium-wave infrared image and the long-wave infrared image;
Registering step five: observing a visible light and near infrared image acquired by the visible light and near infrared imaging assembly (3) and a gray level superposition image of the long infrared image by taking a long infrared image acquired by the long infrared imaging assembly (6) as a reference, and adjusting the z-axis rotation angle and the pitching rotation angle of the visible light and near infrared imaging assembly (3) and the third beam splitter (2.3) The visible light and near infrared images and the long-wave infrared images are aligned, namely the positions of the visible light and near infrared imaging assembly (3) and the third beam splitter (2.3) can be determined;
The gray level superposition image of the visible light plus near infrared image and the long wave infrared image is 1/2 of the sum of the gray level values of the pixels corresponding to the visible light plus near infrared image and the long wave infrared image;
Registering step six: observing visible light and near infrared images acquired by a visible light and near infrared imaging assembly (3), short wave infrared images acquired by a short wave infrared imaging assembly (4), medium wave infrared images acquired by a medium wave infrared imaging assembly (5) and gray level superposition images of long wave infrared images acquired by a long wave infrared imaging assembly (6), and determining that the images of the four wave bands are completely registered;
the gray level superposition image of the visible light and the near infrared image, the short wave infrared image, the medium wave infrared image and the long wave infrared image is 1/4 of the sum of the gray level values of the pixels corresponding to the visible light and the near infrared image, the short wave infrared image, the medium wave infrared image and the long wave infrared image;
Incident radiation of a scene enters from a multiband window (1), after passing through a first beam splitter (2.1), radiation of middle-wave infrared and long-wave infrared bands enters a second beam splitter (2.2) through transmission, and radiation of visible light plus near infrared and short-wave infrared bands enters a third beam splitter (2.3) through reflection; the medium-wave infrared band radiation is reflected by the second beam splitter (2.2) and enters the medium-wave infrared component for focusing imaging, the long-wave infrared band radiation is transmitted by the second beam splitter (2.2) and enters the long-wave infrared component for focusing imaging, the visible light and near-infrared band radiation is reflected by the third beam splitter (2.3) and enters the visible light and near-infrared component for focusing imaging, and the short-wave infrared band radiation is transmitted by the third beam splitter (2.3) and enters the short-wave infrared component for focusing imaging; simultaneously acquiring four-band image information in a target scene through a four-band common-axis photoelectric imaging platform; inputting the four-band image information into a digital video processing board (7) according to application requirements; the digital video processing board (7) performs corresponding image processing according to the selected image processing algorithm; the four-band common-axis photoelectric imaging platform can be combined with the four-band image fusion processing method to complete multi-band image processing of four-band color fusion, three-band color fusion and double-band color fusion, the complementation and the enrichment of multi-band information are fully utilized, and the target detection and identification efficiency is improved.
2. A multiband image fusion processing method, using the four-band co-optical axis optoelectronic imaging platform in the optical registration adjustment method according to claim 1, characterized in that: comprises the following steps of the method,
Image fusion step one: simultaneously acquiring four-band image information in a target scene;
And a second image fusion step: preprocessing the four-band image information obtained in the first image fusion step to obtain preprocessed four-band image information; the pretreatment comprises the following steps: performing blind pixel correction on the short-wave infrared image, and performing non-uniformity correction on the medium-wave infrared image and the long-wave infrared image respectively; respectively enhancing the short wave infrared image, the medium wave infrared image and the long wave infrared image;
And step three, image fusion: performing linear combination of the preprocessed four-band image information obtained in the image fusion step II in YUV space to obtain an initial color image of a corresponding band and corresponding parameters, wherein the corresponding parameters comprise a brightness channel Y i, a blue color difference component U i and a red color difference component V i of the fused image;
Wherein Vis, SWIR, MWIR and LWIR respectively represent visible light, near infrared, short wave infrared, medium wave infrared and long wave infrared images ;k1,k2,k3,k4,m1,m2,m3,m4,m5,m6,m7 and m 8 which are positive rational numbers and empirical values, and k 1+k2>k3+k4 can control the wave bands participating in fusion by adjusting the values of the 12 parameters, and correspondingly realize multi-band image processing of four-band image fusion, three-band image fusion and two-band image fusion, and fully utilize complementation and enrichment of multi-band information; u i and V i respectively correspond to blue color difference components and red color difference components, so that visible light, near infrared Vis and short wave infrared images can be reflected in a blue color difference channel, and medium wave infrared images and long wave infrared images are respectively reflected in a red color difference channel, so that a color image which is more in line with the visual characteristics of human eyes is obtained; y i is the brightness channel of the fusion image, namely the gray fusion result of the multiband image;
and step four, image fusion: substituting the corresponding parameters into the step (2) according to the initial color image and the corresponding parameters of the corresponding wave bands obtained in the step three of image fusion, namely transmitting the color of the reference image to the initial color image and the corresponding parameters, wherein the corresponding parameters comprise Y i、Ui、Vi;
Y o,Uo,Vo is each channel of YUV of the color fusion image obtained finally; σ T,Y,σT,U,σT,V and σ i,Y,σi,U,σi,V are standard deviations of the channels of the color reference image and the initial color image YUV, respectively; mu T,Y,μT,U,μT,V and mu i,Y,μi,U,μi,V are the average value of each channel of the color reference image and the initial color image YUV respectively;
Image fusion step five: according to application requirements, the 12 parameters in the third step are adjusted to control the wave bands involved in fusion, and multi-band image processing corresponding to four-band color fusion, three-band color fusion and dual-band color fusion is realized according to the third and fourth image fusion steps, so that complementation and enrichment of multi-band information are fully utilized, and the target detection and recognition efficiency is improved;
The four wave bands refer to visible light, near infrared, short wave infrared, medium wave infrared and long wave infrared; three bands refer to optional three bands in the four bands; two bands refer to two optional bands of the four bands.
3. The multi-band image fusion processing method of claim 2, wherein: the multi-band image processing corresponding to the four-band color fusion, the three-band color fusion and the dual-band color fusion is realized, the complementation and the enrichment of multi-band information are fully utilized, and the target detection and identification efficiency is improved.
4. A method for measuring the temperature of a medium-wave infrared and long-wave infrared dual-band colorimetric temperature by using a four-band common-axis photoelectric imaging platform in the optical registration adjustment method as claimed in claim 1, which is characterized in that: comprises the following steps of the method,
The first temperature measurement step: simultaneously acquiring medium-wave infrared and long-wave infrared image information in a target scene;
According to the response principle of the detector, the signal level output by the detector in two temperature measuring wave bands [ lambda min,λmax ] is shown as formula (3);
Wherein R V (lambda) is the spectral response rate of the temperature measuring wave band detector; a is the area of the detector unit; epsilon (lambda) object spectral emissivity; d is the aperture of the optical system; f' is the focal length of the optical system; τ a (λ) is the atmospheric spectral transmittance; τ 0 (λ) is the optical system transmittance; m eb (λ, T) is planck's law; u (T) is the signal level output by the detector;
And a temperature measurement step II: preprocessing the medium-wave infrared image information and the long-wave infrared image information obtained in the temperature measurement step I to obtain preprocessed dual-band image information; the pretreatment comprises the following steps: respectively carrying out non-uniformity correction on the medium-wave infrared image and the long-wave infrared image; respectively enhancing the medium-wave infrared image and the long-wave infrared image;
And a temperature measurement step III: because the gray value of a single pixel of the detector is positively correlated with the output signal level U (T) of the detector, the ratio of the pixel signal levels corresponding to the two detectors is equal to the ratio of the pixel gray values corresponding to the two detectors;
the temperature of the measured object is determined by using the ratio of the image element signal levels corresponding to the medium wave infrared and the long wave infrared obtained by the two detectors, the result of the ratio is shown as a formula (4) by comparing the expression of the formula (3) of the medium wave infrared and the long wave infrared,
U 1 (T) is the signal level output by the medium wave infrared detector; u 2 (T) is the signal level output by the long-wave infrared detector; q (T) is the ratio of the signal level output by the medium-wave infrared detector to the signal level output by the long-wave infrared detector;
And a temperature measurement step four: fitting a curve of the gray value ratio Q (T) of each corresponding pixel point of the middle-wavelength infrared image and the long-wavelength infrared image along with the change of the temperature T through blackbody calibration and a corresponding polynomial fitting expression;
Temperature measurement step five: and (3) bringing the ratio Q (T) of the signal level output by the medium-wave infrared detector and the signal level output by the long-wave infrared detector obtained in the temperature measurement step III into a polynomial fitting expression in the temperature measurement step IV to obtain a corresponding temperature T, and obtaining a temperature image with the same resolution as the medium-wave infrared image and the long-wave infrared image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110837286.9A CN113538314B (en) | 2021-07-23 | 2021-07-23 | Four-band common-optical-axis photoelectric imaging platform and image fusion processing method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110837286.9A CN113538314B (en) | 2021-07-23 | 2021-07-23 | Four-band common-optical-axis photoelectric imaging platform and image fusion processing method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113538314A CN113538314A (en) | 2021-10-22 |
| CN113538314B true CN113538314B (en) | 2024-06-11 |
Family
ID=78120976
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110837286.9A Active CN113538314B (en) | 2021-07-23 | 2021-07-23 | Four-band common-optical-axis photoelectric imaging platform and image fusion processing method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113538314B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114239654B (en) * | 2021-12-17 | 2023-05-12 | 电子科技大学 | Temperature identification and classification method based on dual-band pyroelectric device |
| CN116248705B (en) * | 2022-11-29 | 2024-03-15 | 宜昌测试技术研究所 | Multichannel image transmission and processing system of miniature photoelectric pod |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103278916A (en) * | 2013-04-10 | 2013-09-04 | 北京理工大学 | Laser and middle- and long-wavelength infrared common-aperture three-band imaging system |
| CN108154493A (en) * | 2017-11-23 | 2018-06-12 | 南京理工大学 | A kind of pseudo- color blending algorithm of the dual-band infrared image based on FPGA |
| CN108415097A (en) * | 2017-12-29 | 2018-08-17 | 华中科技大学 | A kind of the collection of illustrative plates cooperative detection system and method for multiband infrared imaging |
| CN108419061A (en) * | 2017-02-10 | 2018-08-17 | 杭州海康威视数字技术股份有限公司 | Based on multispectral image co-registration equipment, method and imaging sensor |
| CN112180577A (en) * | 2020-09-25 | 2021-01-05 | 中国科学院西安光学精密机械研究所 | Visible light-short wave infrared-medium wave infrared-long wave infrared four-waveband optical system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9030660B2 (en) * | 2012-09-19 | 2015-05-12 | Raytheon Company | Multi-band imaging spectrometer |
| CN104501956B (en) * | 2014-12-30 | 2016-07-13 | 华中科技大学 | A kind of ultra wide wave band collection of illustrative plates association detection device and detection method |
-
2021
- 2021-07-23 CN CN202110837286.9A patent/CN113538314B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103278916A (en) * | 2013-04-10 | 2013-09-04 | 北京理工大学 | Laser and middle- and long-wavelength infrared common-aperture three-band imaging system |
| CN108419061A (en) * | 2017-02-10 | 2018-08-17 | 杭州海康威视数字技术股份有限公司 | Based on multispectral image co-registration equipment, method and imaging sensor |
| CN108154493A (en) * | 2017-11-23 | 2018-06-12 | 南京理工大学 | A kind of pseudo- color blending algorithm of the dual-band infrared image based on FPGA |
| CN108415097A (en) * | 2017-12-29 | 2018-08-17 | 华中科技大学 | A kind of the collection of illustrative plates cooperative detection system and method for multiband infrared imaging |
| CN112180577A (en) * | 2020-09-25 | 2021-01-05 | 中国科学院西安光学精密机械研究所 | Visible light-short wave infrared-medium wave infrared-long wave infrared four-waveband optical system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113538314A (en) | 2021-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113538314B (en) | Four-band common-optical-axis photoelectric imaging platform and image fusion processing method thereof | |
| US8766808B2 (en) | Imager with multiple sensor arrays | |
| US8451527B2 (en) | Limited F-cone light source | |
| CN113701885B (en) | Off-axis three-reflector full-spectrum polarized spectrum imaging detection device | |
| CN105043552B (en) | Colorimetric temperature measurement system is shown and scaling method | |
| CN104272162A (en) | Imaging optical system, imaging device | |
| Ren et al. | A method and results of color calibration for the Chang'e-3 terrain camera and panoramic camera | |
| Hirsh et al. | Hybrid dual-color MWIR detector for airborne missile warning systems | |
| Perić et al. | Analysis of SWIR imagers application in electro-optical systems | |
| GB2537070A (en) | Zoom objective and camera system | |
| KR102209218B1 (en) | Short Wave Infrared Camera Optical System for The Long Range Image Monitoring | |
| CN115683358A (en) | Radiation calibration device and calibration method based on refrigeration type infrared imaging system | |
| CN113325578B (en) | Optical system of photoelectric pod | |
| CN113029339B (en) | On-orbit multi-source-tracing spectral radiance calibration method for deep space detection imaging spectrometer | |
| CN109118460B (en) | Method and system for synchronously processing light-splitting polarization spectrum information | |
| Chen et al. | The design of integrated SWIR/visible optical lens | |
| CN114485939B (en) | Dynamic tracking and detecting device and method for ultra-wide band spectrum of remote sensing satellite | |
| CN113390508B (en) | Spectrum-space resolution image quality optimized short wave infrared imaging method and device | |
| CN115265786B (en) | Strong light polarization detection device capable of automatically adjusting exposure value and detection method thereof | |
| Schreer et al. | Dual-band camera system with advanced image processing capability | |
| Sun et al. | Research on a radiative correction method for the division of focal plane polarization in mid-infrared imaging system | |
| Toet et al. | TRICLOBS portable triband color lowlight observation system | |
| Куліченко et al. | Color-Index Determination of Leo Satellites Using Color Ip-Cameras | |
| CN116256706A (en) | Multi-wave-segment calibration mirror for radar calibration | |
| Zhang et al. | Non-Ge optics and low-cost electronics designs for LIR imagers |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |