CN113538314A - Four-waveband coaxial-axis photoelectric imaging platform and image fusion processing method thereof - Google Patents
Four-waveband coaxial-axis photoelectric imaging platform and image fusion processing method thereof Download PDFInfo
- Publication number
- CN113538314A CN113538314A CN202110837286.9A CN202110837286A CN113538314A CN 113538314 A CN113538314 A CN 113538314A CN 202110837286 A CN202110837286 A CN 202110837286A CN 113538314 A CN113538314 A CN 113538314A
- Authority
- CN
- China
- Prior art keywords
- wave infrared
- image
- long
- beam splitter
- fusion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 62
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000007499 fusion processing Methods 0.000 title claims abstract description 23
- 238000009529 body temperature measurement Methods 0.000 claims abstract description 56
- 230000003287 optical effect Effects 0.000 claims abstract description 55
- 238000012545 processing Methods 0.000 claims abstract description 54
- 238000001514 detection method Methods 0.000 claims abstract description 14
- 230000004927 fusion Effects 0.000 claims description 100
- 238000003331 infrared imaging Methods 0.000 claims description 75
- 230000000875 corresponding effect Effects 0.000 claims description 68
- 230000005855 radiation Effects 0.000 claims description 24
- 238000003333 near-infrared imaging Methods 0.000 claims description 23
- 230000003595 spectral effect Effects 0.000 claims description 16
- 238000012937 correction Methods 0.000 claims description 9
- 238000007781 pre-processing Methods 0.000 claims description 9
- 238000002834 transmittance Methods 0.000 claims description 9
- 230000005540 biological transmission Effects 0.000 claims description 7
- 230000002708 enhancing effect Effects 0.000 claims description 6
- 230000004044 response Effects 0.000 claims description 6
- 230000005693 optoelectronics Effects 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 3
- 230000002596 correlated effect Effects 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 description 6
- 238000012546 transfer Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 238000003672 processing method Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000009977 dual effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000004861 thermometry Methods 0.000 description 2
- 229910000530 Gallium indium arsenide Inorganic materials 0.000 description 1
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004438 eyesight Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004297 night vision Effects 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 238000005057 refrigeration Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Radiation Pyrometers (AREA)
- Studio Devices (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
Abstract
The invention discloses a four-waveband co-optical axis photoelectric imaging platform and an image fusion processing method thereof, belonging to the technical field of photoelectric detection and image processing. The invention provides a four-band optical axis-sharing photoelectric imaging platform and a multiband image fusion processing method. In addition, the invention also discloses a medium wave infrared and long wave infrared dual-waveband colorimetric temperature measurement method which can be applied in parallel with a multiband image fusion processing method and expands the application range of a four-waveband coaxial-axis photoelectric imaging platform.
Description
Technical Field
The invention relates to a four-waveband common-optical-axis photoelectric imaging platform and an image fusion processing method thereof, belonging to the technical field of photoelectric detection and image processing.
Background
With the development of the multiband imaging detector technology, the multiband fusion imaging technology shows wide application prospects. Visible light + near infrared, short wave infrared, medium wave infrared and long wave infrared are the most commonly used photoelectric imaging bands at present. The visible light and near infrared wave band has scene detail texture information consistent with the vision habit of human eyes; the short wave infrared mainly belongs to reflected information imaging under the condition of normal temperature, the transmission characteristic of the short wave infrared is superior to that of visible light, and the information difference of a target scene is obviously different from the visible light wave band; the medium wave infrared and the long wave infrared mainly reflect the heat radiation information of a target scene, have obvious difference with the reflection characteristic, have respective characteristics, and are commonly used for temperature measurement, armor target identification and the like in the past. Therefore, the four bands each contain different aspects of the characteristics of the target scene, i.e. the spectral information of the target scene is increased, and the effective utilization or fusion of the information becomes an effective means for improving the detection capability of the photoelectric imaging system.
At present, on one hand, the performance of a visible light and near InfraRed imaging detector based on a silicon-based CCD/CMOS is continuously improved, on the other hand, the technology of an InfraRed Focal Plane Array (IRFPA) of short wave, medium wave and long wave is also rapidly developed, and the technology of a bicolor or even multicolor InfraRed Focal Plane detector is practical, namely the technology of a multiband imaging sensor is natural or gradually matured. However, the existing research on the multiband image processing method is insufficient, although the research on the dual-waveband image fusion method has made some progress, except some simple weighted superposition algorithms, the natural color fusion algorithm based on color transfer has small calculated amount, is convenient for real-time processing, and is applied to visible light and infrared color night vision equipment; in recent years, methods of deep learning also show a growing trend in image fusion. Due to the fact that the multiband imaging detector is complex in system and high in price on the whole, and needs to be matched with links such as multichannel registration and image preprocessing, effective development of multiband image processing algorithms is limited, namely research and development of the processing algorithms are also limited by a multiband photoelectric imaging platform.
Disclosure of Invention
The invention aims to provide a four-band co-optical-axis photoelectric imaging platform and a multiband image fusion processing method. In addition, the invention also discloses a medium wave infrared and long wave infrared dual-waveband colorimetric temperature measurement method which can be applied in parallel with a multiband image fusion processing method and expands the application range of a four-waveband coaxial-axis photoelectric imaging platform.
The four wave bands refer to visible light + near infrared, short wave infrared, medium wave infrared and long wave infrared. The three bands refer to any three bands of the four bands. The dual band means any two bands of the four bands.
The purpose of the invention is realized by the following technical scheme.
The invention discloses a four-waveband coaxial photoelectric imaging platform, which comprises a multiband window, a four-waveband coaxial optical system, a visible light and near infrared imaging component, a short wave infrared imaging component, a medium wave infrared imaging component, a long wave infrared imaging component and a digital video processing board.
The four-waveband common-optical-axis optical system comprises a first beam splitter, a second beam splitter and a third beam splitter.
The digital video processing board is used for carrying out corresponding image processing according to an image processing algorithm.
The imaging assembly includes an objective lens and a cartridge.
The included angle between the first beam splitter and the cross section of the multiband window is 45 degrees, the included angle between the second beam splitter and the first beam splitter is 90 degrees, the included angle between the third beam splitter and the first beam splitter is 0 degree, and then the common optical axis of the four-waveband common-optical-axis optical system is realized. In order to avoid field loss and ensure no overlapping between light beams passing through the beam splitters, the lower endpoint of the second beam splitter needs to be above the upper endpoint of the first beam splitter, and the right endpoint of the third beam splitter needs to be left of the left endpoint of the first beam splitter.
The invention discloses a four-waveband coaxial photoelectric imaging platform, which realizes optical registration of four-waveband images by adjusting a four-waveband coaxial optical system, a visible light and near infrared imaging component, a short wave infrared imaging component, a medium wave infrared imaging component and a long wave infrared imaging component.
The optical registration adjusting method is realized by the following steps.
A first registration step: and (4) coarse adjustment. And the visible light and near infrared imaging assembly, the short wave infrared imaging assembly, the medium wave infrared imaging assembly, the long wave infrared imaging assembly, the first beam splitter, the second beam splitter and the third beam splitter are approximately adjusted to specified positions.
A second registration step: and finely adjusting the long-wave infrared imaging component to ensure that the optical axis of the long-wave infrared imaging component is vertical to the multiband window and then fixed. And (4) taking the long-wave infrared imaging assembly as a reference, after the registration step two is completed, adjusting other optical devices through the registration steps three, four, five and six, and completing registration work of other optical devices.
A third registration step: observing the gray level superposition image of the short wave infrared image and the long wave infrared image acquired by the short wave infrared imaging component by taking the long wave infrared image acquired by the long wave infrared imaging component as a reference, and adjusting the z-axis rotation angle and the pitching rotation angle of the short wave infrared imaging component and the first beam splitter
And registering the short-wave infrared image and the long-wave infrared image, namely determining the positions of the short-wave infrared imaging assembly and the first beam splitter.
The gray level superposition image of the short wave infrared image and the long wave infrared image is 1/2 of the sum of the gray levels of the corresponding pixels of the short wave infrared image and the long wave infrared image.
A fourth registration step: observing the gray level superposition image of the medium wave infrared image and the long wave infrared image acquired by the medium wave infrared imaging component by taking the long wave infrared image acquired by the long wave infrared imaging component as a reference, and adjusting the z-axis rotation angle and the pitching rotation angle of the medium wave infrared imaging component and the second beam splitter
Registering medium wave infrared imagesThe image and the long-wave infrared image determine the positions of the medium-wave infrared imaging component and the second beam splitter.
The gray level superposition image of the medium wave infrared image and the long wave infrared image is 1/2 of the sum of the gray levels of the corresponding pixels of the medium wave infrared image and the long wave infrared image.
A fifth registration step: observing the gray level superposition image of the visible light, the near infrared image and the long wave infrared image acquired by the visible light and near infrared imaging component by taking the long wave infrared image acquired by the long wave infrared imaging component as a reference, and adjusting the z-axis rotation angle and the pitching rotation angle of the visible light, the near infrared imaging component and the third beam splitter
And (3) aligning the visible light + near infrared image and the long-wave infrared image, namely determining the positions of the visible light + near infrared imaging component and the third beam splitter.
The gray level superimposed image of the visible light + near infrared image and the long wave infrared image is 1/2 of the sum of the gray levels of the corresponding pixels of the visible light + near infrared image and the long wave infrared image.
A registration step six: and observing the visible light and near infrared image acquired by the visible light and near infrared imaging assembly, the short wave infrared image acquired by the short wave infrared imaging assembly, the medium wave infrared image acquired by the medium wave infrared imaging assembly and the gray level superposition image of the long wave infrared image acquired by the long wave infrared imaging assembly, and determining the complete registration of the images of the four wave bands.
The gray level superposition image of the visible light + near infrared image, the short wave infrared image, the medium wave infrared image and the long wave infrared image is 1/4 of the sum of the gray levels of the corresponding pixels of the visible light + near infrared image, the short wave infrared image, the medium wave infrared image and the long wave infrared image.
Incident radiation of a scene enters from a multi-band window, after passing through a first beam splitter, radiation of medium-wave infrared bands and long-wave infrared bands enters a second beam splitter through transmission, and radiation of visible light, near infrared bands and short-wave infrared bands enters a third beam splitter through reflection. The medium wave infrared band radiation enters the medium wave infrared component for focusing and imaging after being reflected by the second beam splitter, the long wave infrared band radiation enters the long wave infrared component for focusing and imaging after being transmitted by the second beam splitter, the visible light and near infrared band radiation enters the visible light and near infrared component for focusing and imaging after being reflected by the third beam splitter, and the short wave infrared band radiation enters the short wave infrared component for focusing and imaging after being transmitted by the third beam splitter. Four-band image information in a target scene is simultaneously acquired through a four-band common-optical-axis photoelectric imaging platform. And inputting the four-waveband image information into the digital video processing board according to application requirements. And the digital video processing board performs corresponding image processing according to the selected image processing algorithm. Namely, the four-band co-optical-axis photoelectric imaging platform can complete multiband image processing of four-band color fusion, three-band color fusion and two-band color fusion by combining the four-band image fusion processing method, fully utilizes complementation and richness of multiband information, and improves target detection and identification efficiency.
The invention also discloses a multiband image fusion processing method, which comprises the following steps:
image fusion step one: and simultaneously acquiring four-waveband image information in a target scene.
And a second image fusion step: and preprocessing the four-waveband image information obtained in the image fusion step I to obtain preprocessed four-waveband image information. The pretreatment comprises the following steps: blind pixel correction is carried out on the short wave infrared image, and non-uniformity correction is respectively carried out on the medium wave infrared image and the long wave infrared image; and respectively enhancing the short wave infrared image, the medium wave infrared image and the long wave infrared image.
And step three, image fusion: performing linear combination of the preprocessed four-waveband image information obtained in the image fusion step two in a YUV space in a formula (1) to obtain an initial color image of a corresponding waveband and corresponding parameters, wherein the corresponding parameters comprise a brightness channel Y of the fused imageiBlue color difference component UiAnd red color difference component Vi。
In the formula, Vis, SWIR, MWIR and LWIR respectively represent visible light + near infrared, short wave infrared, medium wave infrared and long wave infrared images; k is a radical of1,k2,k3,k4,m1,m2,m3,m4,m5,m6,m7And m8Is a mean positive rational number, is an empirical value, and k1+k2>k3+k4The wave bands participating in fusion can be controlled by adjusting the values of the 12 parameters, multiband image processing of four-wave band image fusion, three-wave band image fusion and two-wave band image fusion is correspondingly realized, and the complementation and the enrichment of multiband information are fully utilized. U shapeiAnd ViCorresponding to blue and red difference components respectively, the visible light + near infrared Vis and the short wave infrared image can be reflected in a blue difference channel, and the medium wave infrared image and the long wave infrared image are reflected in a red difference channel respectively, so that a color image which is more in line with the visual characteristics of human eyes is obtained; y isiThe luminance channel is a brightness channel of the fused image, namely a gray level fusion result of the multiband image.
And a fourth step of image fusion: according to the initial color image and the corresponding parameters of the corresponding wave band obtained in the image fusion step three, substituting the corresponding parameters into formula (2), namely, transmitting the color of the reference image to the initial colorized image and the corresponding parameters, wherein the corresponding parameters comprise Yi、Ui、Vi。
Wherein, Yo,Uo,VoRespectively obtaining YUV channels of the finally obtained color fusion image; sigmaT,Y,σT,U,σT,VAnd σi,Y,σi,U,σi,VRespectively is the standard deviation of each channel of the color reference image and the initial color image YUV; mu.sT,Y,μT,U,μT,VAnd mui,Y,μi,U,μi,VThe average values of the color reference image and the initial color image YUV channels are respectively.
And a fifth step of image fusion: according to application requirements, the wave bands participating in fusion are controlled by adjusting the values of the 12 parameters in the step three, multiband image processing corresponding to four-wave band color fusion, three-wave band color fusion and two-wave band color fusion is realized according to the image fusion steps three and four, complementation and richness of multiband information are fully utilized, and target detection and identification efficiency is improved.
The four wave bands refer to visible light + near infrared, short wave infrared, medium wave infrared and long wave infrared. The three bands refer to any three bands of the four bands. The two bands refer to any two bands of the four bands.
Preferably, the multiband image fusion processing method is applied to the four-waveband coaxial-axis photoelectric imaging platform, multiband image processing corresponding to four-waveband color fusion, three-waveband color fusion and two-waveband color fusion is realized, complementation and richness of multiband information are fully utilized, and target detection and identification efficiency is improved.
Preferably, the four-waveband coaxial-axis photoelectric imaging platform can be applied to a multiband image fusion processing method, and can also be applied to typical multiband image applications such as medium-wave infrared and long-wave infrared dual-waveband colorimetric temperature measurement and heterogeneous image registration by selecting a medium-wave infrared and long-wave infrared dual-waveband colorimetric temperature measurement method and a heterogeneous image registration algorithm.
The invention also discloses a medium wave infrared and long wave infrared dual-waveband colorimetric temperature measurement method which can be applied in parallel with a multiband image fusion processing method and expand the application range of a four-waveband common-optical-axis photoelectric imaging platform.
The medium-wave infrared and long-wave infrared dual-waveband colorimetric temperature measurement method comprises the following steps:
temperature measurement step one: and simultaneously acquiring medium wave infrared and long wave infrared image information in a target scene.
According to the response principle of the detector, the temperature is measured in two temperature measuring wave bands lambdamin,λmax]The signal level output by the detector is as shown in equation (3).
Wherein R isV(lambda) is the spectral responsivity of the temperature measurement band detector; a is the area of a detector unit; epsilon (lambda) object spectral emissivity; d is the clear aperture of the optical system; f' is the focal length of the optical system; tau isa(λ) is the atmospheric spectral transmittance; tau is0(λ) is optical system transmittance; meb(λ, T) is Planck's law; u (t) is the signal level output by the detector.
A second temperature measurement step: and preprocessing the medium wave infrared image information and the long wave infrared image information acquired in the temperature measurement step I to obtain preprocessed double-waveband image information. The pretreatment comprises the following steps: carrying out non-uniformity correction on the medium wave infrared image and the long wave infrared image respectively; and respectively enhancing the medium wave infrared image and the long wave infrared image.
Temperature measurement step three: because the gray value of a single pixel of the detector is positively correlated with the output signal level U (T) of the detector, the ratio of the signal levels of the pixels corresponding to the two detectors is equal to the ratio of the gray values of the pixels corresponding to the two detectors.
The temperature of the measured object is determined by using the ratio of the signal levels of the corresponding pixels of the medium wave infrared and the long wave infrared obtained by the two detectors, the expression of the formula (3) of the medium wave infrared and the long wave infrared is compared, the result of the ratio is shown as the formula (4),
wherein, U1(T) is the signal level output by the medium wave infrared detector; u shape2(T) is the signal level output by the long-wave infrared detector; q (T) is the ratio of the signal level output by the medium wave infrared detector to the signal level output by the long wave infrared detector.
The double-waveband colorimetric temperature measurement of medium and long wave infrared can greatly reduce the spectral emissivity epsilon (lambda) of a temperature measurement target and the spectral transmittance tau in the transmission processa(lambda) and tau0(lambda) influence on temperature measurement accuracy, approximate determination of two different wave bandsThe ratio of the signals is a temperature dependent function.
Temperature measurement step four: and (3) fitting a curve of gray value ratio Q (T) of each corresponding pixel point of the medium-wavelength infrared image and the long-wavelength infrared image along with the change of the temperature T and a corresponding polynomial fitting expression through black body calibration.
A fifth temperature measuring step: and (4) substituting the ratio Q (T) of the signal level output by the medium wave infrared detector and the signal level output by the long wave infrared detector obtained in the temperature measurement step (III) into a polynomial fitting expression in the temperature measurement step (IV) to obtain corresponding temperature T, namely obtaining temperature images with the same resolution as the medium wave infrared image and the long wave infrared image.
Has the advantages that:
1. the invention discloses a four-waveband coaxial-axis photoelectric imaging platform, wherein an included angle between a first beam splitter and a multiband window cross section is 45 degrees, an included angle between a second beam splitter and the first beam splitter is 90 degrees, and an included angle between a third beam splitter and the first beam splitter is 0 degree, so that coaxial axis of a four-waveband coaxial-axis optical system is realized. The scenes in the field of view can be accurately registered, and no parallax exists for the scenes with different object distances after the registration is finished. The coaxial beam splitting optical system is very fit with a multi-channel imaging system requiring strict registration. Meanwhile, the optical splitting system with the common optical axis can complete registration in an optical and mechanical mode without electronic registration, so that the field of view and the image resolution of each channel are not lost.
2. The multiband image fusion processing method disclosed by the invention is based on a formula for carrying out linear combination on four-waveband image information in a YUV space and a color transfer formula, can control the waveband participating in fusion by adjusting the parameter value in the formula of the linear combination according to application requirements, realizes multiband image processing corresponding to four-waveband color fusion, three-waveband color fusion and two-waveband color fusion, fully utilizes the complementation and richness of multiband information, and improves the target detection and identification efficiency. The YUV space is used for color transfer, a large amount of logarithm and exponential operation is reduced, the color space is most beneficial to hardware real-time video processing, and the color space has high reliability and robustness on the premise of ensuring the algorithm processing speed.
3. The invention discloses a medium wave infrared and long wave infrared dual-band colorimetric temperature measurement method, which uses the signal ratio of a conventional medium wave infrared detector and a conventional long wave infrared detector as input, and calculates the temperature of a scene according to a polynomial fitting expression of the temperature calibrated by a black body along with the signal ratio of the medium wave infrared detector and the long wave infrared detector. The uncooled medium-wave and long-wave infrared focal plane detectors are adopted, the operation is stable, the cost is low, the medium-wave and long-wave infrared information and the temperature information in a scene can be obtained at the same time, and a special temperature measuring detector is not needed. The dual-waveband colorimetric temperature measurement can effectively reduce temperature measurement errors caused by different object emissivity and attenuation in the infrared radiation propagation process. The dual-band colorimetric temperature measurement can be used for measuring the temperature of a scene in a large range and a long distance, and is convenient for conveniently discriminating environment targets with different temperatures in an image.
4. The four-waveband co-optical-axis photoelectric imaging platform and the image processing method thereof can realize processing links such as multiband image registration, multiband image preprocessing, multiband image fusion, dual-waveband colorimetric temperature measurement and the like, can simultaneously acquire four-waveband image information of a target scene, process the image information of four wavebands, and provide an experimental platform for subsequent multiband imaging processing algorithm research.
Drawings
FIG. 1 is a schematic diagram of a four-band common-optical-axis photoelectric imaging platform according to the present invention. Wherein: 1 — a multiband window; 2-four-band optical axis-sharing optical system; 3-visible light + near infrared imaging component; 4-short wave infrared imaging component; 5-a medium wave infrared imaging component; 6-a long-wave infrared imaging component; 7-digital video processing board.
FIG. 2 is a structural diagram of a four-band co-optical axis optoelectronic imaging platform SOLIDWORKS according to the present invention.
FIG. 3 is a schematic view of a micro-displacement adjusting bracket of the beam splitter according to the present invention. Wherein: 8-angle of rotation of pitch
Finely adjusting a knob; 9-angle of rotation of pitch
Locking screws; 10-z axis rotation angle rotation fine adjustment knob; the locking knob is rotated by 11-z axis rotation angle.
FIG. 4 is a flow chart of the four-band natural color fusion image algorithm of the present invention.
FIG. 5 is a four-band image and a natural color fusion image thereof according to the present invention, wherein: (a) the image is a visible light and near infrared image, (b) is a short wave infrared image, (c) is a medium wave infrared image, (d) is a long wave infrared image, (e) is a color fusion image, and (f) is a gray fusion image.
FIG. 6 is a flow chart of the two-band colorimetric temperature measurement method of the present invention.
FIG. 7 is a two-band colorimetric thermometry image of the present invention, wherein: (a) the image is a medium wave infrared image, (b) is a long wave infrared image, and (c) is a temperature image.
Detailed description of the invention
For a better understanding of the objects and advantages of the present invention, reference should be made to the following detailed description taken in conjunction with the accompanying drawings and examples.
Example 1:
as shown in fig. 1 and fig. 2, the four-band co-optical-axis photoelectric imaging platform disclosed in this embodiment includes a multiband window 1, a four-band co-optical-axis optical system 2, a visible light + near-infrared imaging component 3, a short-wave infrared imaging component 4, a medium-wave infrared imaging component 5, a long-wave infrared imaging component 6, and a digital video processing board 7.
The four-band optical axis system 2 includes a first beam splitter 2.1, a second beam splitter 2.2, and a third beam splitter 2.3.
The digital video processing board 7 is used for carrying out corresponding image processing according to an image processing algorithm.
The imaging assembly includes an objective lens and a cartridge.
The included angle between the first beam splitter 2.1 and the cross section of the multiband window 1 is 45 degrees, the included angle between the second beam splitter 2.2 and the first beam splitter 2.1 is 90 degrees, the included angle between the third beam splitter 2.3 and the first beam splitter 2.1 is 0 degree, and further the common optical axis of the four-band common optical axis optical system 2 is realized. In order to avoid loss of field of view, there cannot be overlap between the beams passing through the beam splitters, the lower endpoint of the second beam splitter 2.2 needs to be above the upper endpoint of the first beam splitter 2.1, and the right endpoint of the third beam splitter 2.3 needs to be to the left of the left endpoint of the first beam splitter 2.1.
As shown in fig. 3, the four-band co-optical-axis photoelectric imaging platform disclosed in this embodiment realizes optical registration of four-band images by adjusting the four-band co-optical-axis optical system 2, the visible light + near-infrared imaging component 3, the short-wave infrared imaging component 4, the medium-wave infrared imaging component 5, and the long-wave infrared imaging component 6.
The optical registration adjusting method is realized by the following steps.
A first registration step: and (4) coarse adjustment. The visible light + near infrared imaging component 3, the short wave infrared imaging component 4, the medium wave infrared imaging component 5, the long wave infrared imaging component 6 and the first beam splitter 2.1, the second beam splitter 2.2 and the third beam splitter 2.3 are adjusted to approximately the specified positions.
A second registration step: and finely adjusting the long-wave infrared imaging component 6 to ensure that the optical axis of the long-wave infrared imaging component 6 is vertical to the multiband window 1 and then fixed. And (3) taking the long-wave infrared imaging component 6 as a reference, after the registration step two is completed, adjusting other optical devices through the steps three, four, five and six, and completing registration work of other optical devices.
A third registration step: observing the gray level superposition image of the short wave infrared image and the long wave infrared image acquired by the short wave infrared imaging component 4 by taking the long wave infrared image acquired by the long wave infrared imaging component 6 as a reference, and adjusting the z-axis rotation angle and the pitching rotation angle of the short wave infrared imaging component 4 and the first beam splitter 2.1
And registering the short-wave infrared image and the long-wave infrared image, namely determining the positions of the short-wave infrared imaging component 4 and the first beam splitter 2.1.
The gray level superposition image of the short wave infrared image and the long wave infrared image is 1/2 of the sum of the gray levels of the corresponding pixels of the short wave infrared image and the long wave infrared image.
Z-axis and pitch angles of rotation of the first beam splitter 2.1
The structure is shown in fig. 3. The adjustable structure can be locked, and meanwhile, the registration precision and the mechanical structure strength are guaranteed.
A fourth registration step: observing the gray level superposition image of the medium wave infrared image and the long wave infrared image acquired by the medium wave infrared imaging component 5 by taking the long wave infrared image acquired by the long wave infrared imaging component 6 as a reference, and adjusting the z-axis rotation angle and the pitching rotation angle of the medium wave infrared imaging component 5 and the second beam splitter 2.2
And registering the medium wave infrared image and the long wave infrared image, namely determining the positions of the medium wave infrared imaging component 5 and the second beam splitter 2.2.
The gray level superposition image of the medium wave infrared image and the long wave infrared image is 1/2 of the sum of the gray levels of the corresponding pixels of the medium wave infrared image and the long wave infrared image.
Z-axis and pitch angles of rotation of the second beam splitter 2.2
The structure is shown in fig. 3. The adjustable structure can be locked, and meanwhile, the registration precision and the mechanical structure strength are guaranteed.
A fifth registration step: observing the gray level superposition image of the visible light, the near infrared image and the long wave infrared image acquired by the visible light and near infrared imaging component 3 by taking the long wave infrared image acquired by the long wave infrared imaging component 6 as a reference, and adjusting the z-axis rotation angle and the pitching rotation angle of the visible light, the near infrared imaging component 3 and the third beam splitter 2.3
And (3) aligning the visible light + near infrared image and the long wave infrared image, namely determining the positions of the visible light + near infrared imaging component 3 and the third beam splitter 2.3.
The gray level superimposed image of the visible light + near infrared image and the long wave infrared image is 1/2 of the sum of the gray levels of the corresponding pixels of the visible light + near infrared image and the long wave infrared image.
Z-axis and pitch angles of rotation of the third beam splitter 2.3
The structure is shown in fig. 3. The adjustable structure can be locked, and meanwhile, the registration precision and the mechanical structure strength are guaranteed.
A registration step six: and observing the visible light and near infrared image acquired by the visible light and near infrared imaging component 3, the short wave infrared image acquired by the short wave infrared imaging component 4, the medium wave infrared image acquired by the medium wave infrared imaging component 5 and the gray level superposition image of the long wave infrared image acquired by the long wave infrared imaging component 6, and determining the complete registration of the images of the four wave bands.
The gray level superposition image of the visible light + near infrared image, the short wave infrared image, the medium wave infrared image and the long wave infrared image is 1/4 of the sum of the gray levels of the corresponding pixels of the visible light + near infrared image, the short wave infrared image, the medium wave infrared image and the long wave infrared image.
The scenes in the field of view of the four-band imaging assembly can be accurately registered, and no parallax exists for the scenes with different object distances after the registration is completed. The coaxial beam splitting optical system is very fit with a multi-channel imaging system requiring strict registration. Meanwhile, the optical splitting system with the common optical axis can complete registration in an optical and mechanical mode without electronic registration, so that the field of view and the image resolution of each channel are not lost.
Incident radiation of a scene enters from the multi-band window 1, after passing through the first beam splitter 2.1, radiation of medium-wave infrared and long-wave infrared bands enters the second beam splitter 2.2 through transmission, and radiation of visible light, near infrared and short-wave infrared bands enters the third beam splitter 2.3 through reflection. The medium wave infrared band radiation enters the medium wave infrared component for focusing and imaging after being reflected by the second beam splitter 2.2, the long wave infrared band radiation enters the long wave infrared component for focusing and imaging after being transmitted by the second beam splitter 2.2, the visible light and the near infrared band radiation enter the visible light and the near infrared component for focusing and imaging after being reflected by the third beam splitter 2.3, and the short wave infrared band radiation enters the short wave infrared component for focusing and imaging after being transmitted by the third beam splitter 2.3. Four-band image information in a target scene is simultaneously acquired through a four-band common-optical-axis photoelectric imaging platform. The four-band image information is input to the digital video processing board 7 according to application requirements. And the digital video processing board 7 performs corresponding image processing according to the selected image processing algorithm. Namely, the four-band co-optical-axis photoelectric imaging platform can complete multiband image processing of four-band color fusion, three-band color fusion and two-band color fusion by combining the four-band image fusion processing method, fully utilizes complementation and richness of multiband information, and improves target detection and identification efficiency.
The parameters of the medium-wave infrared and long-wave infrared objective lens are that the focal length is 40mm, and F is 1.0; the medium wave infrared and long wave infrared detector component is a non-refrigeration focal plane detector component LA6110 of smoke platform AiRui company, the number of pixels is 640 multiplied by 512, the distance between the pixels is 17 mu m, NETD is less than or equal to 60mK, the frame frequency is 50Hz, and the output video is CameraLink digital video; the response waveband of the medium-wave infrared detector is 3-14 mu m, and the response waveband of the long-wave infrared detector is 8-14 mu m. The parameters of the visible light and near infrared objective lens are that the focal length is 12.5-75 mm, and the minimum F is 1.2; the visible light detector component is a low-illumination CMOS (complementary Metal oxide semiconductor) movement P2101 of Kunshan Shaoxing-Co Microprocessor, the target surface size is 1 inch, the pixel number is 1280 multiplied by 1024, the pixel size is 9.7 mu m multiplied by 9.7 mu m, the frame rate is 50Hz, the output video is CameraLink digital video, and when a lens of F1.4 is used, the output video can be 1 multiplied by 10-3And (4) clearly imaging under lx low-light conditions. The short-wave infrared objective lens has the parameters that the focal length is 12.5-75 mm, and the minimum F is 1.2; the short wave infrared detector component is an InGaAs uncooled focal plane detector component GH-SWCL-15 of Shanxi national Whiteon photoelectric technology Limited, the pixel number is 640 multiplied by 512, the pixel pitch is 15 mu m, the frame frequency is 100Hz, and the output video is CameraLink digital video.
The digital video image processing board adopts a high-speed digital signal processing board taking an FPGA (model Kintex-7) as a core, and is provided with 4 lines of Camera Link digital video input and 2 lines of Camera Link digital video output. The image data which can be selectively output comprises a visible light + near infrared image sequence, a short wave infrared image sequence, a medium wave infrared image sequence, a long wave infrared image sequence, a visible light + near infrared and long wave infrared color fusion image sequence, a visible light + near infrared and long wave infrared gray scale fusion image sequence, a four wave band image color fusion image sequence, an arbitrary three wave band image color fusion image sequence in four wave bands, an arbitrary two wave band image color fusion image sequence in four wave bands and a medium wave infrared and long wave infrared dual wave band infrared temperature measurement image sequence.
The four-waveband co-optical-axis photoelectric imaging platform and the image processing method thereof disclosed by the embodiment can realize processing links such as multiband image registration, multiband image preprocessing, multiband image fusion and dual-waveband colorimetric temperature measurement, can simultaneously acquire four-waveband image information of a target scene, processes the image information of four wavebands, and provides an experimental platform for subsequent multiband imaging processing algorithm research.
The four-waveband coaxial-axis photoelectric imaging platform can also select a medium-wave infrared and long-wave infrared dual-waveband colorimetric temperature measurement algorithm and a heterogeneous image registration algorithm to carry out typical application of multiband images such as medium-wave infrared and long-wave infrared dual-waveband colorimetric temperature measurement and heterogeneous image registration.
The embodiment also discloses a multiband image fusion processing method, a flow chart of which is shown in fig. 4, and the method comprises the following steps:
image fusion step one: and simultaneously acquiring four-waveband image information in a target scene.
And a second image fusion step: and preprocessing the four-waveband image information obtained in the image fusion step I to obtain preprocessed four-waveband image information. The pretreatment comprises the following steps: blind pixel correction is carried out on the short wave infrared image, and non-uniformity correction is respectively carried out on the medium wave infrared image and the long wave infrared image; and respectively enhancing the short wave infrared image, the medium wave infrared image and the long wave infrared image. The preprocessed images are shown in fig. 5(a), 5(b), 5(c), and 5 (d).
And step three, image fusion: performing linear combination of the preprocessed four-waveband image information obtained in the image fusion step two in a YUV space by a formula (5) to obtain an initial color image of a corresponding waveband and corresponding parameters, wherein the corresponding parameters comprise a brightness channel Y of the fused imageiBlue color difference component UiAnd red color difference component Vi。
In the formula, Vis, SWIR, MWIR and LWIR respectively represent visible light + near infrared, short wave infrared, medium wave infrared and long wave infrared images; k is a radical of1,k2,k3,k4,m1,m2,m3,m4,m5,m6,m7And m8Is a mean positive rational number, is an empirical value, and k1+k2>k3+k4The wave bands participating in fusion can be controlled by adjusting the values of the 12 parameters, multiband image processing of four-wave band image fusion, three-wave band image fusion and two-wave band image fusion is correspondingly realized, and the complementation and the enrichment of multiband information are fully utilized. U shapeiAnd ViCorresponding to blue and red difference components respectively, the visible light + near infrared Vis and the short wave infrared image can be reflected in a blue difference channel, and the medium wave infrared image and the long wave infrared image are reflected in a red difference channel respectively, so that a color image which is more in line with the visual characteristics of human eyes is obtained; y isiThe luminance channel is a brightness channel of the fused image, namely a gray level fusion result of the multiband image.
And a fourth step of image fusion: according to the initial color image and the corresponding parameters of the corresponding wave band obtained in the image fusion step three, the corresponding parameters are substituted into formula (6), namely the color of the reference image is transmitted to the initial colorized image and the corresponding parameters, and the corresponding parameters comprise Yi、Ui、Vi。
Wherein, Yo,Uo,VoRespectively obtaining YUV channels of the finally obtained color fusion image; sigmaT,Y,σT,U,σT,VAnd σi,Y,σi,U,σi,VRespectively is the standard deviation of each channel of the color reference image and the initial color image YUV; mu.sT,Y,μT,U,μT,VAnd mui,Y,μi,U,μi,VThe average values of the color reference image and the initial color image YUV channels are respectively.
And a fifth step of image fusion: according to application requirements, the wave bands participating in fusion are controlled by adjusting the values of the 12 parameters in the step three, multiband image processing corresponding to four-wave band color fusion, three-wave band color fusion and two-wave band color fusion is realized according to the image fusion steps three and four, complementation and richness of multiband information are fully utilized, and target detection and identification efficiency is improved.
The four wave bands refer to visible light + near infrared, short wave infrared, medium wave infrared and long wave infrared. The three bands refer to any three bands of the four bands. The two bands refer to any two bands of the four bands.
The multiband image fusion processing method is applied to the four-waveband coaxial-axis photoelectric imaging platform, multiband image processing corresponding to four-waveband color fusion, three-waveband color fusion and two-waveband color fusion is achieved, complementation and richness of multiband information are fully utilized, and target detection and identification efficiency is improved.
The color fusion result of the four-band image is shown in fig. 5(e), and the grayscale fusion result of the four-band image is shown in fig. 5 (f).
The multiband image fusion processing method disclosed by the embodiment is based on a formula for performing linear combination on four-waveband image information in a YUV space and a color transfer formula, can control the waveband participating in fusion by adjusting the parameter value in the formula of the linear combination according to application requirements, realizes multiband image processing corresponding to four-waveband color fusion, three-waveband color fusion and two-waveband color fusion, fully utilizes complementation and richness of multiband information, and improves target detection and identification efficiency. The YUV space is used for color transfer, a large amount of logarithm and exponential operation is reduced, the color space is most beneficial to hardware real-time video processing, and the color space has high reliability and robustness on the premise of ensuring the algorithm processing speed.
The embodiment also discloses a medium-wave infrared and long-wave infrared dual-band colorimetric temperature measurement algorithm which can be applied in parallel with a multi-band image fusion processing method to expand the application range of the four-band common-optical-axis photoelectric imaging platform.
The flow chart of the medium-wave infrared and long-wave infrared dual-waveband colorimetric temperature measurement method is shown in figure 6, and the method comprises the following steps:
temperature measurement step one: and simultaneously acquiring medium wave infrared and long wave infrared image information in a target scene.
According to the response principle of the detector, the temperature is measured in two temperature measuring wave bands lambdamin,λmax]The signal level output by the detector is as shown in equation (7).
Wherein R isV(lambda) is the spectral responsivity of the temperature measurement band detector; a is the area of a detector unit; epsilon (lambda) object spectral emissivity; d is the clear aperture of the optical system; f' is the focal length of the optical system; tau isa(λ) is the atmospheric spectral transmittance; tau is0(λ) is optical system transmittance; meb(λ, T) is Planck's law; u (t) is the signal level output by the detector.
A second temperature measurement step: and preprocessing the medium wave infrared image information and the long wave infrared image information acquired in the temperature measurement step I to obtain preprocessed double-waveband image information. The pretreatment comprises the following steps: carrying out non-uniformity correction on the medium wave infrared image and the long wave infrared image respectively; and respectively enhancing the medium wave infrared image and the long wave infrared image. The preprocessed images are shown in fig. 7(a) and 7 (b).
Temperature measurement step three: because the gray value of a single pixel of the detector is positively correlated with the output signal level U (T) of the detector, the ratio of the signal levels of the pixels corresponding to the two detectors is equal to the ratio of the gray values of the pixels corresponding to the two detectors.
The temperature of the measured object is determined by using the ratio of the signal levels of the corresponding pixels of the medium wave infrared and the long wave infrared obtained by the two detectors, the expression of the formula (7) of the medium wave infrared and the long wave infrared is compared, the result of the ratio is shown as the formula (8),
wherein, U1(T) is the signal level output by the medium wave infrared detector; u shape2(T) is the signal level output by the long-wave infrared detector; q (T) is the ratio of the signal level output by the medium wave infrared detector to the signal level output by the long wave infrared detector.
The double-waveband colorimetric temperature measurement of medium and long wave infrared can greatly reduce the spectral emissivity epsilon (lambda) of a temperature measurement target and the spectral transmittance tau in the transmission processa(lambda) and tau0The effect of (λ) on the accuracy of thermometry is approximated by the ratio of two different band signals being a temperature dependent function.
Temperature measurement step four: and (3) fitting a curve of gray value ratio Q (T) of each corresponding pixel point of the medium-wavelength infrared image and the long-wavelength infrared image along with the change of the temperature T and a corresponding polynomial fitting expression through black body calibration.
A fifth temperature measuring step: and (4) substituting the ratio Q (T) of the signal level output by the medium wave infrared detector and the signal level output by the long wave infrared detector obtained in the temperature measurement step (III) into a polynomial fitting expression in the temperature measurement step (IV) to obtain corresponding temperature T, namely obtaining temperature images with the same resolution as the medium wave infrared image and the long wave infrared image.
The temperature image obtained by the two-band colorimetric temperature measurement is shown in fig. 7 (c).
In the method for colorimetric temperature measurement of a medium-wave infrared and long-wave infrared dual-band, a signal ratio of a conventional medium-wave infrared detector and a conventional long-wave infrared detector is used as an input, and the temperature of a scene is calculated according to a polynomial fitting expression of the temperature calibrated by a black body along with the signal ratio of the medium-wave infrared detector and the long-wave infrared detector. The uncooled medium-wave and long-wave infrared focal plane detectors are adopted, the operation is stable, the cost is low, the medium-wave and long-wave infrared information and the temperature information in a scene can be obtained at the same time, and a special temperature measuring detector is not needed. The dual-waveband colorimetric temperature measurement can effectively reduce temperature measurement errors caused by different object emissivity and attenuation in the infrared radiation propagation process. The dual-band colorimetric temperature measurement can be used for measuring the temperature of a scene in a large range and a long distance, and is convenient for conveniently discriminating environment targets with different temperatures in an image.
In summary, the above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (5)
1. Four wave band optical axis optoelectronic imaging platform altogether, its characterized in that: the device comprises a multiband window (1), a four-waveband coaxial optical system (2), a visible light and near infrared imaging component (3), a short wave infrared imaging component (4), a medium wave infrared imaging component (5), a long wave infrared imaging component (6) and a digital video processing board (7);
the four-band co-optical-axis optical system (2) comprises a first beam splitter (2.1), a second beam splitter (2.2) and a third beam splitter (2.3);
the digital video processing board (7) is used for carrying out corresponding image processing according to an image processing algorithm;
the imaging assembly comprises an objective lens and a movement;
the included angle between the first beam splitter and the cross section of the multiband window (1) is 45 degrees, the included angle between the second beam splitter (2.2) and the first beam splitter (2.1) is 90 degrees, the included angle between the third beam splitter (2.3) and the first beam splitter (2.1) is 0 degree, and further the common optical axis of the four-band common optical axis optical system (2) is realized; in order to avoid field loss and ensure no overlap between light beams passing through the beam splitter, the lower endpoint of the second beam splitter (2.2) needs to be above the upper endpoint of the first beam splitter (2.2), and the right endpoint of the third beam splitter (2.3) needs to be left of the left endpoint of the first beam splitter (2.2).
2. The four-band co-optical-axis optoelectronic imaging platform of claim 1, wherein: the optical registration of the four-waveband common-optical-axis optical system (2), the visible light and near infrared imaging component (3), the short-wave infrared imaging component (4), the medium-wave infrared imaging component (5) and the long-wave infrared imaging component (6) is realized by adjusting the four-waveband common-optical-axis optical system;
the optical registration adjusting method is realized by the following steps;
a first registration step: coarse adjustment; the method comprises the following steps of roughly adjusting a visible light + near infrared imaging assembly (3), a short wave infrared imaging assembly (4), a medium wave infrared imaging assembly (5), a long wave infrared imaging assembly (6), a first beam splitter (2.1), a second beam splitter (2.2) and a third beam splitter (2.3) to specified positions;
a second registration step: finely adjusting the long-wave infrared imaging component (6) to enable the optical axis of the long-wave infrared imaging component (6) to be vertical to the multiband window (1) and then fixing; taking the long-wave infrared imaging assembly (6) as a reference, after the registration step two is completed, adjusting other optical devices through the steps three, four, five and six to complete the registration work of other optical devices;
a third registration step: the long-wave infrared image acquired by the long-wave infrared imaging component (6) is used as a reference, the gray level superposition image of the short-wave infrared image and the long-wave infrared image acquired by the short-wave infrared imaging component (4) is observed, and the z-axis rotation angle and the pitching rotation angle of the short-wave infrared imaging component (4) and the first beam splitter (2.1) are adjusted
Registering the short-wave infrared image and the long-wave infrared image, namely determining the positions of the short-wave infrared imaging component (4) and the first beam splitter (2.1);
the gray level superposition image of the short wave infrared image and the long wave infrared image is 1/2 of the sum of the gray levels of the corresponding pixels of the short wave infrared image and the long wave infrared image;
a fourth registration step: observing the gray level superposition image of the medium wave infrared image and the long wave infrared image acquired by the medium wave infrared imaging component (5) by taking the long wave infrared image acquired by the long wave infrared imaging component (6) as a reference, and adjusting the z-axis rotation angle and the pitching rotation angle of the medium wave infrared imaging component (5) and the second beam splitter (2.2)
Registering the mid-wave infrared image with the long-wave infrared image, i.e. determining the mid-wave infrared imaging assembly (5) andthe position of the second beam splitter (2.2);
the gray level superposition image of the medium wave infrared image and the long wave infrared image is 1/2 of the sum of the gray levels of the corresponding pixels of the medium wave infrared image and the long wave infrared image;
a fifth registration step: observing the gray level superposition image of the visible light, the near infrared image and the long wave infrared image acquired by the visible light and near infrared imaging component (3) by taking the long wave infrared image acquired by the long wave infrared imaging component (6) as a reference, and adjusting the z-axis rotation angle and the pitching rotation angle of the visible light and near infrared imaging component (3) and the third beam splitter (2.3)
The positions of the visible light and near infrared imaging component (3) and the third beam splitter (2.3) can be determined by aligning the visible light and near infrared image with the long-wave infrared image;
the gray level superimposed image of the visible light + near infrared image and the long wave infrared image is 1/2 of the sum of the gray levels of the corresponding pixels of the visible light + near infrared image and the long wave infrared image;
a registration step six: observing visible light and near infrared images acquired by the visible light and near infrared imaging component (3), short wave infrared images acquired by the short wave infrared imaging component (4), medium wave infrared images acquired by the medium wave infrared imaging component (5) and gray level superposition images of long wave infrared images acquired by the long wave infrared imaging component (6), and determining the complete registration of the images of the four wave bands;
the gray level superposition image of the visible light + near infrared image, the short wave infrared image, the medium wave infrared image and the long wave infrared image is 1/4 of the sum of the gray levels of the corresponding pixels of the visible light + near infrared image, the short wave infrared image, the medium wave infrared image and the long wave infrared image;
incident radiation of a scene enters from a multi-band window (1), after passing through a first beam splitter (2.1), radiation of medium-wave infrared and long-wave infrared bands enters a second beam splitter (2.2) through transmission, and radiation of visible light, near infrared and short-wave infrared bands enters a third beam splitter (2.3) through reflection; the medium wave infrared band radiation enters a medium wave infrared component for focusing and imaging after being reflected by a second beam splitter (2.2), the long wave infrared band radiation enters a long wave infrared component for focusing and imaging after being transmitted by the second beam splitter (2.2), the visible light and near infrared band radiation enters a visible light and near infrared component for focusing and imaging after being reflected by a third beam splitter (2.3), and the short wave infrared band radiation enters a short wave infrared component for focusing and imaging after being transmitted by the third beam splitter (2.3); simultaneously acquiring four-band image information in a target scene through a four-band common-optical-axis photoelectric imaging platform; inputting the four-waveband image information into a digital video processing board (7) according to application requirements; the digital video processing board (7) performs corresponding image processing according to the selected image processing algorithm; namely, the four-band co-optical-axis photoelectric imaging platform can complete multiband image processing of four-band color fusion, three-band color fusion and two-band color fusion by combining the four-band image fusion processing method, fully utilizes complementation and richness of multiband information, and improves target detection and identification efficiency.
3. A multi-band image fusion processing method is characterized in that: comprises the following steps of (a) carrying out,
image fusion step one: simultaneously acquiring four-waveband image information in a target scene;
and a second image fusion step: preprocessing the four-waveband image information obtained in the image fusion step I to obtain preprocessed four-waveband image information; the pretreatment comprises the following steps: blind pixel correction is carried out on the short wave infrared image, and non-uniformity correction is respectively carried out on the medium wave infrared image and the long wave infrared image; respectively enhancing the short wave infrared image, the medium wave infrared image and the long wave infrared image;
and step three, image fusion: performing linear combination of the preprocessed four-waveband image information obtained in the image fusion step two in a YUV space in a formula (1) to obtain an initial color image of a corresponding waveband and corresponding parameters, wherein the corresponding parameters comprise a brightness channel Y of the fused imageiBlue color difference component UiAnd red color difference component Vi；
In the formula, Vis, SWIR, MWIR and LWIR respectively represent visible light + near infrared, short wave infrared, medium wave infrared and long wave infrared images; k is a radical of1,k2,k3,k4,m1,m2,m3,m4,m5,m6,m7And m8Is a mean positive rational number, is an empirical value, and k1+k2>k3+k4The wave bands participating in fusion can be controlled by adjusting the values of the 12 parameters, multiband image processing of four-wave band image fusion, three-wave band image fusion and two-wave band image fusion is correspondingly realized, and the complementation and the richness of multiband information are fully utilized; u shapeiAnd ViCorresponding to blue and red difference components respectively, the visible light + near infrared Vis and the short wave infrared image can be reflected in a blue difference channel, and the medium wave infrared image and the long wave infrared image are reflected in a red difference channel respectively, so that a color image which is more in line with the visual characteristics of human eyes is obtained; y isiA brightness channel for fusing the images is a gray level fusion result of the multiband images;
and a fourth step of image fusion: according to the initial color image and the corresponding parameters of the corresponding wave band obtained in the image fusion step three, substituting the corresponding parameters into formula (2), namely, transmitting the color of the reference image to the initial colorized image and the corresponding parameters, wherein the corresponding parameters comprise Yi、Ui、Vi；
Wherein, Yo,Uo,VoRespectively obtaining YUV channels of the finally obtained color fusion image; sigmaT,Y,σT,U,σT,VAnd σi,Y,σi,U,σi,VRespectively is the standard deviation of each channel of the color reference image and the initial color image YUV; mu.sT,Y,μT,U,μT,VAnd mui,Y,μi,U,μi,VRespectively being a colour reference pictureMean values of YUV channels of the image and the initial color image;
and a fifth step of image fusion: according to application requirements, the wave bands participating in fusion are controlled by adjusting the values of the 12 parameters in the step three, multiband image processing corresponding to four-wave band color fusion, three-wave band color fusion and two-wave band color fusion is realized according to the image fusion steps three and four, the complementation and richness of multiband information are fully utilized, and the target detection and identification efficiency is improved;
the four wave bands refer to visible light plus near infrared, short wave infrared, medium wave infrared and long wave infrared; the three wave bands refer to optional three wave bands in the four wave bands; the two bands refer to any two bands of the four bands.
4. A multiband image fusion processing method according to claim 3, applied to the four-band co-optical axis optoelectronic imaging platform according to claim 1 or 2, wherein: the multiband image processing corresponding to four-waveband color fusion, three-waveband color fusion and two-waveband color fusion is realized, the complementation and the richness of multiband information are fully utilized, and the target detection and identification efficiency is improved.
5. A mid-wave infrared and long-wave infrared dual-waveband colorimetric temperature measurement method is applied in parallel with the multiband image fusion processing method of claim 3 or 4, and the application range of the four-waveband coaxial-axis photoelectric imaging platform of claim 1 or 2 is expanded; the method is characterized in that: comprises the following steps of (a) carrying out,
temperature measurement step one: simultaneously acquiring medium wave infrared and long wave infrared image information in a target scene;
according to the response principle of the detector, the temperature is measured in two temperature measuring wave bands lambdamin,λmax]The signal level output by the detector is as shown in formula (3);
wherein R isV(lambda) is the spectral response of the temperature-measuring band detectorRate; a is the area of a detector unit; epsilon (lambda) object spectral emissivity; d is the clear aperture of the optical system; f' is the focal length of the optical system; tau isa(λ) is the atmospheric spectral transmittance; tau is0(λ) is optical system transmittance; meb(λ, T) is Planck's law; u (T) is the signal level output by the detector;
a second temperature measurement step: preprocessing the medium-wave infrared image information and the long-wave infrared image information acquired in the temperature measurement step I to obtain preprocessed two-waveband image information; the pretreatment comprises the following steps: carrying out non-uniformity correction on the medium wave infrared image and the long wave infrared image respectively; respectively enhancing the medium wave infrared image and the long wave infrared image;
temperature measurement step three: because the gray value of a single pixel of the detector is positively correlated with the output signal level U (T) of the detector, the ratio of the signal levels of the pixels corresponding to the two detectors is equal to the ratio of the gray values of the pixels corresponding to the two detectors;
the temperature of the measured object is determined by using the ratio of the signal levels of the corresponding pixels of the medium wave infrared and the long wave infrared obtained by the two detectors, the expression of the formula (3) of the medium wave infrared and the long wave infrared is compared, the ratio result is shown as the formula (4),
wherein, U1(T) is the signal level output by the medium wave infrared detector; u shape2(T) is the signal level output by the long-wave infrared detector; q (T) is the ratio of the signal level output by the medium wave infrared detector to the signal level output by the long wave infrared detector;
the double-waveband colorimetric temperature measurement of medium and long wave infrared can greatly reduce the spectral emissivity epsilon (lambda) of a temperature measurement target and the spectral transmittance tau in the transmission processa(lambda) and tau0(λ) influence on temperature measurement accuracy by approximately determining that the ratio of two different band signals is a temperature-related function;
temperature measurement step four: fitting out a curve of gray value ratio Q (T) of each corresponding pixel point of the medium-wave infrared image and the long-wave infrared image along with the change of the temperature T and a corresponding polynomial fitting expression through black body calibration;
a fifth temperature measuring step: and (4) substituting the ratio Q (T) of the signal level output by the medium wave infrared detector and the signal level output by the long wave infrared detector obtained in the temperature measurement step (III) into a polynomial fitting expression in the temperature measurement step (IV) to obtain corresponding temperature T, namely obtaining temperature images with the same resolution as the medium wave infrared image and the long wave infrared image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110837286.9A CN113538314B (en) | 2021-07-23 | 2021-07-23 | Four-band common-optical-axis photoelectric imaging platform and image fusion processing method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110837286.9A CN113538314B (en) | 2021-07-23 | 2021-07-23 | Four-band common-optical-axis photoelectric imaging platform and image fusion processing method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113538314A true CN113538314A (en) | 2021-10-22 |
| CN113538314B CN113538314B (en) | 2024-06-11 |
Family
ID=78120976
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110837286.9A Active CN113538314B (en) | 2021-07-23 | 2021-07-23 | Four-band common-optical-axis photoelectric imaging platform and image fusion processing method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113538314B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114239654A (en) * | 2021-12-17 | 2022-03-25 | 电子科技大学 | Temperature identification and classification method based on dual-waveband pyroelectric device |
| CN116248705A (en) * | 2022-11-29 | 2023-06-09 | 宜昌测试技术研究所 | Multichannel image transmission and processing system of miniature photoelectric pod |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103278916A (en) * | 2013-04-10 | 2013-09-04 | 北京理工大学 | Laser and middle- and long-wavelength infrared common-aperture three-band imaging system |
| US20140078492A1 (en) * | 2012-09-19 | 2014-03-20 | Raytheon Company | Multi-band imaging spectrometer |
| US20160202122A1 (en) * | 2014-12-30 | 2016-07-14 | Huazhong University Of Science And Technology | Detecting device and method combining images with spectrums in ultra-wide waveband |
| CN108154493A (en) * | 2017-11-23 | 2018-06-12 | 南京理工大学 | A kind of pseudo- color blending algorithm of the dual-band infrared image based on FPGA |
| CN108419061A (en) * | 2017-02-10 | 2018-08-17 | 杭州海康威视数字技术股份有限公司 | Based on multispectral image co-registration equipment, method and imaging sensor |
| CN108415097A (en) * | 2017-12-29 | 2018-08-17 | 华中科技大学 | A kind of the collection of illustrative plates cooperative detection system and method for multiband infrared imaging |
| CN112180577A (en) * | 2020-09-25 | 2021-01-05 | 中国科学院西安光学精密机械研究所 | Visible light-short wave infrared-medium wave infrared-long wave infrared four-waveband optical system |
-
2021
- 2021-07-23 CN CN202110837286.9A patent/CN113538314B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140078492A1 (en) * | 2012-09-19 | 2014-03-20 | Raytheon Company | Multi-band imaging spectrometer |
| CN103278916A (en) * | 2013-04-10 | 2013-09-04 | 北京理工大学 | Laser and middle- and long-wavelength infrared common-aperture three-band imaging system |
| US20160202122A1 (en) * | 2014-12-30 | 2016-07-14 | Huazhong University Of Science And Technology | Detecting device and method combining images with spectrums in ultra-wide waveband |
| CN108419061A (en) * | 2017-02-10 | 2018-08-17 | 杭州海康威视数字技术股份有限公司 | Based on multispectral image co-registration equipment, method and imaging sensor |
| CN108154493A (en) * | 2017-11-23 | 2018-06-12 | 南京理工大学 | A kind of pseudo- color blending algorithm of the dual-band infrared image based on FPGA |
| CN108415097A (en) * | 2017-12-29 | 2018-08-17 | 华中科技大学 | A kind of the collection of illustrative plates cooperative detection system and method for multiband infrared imaging |
| CN112180577A (en) * | 2020-09-25 | 2021-01-05 | 中国科学院西安光学精密机械研究所 | Visible light-short wave infrared-medium wave infrared-long wave infrared four-waveband optical system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114239654A (en) * | 2021-12-17 | 2022-03-25 | 电子科技大学 | Temperature identification and classification method based on dual-waveband pyroelectric device |
| CN114239654B (en) * | 2021-12-17 | 2023-05-12 | 电子科技大学 | Temperature identification and classification method based on dual-band pyroelectric device |
| CN116248705A (en) * | 2022-11-29 | 2023-06-09 | 宜昌测试技术研究所 | Multichannel image transmission and processing system of miniature photoelectric pod |
| CN116248705B (en) * | 2022-11-29 | 2024-03-15 | 宜昌测试技术研究所 | Multichannel image transmission and processing system of miniature photoelectric pod |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113538314B (en) | 2024-06-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2394427B1 (en) | Optimized imaging system for collection of high resolution imagery | |
| US8766808B2 (en) | Imager with multiple sensor arrays | |
| CN106973240B (en) | Digital camera imaging method for realizing high-definition display of high-dynamic-range image | |
| CN113538314B (en) | Four-band common-optical-axis photoelectric imaging platform and image fusion processing method thereof | |
| CN102538983B (en) | CCD (Charge Coupled Device) temperature measuring device | |
| CN109506782A (en) | Transient state temperature field test method and its test macro based on high-speed imaging technology | |
| CN105043552B (en) | Colorimetric temperature measurement system display and calibration method | |
| Ren et al. | A method and results of color calibration for the Chang'e-3 terrain camera and panoramic camera | |
| CN104458013A (en) | Engine thermal protection structure temperature field multi-mode measuring system | |
| US6969856B1 (en) | Two band imaging system | |
| CN112710397A (en) | Two-point correction method and system based on temperature substitution | |
| CN109163809B (en) | Multi-aperture view field partially overlapped dual-band thermal imaging method and device | |
| Perić et al. | Analysis of SWIR imagers application in electro-optical systems | |
| CN118209296A (en) | Calibration method of camera system | |
| CN111649830B (en) | Color CCD self-calibration temperature measurement device and method based on radiation spectrum | |
| Hirsh et al. | Hybrid dual-color MWIR detector for airborne missile warning systems | |
| CN212133888U (en) | Color CCD self-calibration temperature measuring device based on radiation spectrum | |
| Xie et al. | Development of three-wavelength CCD image pyrometer used for the temperature field measurements of continuous casting billets | |
| CN115683358A (en) | Radiation calibration device and calibration method based on refrigeration type infrared imaging system | |
| CN113029339B (en) | On-orbit multi-source-tracing spectral radiance calibration method for deep space detection imaging spectrometer | |
| Schreer et al. | Dual-band camera system with advanced image processing capability | |
| Gallastegi et al. | Absorption-based ranging from ambient thermal radiation without known emissivities | |
| Toet et al. | TRICLOBS portable triband color lowlight observation system | |
| Reichle et al. | Multichannel and multicolour infrared thermography in Tore Supra | |
| CN115265786B (en) | Strong light polarization detection device capable of automatically adjusting exposure value and detection method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |