CN104574332A - Image fusion method for airborne optoelectronic pod - Google Patents

Abstract

An image fusion method for an airborne optoelectronic pod comprises steps as follows: (1) adjusting the optical axis of a visible light imaging system and the optical axis of an infrared imaging system to be in a parallel state; (2) performing linkage zoom calculation with the focal length of the visible light imaging system serving as the reference to acquire the corresponding focusing length of the infrared imaging system; (3) calculating respective optical amplification multiples according to the focusing length of the visible light imaging system and the focusing length of the infrared imaging system, performing image scaling, calculating deviation angles according to positions of focusing length sections where the focusing lengths are located, and performing image translation compensation; (4) fusing images of the visible light imaging system and the infrared imaging system after image registration. With the adoption of the method, the visible light imaging system and the infrared imaging system in the pod are associated, infrared image features of targets and visible light image features are fused into one way of image video to be output, and the requirement of the optoelectronic pod for all-weather day-and-light reconnaissance is met to the greatest extent.

Description

A kind of airborne photoelectric gondola image interfusion method

Technical field

The present invention relates to a kind of airborne photoelectric gondola image interfusion method, can realize that continuous vari-focus is infrared, the Real-time image fusion of visible images exports, thus in a road video, obtain infrared, visible ray two kinds of characteristic informations of target simultaneously.

Background technology

Photoelectric nacelle sends video information to ground handling operator by a road video TTC channel in the course of the work, and operating personnel carry out different operating according to video information, the operations such as realize target search, tracking, location.Multi-load photoelectric nacelle is equipped with infrared, Visible imaging system usually, but TTC channel can only transmit single channel video at one time to operating personnel, if need to observe an other road video, then need to perform video switching command, this brings very big inconvenience to embody rule.First, single channel video information is limited, causes target recognition accuracy to decline, meanwhile, easily causes track rejection in handoff procedure, need manual search target again, affect work efficiency.

Summary of the invention

The technical matters that the present invention solves is: overcome the deficiencies in the prior art, provide a kind of airborne photoelectric gondola image interfusion method, Visible imaging system in gondola is associated with infrared imaging system, the Infrared Image Features of target and visible images Fusion Features are exported in a road image/video.At utmost meet photoelectric nacelle to round-the-clock demand of scouting round the clock.

Technical solution of the present invention is:

A kind of airborne photoelectric gondola image interfusion method, be provided with Visible imaging system and infrared imaging system in described airborne photoelectric gondola, described image interfusion method step is as follows:

(1) optical axis calibration: the optical axis of Visible imaging system and the optical axis of infrared imaging system are adjusted to parastate;

(2) calculate for benchmark carries out interlock zoom with the focal length of Visible imaging system, obtain the respective focal value of infrared imaging system, make Visible imaging system and infrared imaging system link, obtain Visible imaging system and infrared imaging system image separately;

(3) image registration: calculate respective optical magnification by the focal length value of Visible imaging system and the focal length value of infrared imaging system and carry out image scaling, calculates deviation angle angle value by focal length fragment position residing for focal length and carries out image translation compensation;

(4) image of the image of the Visible imaging system after image registration and infrared imaging system is carried out image co-registration.

The optical axis of Visible imaging system and the optical axis of infrared imaging system are adjusted to parastate by described step (1), carry out especially by following steps:

(2.1) seek the target of the square boundary got within the scope of 500m ~ 3000m as spotting, the border of described spotting is even;

(2.2) the visible ray cross drone in Visible imaging system is aimed at the border, right angle of described spotting, as reference-calibrating;

(2.3) infrared imaging system is switched to, read the side-play amount of the infrared cross drone in infrared imaging system, the coordinate offset amount of described infrared cross drone refers to: the offset pixel values X between infrared cross drone target center and target longitudinal boundary and offset pixel values Y between infrared cross drone target center and target lateral border;

(2.4) offset pixel values obtained according to step (2.3) calculates the deviation angle angle value C between infrared imaging system and Visible imaging system;

(2.5) calculated the thickness V of the adjustment pad added needed for the described deviation angle angle value C of adjustment by formula V=tanCH, wherein, H is the distance between camera structure two mounting holes on a mounting board;

(2.6) according to the thickness V of the adjustment pad calculated in step (2.5) be infrared imaging system add adjustment pad, the optical axis angle of adjustment infrared imaging system, makes the optical axis of Visible imaging system parallel with the optical axis of infrared imaging system.

Described step (2) calculates for benchmark carries out interlock zoom with the focal length of Visible imaging system, obtains the respective focal value of infrared imaging system, realizes in the following way:

$\mathrm{VF}=\frac{({\mathrm{VF}}_{\max }-{\mathrm{VF}}_{\min })(\mathrm{IF}-{\mathrm{IF}}_{\min })}{{\mathrm{IF}}_{\max }-{\mathrm{IF}}_{\min }}+{\mathrm{VF}}_{\min };$

Wherein VF is the respective focal value of infrared imaging system, and IF is the focal length value of Visible imaging system, VF _maxfor the maximum focus value of infrared imaging system, VF _minfor the minimum focus value of infrared imaging system, IF _maxfor the maximum focus value of Visible imaging system, VF _minfor the minimum focus value of Visible imaging system.

Described step (3) is carried out image registration and is carried out especially by following steps:

(4.1) Visible imaging system focal length value f ' is read _can;

(4.2) infrared imaging system focal length value f ' is read _red;

(4.3) according to the Visible imaging system focal length value f ' obtained in (4.1) _canwith the infrared imaging system focal length value f ' obtained in (4.2) _red, and pass through formula with calculate the projection image height y ' of Visible imaging system on visible light image sensor respectively _canwith the projection image height y ' of infrared imaging system on infrared image sensor _red;

Then infrared imaging system and Visible imaging system imaging size ratio G are:

r _canfor known visible light image sensor pixel dimension, r _redfor infrared image sensor pixel dimension;

(4.4) be that benchmark carries out G image scaling doubly to infrared image with visible images;

(4.5) offset pixels number is calculated by focal length fragment position residing for the focal length of Visible imaging system;

(4.6) needed for the image calculated according to (4.5), offset pixels number carries out translation compensation to the image of infrared imaging system.

Described step (4.5) calculates offset pixels number by focal length fragment position residing for the focal length of Visible imaging system and is specially:

(5.1) focal range of Visible imaging system is divided into 10 focal length sections;

(5.2) Visible imaging system light shaft offset amount relative between infrared imaging system is gathered at 10 focal length section separation places, by the side-play amount numerical value of collection and the focal length value of correspondence thereof, being plotted in ordinate is focal length value, horizontal ordinate is in the coordinate system of side-play amount, be separated by between collection point at two and do approximate fits with linear mode, obtain focal length-side-play amount two-dimensional coordinate system;

(5.3) offset pixels number is calculated according to (5.2) focal length-side-play amount two-dimensional coordinate system.

The present invention's advantage is compared with prior art:

Single a certain road selected video image information can only be exported compared to the current existing various types of photoelectric nacelle that loads more, there is target signature display not exclusively, the deficiency of camouflage of target recognition capability difference.Two-way video characteristics of image real time fusion can, by outer for visible red two-way imaging system interlock, be a road video frequency output by the application of image interfusion method of the present invention in photoelectric nacelle.Greatly strengthen the ability to target signature identification and camouflage of target identification.Requisite solid foundation is provided for realizing the round-the-clock investigation round the clock of photoelectric nacelle.

Accompanying drawing explanation

Fig. 1 is method flow diagram of the present invention;

Fig. 2 is optical axis calibration process flow diagram of the present invention;

Fig. 3 is image registration process flow diagram of the present invention;

Fig. 4 is that object delineation is chosen in optical axis calibration;

Fig. 5 is that optical axis calibration side-play amount reads schematic diagram.

Embodiment

The invention provides a kind of airborne photoelectric gondola image interfusion method, Visible imaging system and infrared imaging system are installed in airborne photoelectric gondola.Principle of the present invention is: carry out calibration to Visible imaging system, the infrared imaging system light axis consistency respectively in zooming procedure, makes it meet the requirement of zoom light axis consistency.Utilize parallelism of optical axis Calibration System to carry out the calibration of optical axis high precision parallelism of optical axis to Visible imaging system and infrared imaging system, make it meet depth of parallelism requirement.By calculating, infrared imaging system being associated with Visible imaging system, being controlled the zoom system, pancreatic system synchronous interaction of two imaging systems by computing machine control chip.It is interval that two optical system focal range differences produce different zoom, adopts digital picture zoom mode to compensate.Parallelism of optical axis error, zooming procedure light axis consistency error, adopt image translation mode to compensate.Thus realize the fusion of visible ray and infrared two-way optical imaging system.

As shown in Figure 1, the step of figure phase fusion method of the present invention is as follows:

(1) optical axis calibration: the optical axis of Visible imaging system and the optical axis of infrared imaging system are adjusted to parastate;

As shown in Figure 2, carry out especially by following steps:

(1.1) as shown in Figure 4, seek the target of the square boundary got within the scope of 500m ~ 3000m as spotting, the border of described spotting is even;

(1.2) as shown in Figure 5, the visible ray cross drone in Visible imaging system is aimed at the border, right angle of described spotting, as reference-calibrating;

(1.3) infrared imaging system is switched to, read the side-play amount of the infrared cross drone in infrared imaging system, the side-play amount of described infrared cross drone refers to: the offset pixel values between infrared cross drone target center and target transverse and longitudinal two borders, wherein transversal displacement is X, and vertical misalignment amount is Y;

(1.4) offset pixel values obtained according to step (1.3) calculates the deviation angle angle value C between infrared imaging system and Visible imaging system;

(1.5) calculate by formula V=tanCH the thickness V that transverse and longitudinal both direction adjusts the adjustment pad added needed for described deviation angle angle value C respectively, wherein, H is the distance between camera structure two mounting holes on a mounting board; In the present invention, camera structure needs to be fixed on the installing plate in gondola, and mounting hole has two, all on a mounting board.

(1.6) according to the thickness V of the adjustment pad calculated in step (1.5) be infrared imaging system add adjustment pad thickness, adjust the optical axis angle of infrared imaging system transverse and longitudinal both direction respectively, make the optical axis of Visible imaging system parallel with the optical axis of infrared imaging system.

(2) calculate for benchmark carries out interlock zoom with the focal length of Visible imaging system, obtain the respective focal value of infrared imaging system, make Visible imaging system and infrared imaging system link, obtain Visible imaging system and infrared imaging system image separately;

Realize in the following way:

$\mathrm{VF}=\frac{({\mathrm{VF}}_{\max }-{\mathrm{VF}}_{\min })(\mathrm{IF}-{\mathrm{IF}}_{\min })}{{\mathrm{IF}}_{\max }-{\mathrm{IF}}_{\min }}+{\mathrm{VF}}_{\min }$

Wherein VF is the respective focal value of infrared imaging system, and IF is the focal length value of Visible imaging system, VF _maxfor the maximum focus value of infrared imaging system, VF _minfor the minimum focus value of infrared imaging system, IF _maxfor the maximum focus value of Visible imaging system, VF _minfor the minimum focus value of Visible imaging system.

(3) image registration: calculate respective optical magnification by the focal length value of Visible imaging system and the focal length value of infrared imaging system and carry out image scaling, calculates deviation angle angle value by focal length fragment position residing for focal length and carries out image translation compensation;

As shown in Figure 3, carry out especially by following steps:

(3.1) Visible imaging system focal length value f ' is read _can;

(3.2) infrared imaging system focal length value f ' is read _red;

(3.3) according to the Visible imaging system focal length value f ' obtained in (3.1) _canwith the infrared imaging system focal length value f ' obtained in (3.2) _red, and pass through formula with calculate the projection image height y ' of Visible imaging system on visible light image sensor respectively _canwith the projection image height y ' of infrared imaging system on infrared image sensor _red;

Then infrared imaging system and Visible imaging system imaging size ratio G are:

Visible light image sensor pixel dimension is r _can, infrared image sensor pixel dimension is r _red;

(3.4) be that benchmark carries out G image scaling doubly to infrared image with visible images;

(3.5) offset pixels number is calculated by focal length fragment position residing for the focal length of Visible imaging system;

Be specially:

A the focal range of Visible imaging system is divided into 10 focal length sections by ();

B () gathers Visible imaging system light shaft offset amount relative between infrared imaging system at 10 focal length section separation places, by the side-play amount numerical value of collection and the focal length value of correspondence thereof, being plotted in ordinate is focal length value, horizontal ordinate is in the coordinate system of side-play amount, be separated by between collection point at two and do approximate fits with linear mode, obtain focal length-side-play amount two-dimensional coordinate system;

C () calculates offset pixels number according to (b) focal length-side-play amount two-dimensional coordinate system;

(3.6) needed for the image calculated according to (3.5), offset pixels number carries out translation compensation to the image of infrared imaging system.

(4) image of the image of the Visible imaging system after image registration and infrared imaging system is carried out image co-registration.

Claims (5)

1. an airborne photoelectric gondola image interfusion method, is characterized in that: be provided with Visible imaging system and infrared imaging system in described airborne photoelectric gondola, and described image interfusion method step is as follows:

(1) optical axis calibration: the optical axis of Visible imaging system and the optical axis of infrared imaging system are adjusted to parastate;

(2) calculate for benchmark carries out interlock zoom with the focal length of Visible imaging system, obtain the respective focal value of infrared imaging system, make Visible imaging system and infrared imaging system link, obtain Visible imaging system and infrared imaging system image separately;

(3) image registration: calculate respective optical magnification by the focal length value of Visible imaging system and the focal length value of infrared imaging system and carry out image scaling, calculates deviation angle angle value by focal length fragment position residing for focal length and carries out image translation compensation;

(4) image of the image of the Visible imaging system after image registration and infrared imaging system is carried out image co-registration.

2. a kind of photoelectric nacelle figure phase fusion method according to claim 1, is characterized in that: the optical axis of Visible imaging system and the optical axis of infrared imaging system are adjusted to parastate by described step (1), carry out especially by following steps:

(2.1) seek the target of the square boundary got within the scope of 500m ~ 3000m as spotting, the border of described spotting is even;

(2.2) the visible ray cross drone in Visible imaging system is aimed at the border, right angle of described spotting, as reference-calibrating;

(2.3) infrared imaging system is switched to, read the side-play amount of the infrared cross drone in infrared imaging system, the coordinate offset amount of described infrared cross drone refers to: the offset pixel values X between infrared cross drone target center and target longitudinal boundary and offset pixel values Y between infrared cross drone target center and target lateral border;

(2.4) offset pixel values obtained according to step (2.3) calculates the deviation angle angle value C between infrared imaging system and Visible imaging system;

(2.5) calculated the thickness V of the adjustment pad added needed for the described deviation angle angle value C of adjustment by formula V=tanCH, wherein, H is the distance between camera structure two mounting holes on a mounting board;

(2.6) according to the thickness V of the adjustment pad calculated in step (2.5) be infrared imaging system add adjustment pad, the optical axis angle of adjustment infrared imaging system, makes the optical axis of Visible imaging system parallel with the optical axis of infrared imaging system.

3. a kind of photoelectric nacelle figure phase fusion method according to claim 1, it is characterized in that: described step (2) calculates for benchmark carries out interlock zoom with the focal length of Visible imaging system, obtain the respective focal value of infrared imaging system, realize in the following way:

$\mathrm{VF}=\frac{({\mathrm{VF}}_{\max }-{\mathrm{VF}}_{\min })(\mathrm{IF}-{\mathrm{IF}}_{\min })}{{\mathrm{IF}}_{\max }-{\mathrm{IF}}_{\min }}+{\mathrm{VF}}_{\min };$

Wherein VF is the respective focal value of infrared imaging system, and IF is the focal length value of Visible imaging system, VF _maxfor the maximum focus value of infrared imaging system, VF _minfor the minimum focus value of infrared imaging system, IF _maxfor the maximum focus value of Visible imaging system, VF _minfor the minimum focus value of Visible imaging system.

4. a kind of photoelectric nacelle figure phase fusion method according to claim 1, is characterized in that: described step (3) is carried out image registration and carried out especially by following steps:

(4.1) Visible imaging system focal length value f' is read _can;

(4.2) infrared imaging system focal length value f' is read _red;

(4.3) according to the Visible imaging system focal length value f' obtained in (4.1) _canwith the infrared imaging system focal length value f' obtained in (4.2) _red, and pass through formula with calculate the projection image height y' of Visible imaging system on visible light image sensor respectively _canwith the projection image height y' of infrared imaging system on infrared image sensor _red;

Then infrared imaging system and Visible imaging system imaging size ratio G are:

r _canfor known visible light image sensor pixel dimension, r _redfor infrared image sensor pixel dimension;

(4.4) be that benchmark carries out G image scaling doubly to infrared image with visible images;

(4.5) offset pixels number is calculated by focal length fragment position residing for the focal length of Visible imaging system;

(4.6) needed for the image calculated according to (4.5), offset pixels number carries out translation compensation to the image of infrared imaging system.

5. a kind of photoelectric nacelle figure phase fusion method according to claim 4, is characterized in that: described step (4.5) calculates offset pixels number by focal length fragment position residing for the focal length of Visible imaging system and is specially:

(5.1) focal range of Visible imaging system is divided into 10 focal length sections;

(5.2) Visible imaging system light shaft offset amount relative between infrared imaging system is gathered at 10 focal length section separation places, by the side-play amount numerical value of collection and the focal length value of correspondence thereof, being plotted in ordinate is focal length value, horizontal ordinate is in the coordinate system of side-play amount, be separated by between collection point at two and do approximate fits with linear mode, obtain focal length-side-play amount two-dimensional coordinate system;

(5.3) offset pixels number is calculated according to (5.2) focal length-side-play amount two-dimensional coordinate system.