CN103390281B - A kind of two spectrum night vision device onboard systems and two spectrum Combined design method - Google Patents
A kind of two spectrum night vision device onboard systems and two spectrum Combined design method Download PDFInfo
- Publication number
- CN103390281B CN103390281B CN201310323416.2A CN201310323416A CN103390281B CN 103390281 B CN103390281 B CN 103390281B CN 201310323416 A CN201310323416 A CN 201310323416A CN 103390281 B CN103390281 B CN 103390281B
- Authority
- CN
- China
- Prior art keywords
- image
- spectrum
- infrared
- registration
- completing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Landscapes
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of two spectrum night vision device onboard systems and two spectrum Combined design method, comprise the infrared pair of light spectrum image-forming module for the imaging to visible spectrum and infrared spectrum and the collection to view data, process, fusion and transmission; For completing the image receiving system that view data receives; For completing the display terminal that image shows to observer; For completing the control terminal of the control to imaging pattern function and switch.The present invention adopts electronic fusion mode can solve two spectrum picture with imaging shaft offset issue, can reduce imaging system requirement on machining accuracy, for the observation of train forward sight provides image more clearly; Fusion method based on scene motion target provides reference source for two spectrum imaging system, reduces operand and improves registration reliability, be convenient to hardware implementing, possess real-time; The Weighted Fusion method of bilateral filtering is coordinated based on Multiscale Theory, simple, promote axis information information, reduce arithmetic capability.
Description
Technical field
The invention belongs to train forward sight technical field, particularly relate to a kind of two spectrum night vision device onboard systems and two spectrum Combined design method.
Background technology
Night vision device shifts the technology of coming from tank night vision device, is that a typical military technology transfers civilian product to.Along with the fast development of train, high-speed rail transportation and security protection consciousness are strengthened day by day, the requirement of people to train safe safeguards technique is more and more higher, and existing train illumination system sighting distance in rainy night, the snow visibility such as sky or dense fog lower situation is not far, one of the disadvantage potential safety hazard becoming train driving of poor effect.
Existing train forward sight recording geometry adopts CCD or cmos imaging system supplymentary observation, but this single spectrum picture cannot reflect road conditions at inclement weather at all, especially avenges at mist, the rainy day is more obvious.
Infrared imaging system resolution is low, and details, contrast are not good, are not easily used alone yet.
Low-light Level Night Vision Technology, by being with the night vision goggles of image intensifier tube, the weak target picture bright to illumination at night strengthens, for the photoelectric imaging technology observed.There is the phenomenon excessively strengthened in Low-light Level Night Vision Technology, being subject to weather condition affects work, the problem of noiseproof feature difference by day.
In image fusion technology field, existing registration technology is immature, especially at infrared and visible ray registration, because infrared light and the visible ray difference in picture characteristics is large, correlativity is little, adds in scene information not exist to lack known unique point, and therefore registration is unreliable, registration calculated amount is large in addition, does not also find that there is complete in algorithm, stable registration Algorithm preferably at present.
Summary of the invention
The object of the embodiment of the present invention is to provide a kind of two spectrum night vision device onboard systems and two spectrum Combined design method, be intended to solve the near problem of existing inclement weather Visible imaging system observed range, the not easily observation problem that infrared imaging system resolution is low, contrast is fuzzy, characteristic information is indefinite, target detail is not easy the aspects such as identification, solve single mode sensing system weak effect problem, solve the integrity problem that two spectrum picture merges registration.
The embodiment of the present invention is achieved in that a kind of two spectrum night vision device onboard system, and before this pair of spectrum, viewing system comprises: infrared pair of light spectrum image-forming module, image receiving system, display terminal, control terminal;
For infrared pair of light spectrum image-forming module of the imaging to visible spectrum and infrared spectrum and the collection to view data, process, fusion and transmission;
For completing the image receiving system that view data receives, connecting infrared pair of light spectrum image-forming module, receiving the imaging signal of infrared pair of light spectrum image-forming module;
For completing the display terminal that image shows to observer, connecting image receiving system, receiving the picture signal of image receiving system;
For completing the control terminal to imaging pattern function and switch control rule, connect infrared pair of light spectrum image-forming module.
Further, infrared pair of light spectrum image-forming module also comprises: infrared optics object lens, infrared camera, visible light optical object lens, Visible Light Camera, image co-registration treatment circuit, image encoder, image delivering system;
Filtration, infrared light for completing visible ray are anti-reflection, the infrared optics object lens of focus adjustment function;
For completing the collection of infrared signal, by the infrared camera of infrared spectroscopy signals to the conversion of electric signal;
The visible light optical object lens that visible ray is anti-reflection for completing, focus adjustment and aperture regulate;
For completing the collection of visible ray, by the Visible Light Camera that visible light signal is changed to electric signal;
For completing the image co-registration treatment circuit of two spectrum picture brightness regulation, geometry correction, centre registration, fusion, enhancing;
Being connected with image co-registration treatment circuit, for completing the image encoder of digital picture to the conversion of standard image format, receiving the image of image co-registration treatment circuit;
Be connected with image co-registration treatment circuit, for completing the image delivering system of the outside transfer function of image, connect image encoder, completing the transmission of image.
Further, image co-registration treatment circuit also comprises: FPGA, DSP, dimension correction memory RAM, FLASH.
FPGA, connect Visible Light Camera and infrared camera, detail extraction, the contours extract of infrared image is jointly completed in conjunction with dimension correction storer, complete the detail extraction of visible images, contours extract, complete image outline coupling, complete the conversion of registration geometric scale, complete image detail and profile fusion, complete a series of process of image enhaucament, simultaneously to the cooperation control between each circuit unit of whole fusion treatment Circuits System;
DSP, connects image encoder, in conjunction with the RAM connected, FLASH, for treated digital video signal combination row, field sync signal are synthesized analog video signal and go forward side by side row cutting so that display.DSP, connects image encoder, in conjunction with the SRAM connected, FLASH, for treated digital video signal combination row, field sync signal are synthesized analog video signal and go forward side by side row cutting so that display.
Further, display terminal adopts LCD screen to show.
Further, control terminal adopts the mode of button to control.
The object of the embodiment of the present invention is the method for designing providing a kind of pair of spectrum night vision device onboard system, and the key Design method of this pair of spectrum night vision device onboard system comprises two spectrum picture fusion method design.
The gap of infrared system imaging effect and visual light imaging effect is mainly manifested in that resolution is low, contrast is fuzzy, characteristic information is indefinite, target detail is not easy aspects such as recognizing, needs to carry out special enhancing algorithm process; VISIBLE LIGHT SYSTEM imaging system is near at night, misty rain weather image-forming range, does not possess penetration power, affects greatly by light condition.Need to use under various weather environment as viewing system before train, adopt separately the forward sight system effect of a type not good.Because visible ray and infrared image difference in characteristics of image is larger, correlativity is poor, traditional integration technology is the fusion based on characteristics of image or mutual information completely, calculated amount is very large, and because the conditions such as scene light change, considerable influence is produced to the extraction of feature based or mutual information, cause the stability of registration lower.In this case, the present invention proposes and adopt the method for registering based on scene motion track to solve two spectrum registration reference source problem, adopt the method for multiple dimensioned details Weighted Fusion promote axis information information and reduce fusion calculation amount.Meanwhile, the present invention have studied the real time fusion system of a kind of infrared spectrum imaging and visible light imaging.Its method is fixed at two spectrum camera; and visual field overlapping when; first adopt same template to carry out calibration to two spectrum imaging system, the system when change of scale matrix that generation designs in advance is for truly applying is calibrated in advance, is not protected in the present invention of this Calibration Method.Because the problems such as installation question, vibration problems or sensor ageing, lens focus conversion cause imaging system images effect and preset value to offset in true application process, at this moment need to carry out smart registration, here the method adopted realizes smart registration based on scene motion target trajectory method, adopts the method for multiple dimensioned Weighted Fusion to realize two spectrum picture and merge after rough registration and smart registration.Concrete steps are as follows:
Steps A, visible spectrum image and infrared spectroscopic imaging carried out to enhancing process, adopt the change of scale matrix demarcated in advance to complete rough registration and demarcate.
Step B, by the single object in the single object in infrared visual field and visible ray visual field is carried out motion detect, determine its movement locus.Complete the matching of two movement locus, determine the change of scale matrix of essence coupling, use this matrix to generate infrared spectroscopic imaging to be fused and visible light image.
Step B comprises following concrete steps:
B1: visible images carries out inter-frame difference motion and detects between multiframe, utilizes mean-shift algorithm to carry out motion tracking; Infrared image carries out inter-frame difference detection between multiframe, utilizes mean-shift algorithm to carry out motion tracking.And record movement locus in the picture.
B2: curve registration is carried out to two kinds of movement locus.
B3: generate registration transformation matrix according to registration result, and geometric scale conversion is carried out to one of them image.
Step C, separately nonoverlapping piece is divided into infrared spectroscopic imaging and visible light image, calculates quantity of information wherein respectively.Gaussian pyramid is used to be divided into the image of different scale infrared spectroscopic imaging and visible light image, minutia is extracted at different scale, and be weighted assessment according to quantity of information feature, finally adopt laplacian pyramid to synthesize, form new fused images.
Step C comprises following concrete steps:
C1: use gaussian pyramid to be divided into the image of different scale infrared spectroscopic imaging and visible light image;
C2: use bilateral filtering to extract minutia at different scale;
C3: superposition is weighted to minutia at different scale.
C4: adopt laplacian pyramid to synthesize, form new fused images.
Step D, to merge after image carry out enhancings process.
The present invention adopts technique scheme compared with existing technical scheme, has following technique effect:
(1) the present invention adopts infrared camera and Visible Light Camera to carry out the method for registration fusion, relative to adopting separately viewing system before infrared camera or Visible Light Camera, in round-the-clock, sharpness, has more standby imaging advantage.
(2) the technical program is simple, realizes image co-registration treatment circuit and can play whole system by rapid build by the method for the system integration, can realize prototype, be conducive to enhancing productivity.
(3) adopt the scheme detecting also registration based on scene motion target trajectory, for different sensors system provides same reference source when actual scene is applied, be conducive to promoting registration accuracy and registration reliability.The registration approach of based target track, more simple and reliable relative to the existing method for registering based on image outline, the method for registering based on mutual information, and operand is little, is convenient to real-time implementation.
(4) Multiscale Theory is adopted to coordinate bilateral filtering minutia to be weighted the scheme of fusion, more can by the Weak target feature instantiation in infrared spectroscopic imaging, visible light image out.Fused images is made to have more standby ornamental.This registration approach is simple, is convenient to hardware implementing.
Provided by the invention pair of spectrum night vision device onboard system, by setting up user's control circuit module, form and do not limit, user, according to the control overflow designed, designed of equipment, achieves the changeability of equipment and the variation of controlling functions.The present invention, by computer module and the closed-loop control of user's control circuit module composition, meets the correctness of authentication of users control circuit and control software design, the needs of accuracy.
In addition, effect of the present invention is as follows: 1) adopt electronic fusion mode can solve two spectrum picture with imaging shaft offset issue, imaging system requirement on machining accuracy can be reduced, 2) problem of single light spectrum image-forming poor effect can be solved, for the observation of train forward sight provides image more clearly.3) fusion method based on scene motion target provides reference source for two spectrum imaging system, reduces operand and improves registration reliability, be convenient to hardware implementing, possess real-time.4) the Weighted Fusion method of bilateral filtering is coordinated based on Multiscale Theory, simple, promote axis information information, reduce arithmetic capability.
Accompanying drawing explanation
Fig. 1 is the structured flowchart of two spectrum night vision device onboard systems that the embodiment of the present invention provides;
Fig. 2 is the structured flowchart of the infrared pair of light spectrum image-forming module that the embodiment of the present invention provides.
In figure: 1, infrared pair of light spectrum image-forming module; 2, image receiving system; 3, display terminal; 4, control terminal; 1-1, infrared optics object lens; 1-2, infrared camera; 1-3, image co-registration treatment circuit; 1-4, image encoder; 1-5, image delivering system; 1-6, visible light optical object lens; 1-7 Visible Light Camera.
Embodiment
In order to make object of the present invention, technical scheme and advantage clearly understand, below in conjunction with embodiment, the present invention is further elaborated.Should be appreciated that specific embodiment described herein only in order to explain the present invention, be not intended to limit the present invention.
Fig. 1 shows provided by the invention pair of spectrum night vision device onboard system structure.For convenience of explanation, illustrate only part related to the present invention.
Of the present invention pair of spectrum night vision device onboard system, this pair of spectrum night vision device onboard system comprises: infrared pair of light spectrum image-forming module, image receiving system, display terminal, control terminal;
For infrared pair of light spectrum image-forming module of the imaging to visible spectrum and infrared spectrum and the collection to view data, process, fusion and transmission;
For completing the image receiving system that view data receives, connecting infrared pair of light spectrum image-forming module, receiving the imaging signal of infrared pair of light spectrum image-forming module;
For completing the display terminal that image shows to observer, connecting image receiving system, receiving the picture signal of image receiving system;
For completing the control terminal to imaging pattern function and switch control rule, connect infrared pair of light spectrum image-forming module.
As a prioritization scheme of the embodiment of the present invention, infrared pair of light spectrum image-forming module also comprises: infrared optics object lens, infrared camera, visible light optical object lens, Visible Light Camera, image co-registration treatment circuit, image encoder, image delivering system;
Filtration, infrared light for completing visible ray are anti-reflection, the infrared optics object lens of focus adjustment function;
For completing the collection of infrared signal, by the infrared camera of infrared spectroscopy signals to the conversion of electric signal;
The visible light optical object lens that visible ray is anti-reflection for completing, focus adjustment and aperture regulate;
For completing the collection of visible ray, by the Visible Light Camera that visible light signal is changed to electric signal;
For completing the image co-registration treatment circuit of two spectrum picture brightness regulation, geometry correction, centre registration, fusion, enhancing;
Being connected with image co-registration treatment circuit, for completing the image encoder of digital picture to the conversion of standard image format, receiving the image of image co-registration treatment circuit;
Be connected with image co-registration treatment circuit, for completing the image delivering system of the outside transfer function of image, connect image encoder, completing the transmission of image.
As a prioritization scheme of the embodiment of the present invention, image co-registration treatment circuit also comprises: FPGA, DSP, dimension correction memory RAM, FLASH.
FPGA, connect Visible Light Camera and infrared camera, detail extraction, the contours extract of infrared image is jointly completed in conjunction with dimension correction storer, complete the detail extraction of visible images, contours extract, complete image outline coupling, complete the conversion of registration geometric scale, complete image detail and profile fusion, complete a series of process of image enhaucament, simultaneously to the cooperation control between each circuit unit of whole fusion treatment Circuits System;
DSP, connects image encoder, in conjunction with the RAM connected, FLASH, for treated digital video signal combination row, field sync signal are synthesized analog video signal and go forward side by side row cutting so that display.DSP, connects image encoder, in conjunction with the RAM connected, FLASH, for treated digital video signal combination row, field sync signal are synthesized analog video signal and go forward side by side row cutting so that display.
As a prioritization scheme of the embodiment of the present invention, display terminal adopts LCD screen to show.
As a prioritization scheme of the embodiment of the present invention, control terminal adopts the mode of button to control.
Below in conjunction with drawings and the specific embodiments, application principle of the present invention is further described.
1.1 application principle
As shown in Figure 1, two spectrum night vision device onboard systems of the embodiment of the present invention primarily of infrared pair of light spectrum image-forming module 1, image receiving system 2, display terminal 3, control terminal 4 forms;
Infrared pair of light spectrum image-forming module 1, connects image receiving system 2 and control terminal 4, for the imaging to visible ray and the collection to view data, process and transmission;
Image receiving system 2, connects display terminal 3, for completing the reception of view data;
Display terminal 3, shows to observer for completing image;
Control terminal 4, for completing the control to the functions such as imaging pattern and switch,
Infrared pair of light spectrum image-forming module 1 by infrared optics object lens 1-1, infrared camera 1-2, image co-registration treatment circuit 1-3, image encoder 1-4, image delivering system 1-5, shown in visible light optical object lens 1-6, Visible Light Camera 1-7 Fig. 1:
The functions such as infrared optics object lens 1-1, connects infrared camera 1-2, focus adjustment anti-reflection for filtration, the infrared light completing infrared light;
Infrared camera 1-2, connection layout is as fusion treatment circuit 1-3, and for completing the collection of infrared signal, light signal is to the conversion of electric signal;
Image co-registration treatment circuit 1-3, connects image encoder 1-4, for completing two spectrum picture brightness regulation, geometry correction, centre registration, fusion, enhancing process;
Image encoder 1-4, connection layout image transmission system 1-5, for completing the conversion of digital picture to standard image format;
Image delivering system 1-5, for sending view data.
Visible ray object lens 1-6, is connected with Visible Light Camera 1-7, and for completing, visible ray is anti-reflection, focus adjustment and aperture regulate.
Visible Light Camera 1-7, is connected with image co-registration treatment circuit 1-3, for realizing the conversion of visible spectrum signal to electric signal.
The detailed composition structure of image co-registration treatment circuit 1-3 is as shown in Figure 2:
Image co-registration treatment circuit 1-3 by FPGA, DSP, dimension correction memory RAM, FLASH form:
FPGA, connect Visible Light Camera and infrared camera, detail extraction, the contours extract of infrared image is jointly completed in conjunction with dimension correction storer, complete the detail extraction of visible images, contours extract, complete image outline coupling, complete the conversion of registration geometric scale, complete image detail and profile fusion, complete a series of process of image enhaucament, simultaneously to the cooperation control between each circuit unit of whole fusion treatment Circuits System;
DSP, connects image encoder, in conjunction with the RAM connected, FLASH, for treated digital video signal combination row, field sync signal are synthesized analog video signal and go forward side by side row cutting so that display.Display terminal 3 adopts LCD screen to show, and control terminal 4 adopts the mode of button to control.
The gap of infrared system imaging effect and visual light imaging effect is mainly manifested in that resolution is low, contrast is fuzzy, characteristic information is indefinite, target detail is not easy aspects such as recognizing, needs to carry out special enhancing algorithm process; VISIBLE LIGHT SYSTEM imaging system is near at night, misty rain weather image-forming range, does not possess penetration power, affects greatly by light condition.Need to use under various weather environment as viewing system before train, adopt separately the forward sight system effect of a type not good, therefore the present invention have studied the real time fusion system of a kind of infrared spectrum imaging and visible light imaging.Its method is fixed at two spectrum camera, and when visual field is overlapping, first adopt same template to carry out calibration to two spectrum imaging system, the thick correction matrix when change of scale matrix that generation designs in advance is for truly applying.Because the problems such as installation question, vibration problems or sensor ageing, lens focus conversion cause imaging system images effect and preset value to offset in true application process, at this moment need to carry out smart registration, here the method adopted realizes smart registration based on the method for scene motion target detection, adopts the method for multiple dimensioned Weighted Fusion to realize two spectrum picture and merge after rough registration and smart registration.Concrete steps are as follows:
Steps A, visible spectrum image and infrared spectroscopic imaging carried out to enhancing process, adopt the change of scale matrix demarcated in advance to complete rough registration and demarcate.
Step B, by the single object in the single object in infrared visual field and visible ray visual field is carried out motion detect, determine its movement locus.Complete the matching of two movement locus, determine the change of scale matrix of essence coupling, use this matrix to generate infrared spectroscopic imaging to be fused and visible light image.
Step B comprises following concrete steps:
B1: visible images carries out inter-frame difference motion and detects between multiframe, utilizes mean-shift algorithm to carry out motion tracking; Infrared image carries out inter-frame difference detection between multiframe, utilizes mean-shift algorithm to carry out motion tracking.And record movement locus in the picture.
B2: curve registration is carried out to two kinds of movement locus.
B3: generate registration transformation matrix according to registration result, and geometric scale conversion is carried out to one of them image.
Step C, separately nonoverlapping piece is divided into infrared spectroscopic imaging and visible light image, calculates quantity of information wherein respectively.Gaussian pyramid is used to be divided into the image of different scale infrared spectroscopic imaging and visible light image, minutia is extracted at different scale, and be weighted assessment according to quantity of information feature, finally adopt laplacian pyramid to synthesize, form new fused images.
Step C comprises following concrete steps:
C1: use gaussian pyramid to be divided into the image of different scale infrared spectroscopic imaging and visible light image;
C2: use bilateral filtering to extract minutia at different scale;
C3: superposition is weighted to minutia at different scale.
C4: adopt laplacian pyramid to synthesize, form new fused images.
Step D, to merge after image carry out enhancings process.
1.2 principle of work of the present invention
Infrared optics object lens 1-1 in infrared pair of light spectrum image-forming module 1 receives visible ray and filters, infrared light is anti-reflection, the work such as focus adjustment, infrared focus plane sensor in infrared camera 1-2 completes the conversion of light signal to electric signal, driving circuit drives infrared focus plane sensor, bias voltage needed for infrared focus plane normal operation of sensor is provided, clock and control signal, temperature signal etc., the A/D that data acquisition circuit completes the analog video signal that sensor exports changes and is transferred to image co-registration treatment circuit 1-3, FPGA in image co-registration treatment circuit 1-3 combines the patch memory SRAM connected on it, blind spot storer SRAM and EEPROM, jointly complete and Nonuniformity Correction is carried out to view data, image enhaucament, a series of process such as blind element compensation, cooperation control between simultaneously completing each circuit unit of whole infrared imaging Circuits System, DSP combines the SRAM connected on it, treated digital video signal is combined row by FLASH, field sync signal synthesizes analog video signal and goes forward side by side row cutting so that display, complete the correction of image, strengthen, contrast adjustment, overheated condition calculates, the image of final generation high-resolution, image encoder 1-4 completes the conversion of digital picture to standard image format, image transmits to image receiving system 2 and control terminal 4 by image delivering system 1-5, view data receives and transfers to display terminal 3 by image receiving system 2, complete image to show to observer, user completes the control to the functions such as imaging pattern and switch by control terminal 4.
The foregoing is only preferred embodiment of the present invention, not in order to limit the present invention, all any amendments done within the spirit and principles in the present invention, equivalent replacement and improvement etc., all should be included within protection scope of the present invention.
Claims (4)
1. a two spectrum night vision device onboard system, it is characterized in that, this pair of spectrum night vision device onboard system comprises:
For infrared pair of light spectrum image-forming module of the imaging to visible spectrum and infrared spectrum and the collection to view data, process, fusion and transmission;
For completing the image receiving system that view data receives, connecting infrared pair of light spectrum image-forming module, receiving the imaging signal of infrared pair of light spectrum image-forming module;
For completing the display terminal that image shows to observer, connecting image receiving system, receiving the picture signal of image receiving system;
For completing the control terminal of the control to imaging pattern function and switch, connect infrared pair of light spectrum image-forming module;
Infrared pair of light spectrum image-forming module also comprises: infrared optics object lens, infrared camera, visible light optical object lens, Visible Light Camera, image co-registration treatment circuit, image encoder, image delivering system;
Filtration, infrared light for completing infrared light are anti-reflection, the infrared optics object lens of focus adjustment function;
For completing the collection of infrared signal, by the infrared camera of infrared spectroscopy signals to the conversion of electric signal;
The visible light optical object lens that visible ray is anti-reflection for completing, focus adjustment and aperture regulate;
For completing the collection of visible ray, by the Visible Light Camera that visible light signal is changed to electric signal;
For completing the image co-registration treatment circuit of two spectrum picture brightness regulation, geometry correction, centre registration, fusion, enhancing;
Being connected with image co-registration treatment circuit, for completing the image encoder of digital picture to the conversion of standard image format, receiving the image of image co-registration treatment circuit;
For completing the image delivering system of the outside transfer function of image, connecting image encoder, completing the transmission of image;
Image co-registration treatment circuit also comprises: FPGA, DSP, dimension correction memory RAM, FLASH:
FPGA, connect Visible Light Camera and infrared camera, detail extraction, the contours extract of infrared image is jointly completed in conjunction with dimension correction storer, complete the detail extraction of visible images, contours extract, complete image outline coupling, complete the conversion of registration geometric scale, complete image detail and profile fusion, complete a series of process of image enhaucament, simultaneously to the cooperation control between each circuit unit of whole fusion treatment Circuits System;
DSP, connects image encoder, in conjunction with the RAM connected, FLASH, for treated digital video signal combination row, field sync signal are synthesized analog video signal and go forward side by side row cutting so that display.
2. two spectrum Combined design methods of a two spectrum night vision device onboard system, it is characterized in that, the infrared spectroscopic imaging that the method utilizes two spectrum observation to arrive and visible spectrum image, the change of scale matrix designed in advance is utilized to complete rough registration, adopt the method based on scene motion target detection to complete smart registration, after registration completes, the method for the multiple dimensioned Weighted Fusion of employing generates fused images; Concrete step is as follows:
Steps A, visible spectrum image and infrared spectroscopic imaging carried out to enhancing process, adopt the change of scale matrix demarcated in advance to complete rough registration and demarcate;
Step B, by the single object in the single object in infrared visual field and visible ray visual field is carried out motion detect, determine its movement locus; Complete the matching of two movement locus, determine the change of scale matrix of essence coupling, use this matrix to generate infrared spectroscopic imaging to be fused and visible light image;
Step C, separately nonoverlapping piece is divided into infrared spectroscopic imaging and visible light image, calculates quantity of information wherein respectively; Gaussian pyramid is used to be divided into the image of different scale infrared spectroscopic imaging and visible light image, minutia is extracted at different scale, and be weighted assessment according to quantity of information feature, finally adopt laplacian pyramid to synthesize, form new fused images;
Step D, to merge after image carry out enhancings process.
3. two spectrum Combined design methods of two spectrum night vision device onboard system as claimed in claim 2, it is characterized in that, step B comprises following steps:
B1: visible images carries out inter-frame difference motion and detects between multiframe, utilizes mean-shift algorithm to carry out motion tracking; Infrared image carries out inter-frame difference detection between multiframe, utilizes mean-shift algorithm to carry out motion tracking, and records movement locus in the picture;
B2: curve registration is carried out to two kinds of movement locus;
B3: generate registration transformation matrix according to registration result, and geometric scale conversion is carried out to one of them image.
4. two spectrum Combined design methods of two spectrum night vision device onboard system as claimed in claim 2, it is characterized in that, step C comprises following steps:
C1: use gaussian pyramid to be divided into the image of different scale infrared spectroscopic imaging and visible light image;
C2: use bilateral filtering to extract minutia at different scale;
C3: superposition is weighted to minutia at different scale;
C4: adopt laplacian pyramid to synthesize, form new fused images.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310323416.2A CN103390281B (en) | 2013-07-29 | 2013-07-29 | A kind of two spectrum night vision device onboard systems and two spectrum Combined design method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310323416.2A CN103390281B (en) | 2013-07-29 | 2013-07-29 | A kind of two spectrum night vision device onboard systems and two spectrum Combined design method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103390281A CN103390281A (en) | 2013-11-13 |
| CN103390281B true CN103390281B (en) | 2016-04-13 |
Family
ID=49534540
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310323416.2A Expired - Fee Related CN103390281B (en) | 2013-07-29 | 2013-07-29 | A kind of two spectrum night vision device onboard systems and two spectrum Combined design method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103390281B (en) |
Families Citing this family (31)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| LU92406B1 (en) * | 2014-03-19 | 2015-09-21 | Iee Sarl | Camera head with integrated calibration tables |
| KR101601475B1 (en) * | 2014-08-25 | 2016-03-21 | 현대자동차주식회사 | Pedestrian detection device and method for driving vehicle at night |
| WO2016029403A1 (en) * | 2014-08-28 | 2016-03-03 | 中国科学院自动化研究所 | Head-mounted molecular image navigation system |
| CN104299231B (en) * | 2014-09-26 | 2017-04-19 | 北京环境特性研究所 | Method and system for registering images of multiple sensors in real time |
| CN106101530A (en) * | 2016-06-12 | 2016-11-09 | 张家港全智电子科技有限公司 | A kind of method that high-speed adaptability night vision image strengthens |
| CN106161996A (en) * | 2016-08-26 | 2016-11-23 | 张家港全智电子科技有限公司 | A kind of high-speed adaptability night vision image strengthens system |
| CN107918929B (en) * | 2016-10-08 | 2019-06-21 | 杭州海康威视数字技术股份有限公司 | A kind of image interfusion method, apparatus and system |
| CN108088565A (en) * | 2016-11-23 | 2018-05-29 | 杭州海康威视数字技术股份有限公司 | Transmit the method and device of scene of fire image, hand-held fire-fighting thermal imaging system |
| CN106803940A (en) * | 2017-03-06 | 2017-06-06 | 广州英卓电子科技有限公司 | A kind of far infrared intelligent monitoring system |
| CN107128245A (en) * | 2017-04-10 | 2017-09-05 | 江苏车视杰电子有限公司 | A kind of vehicle blind zone early warning system |
| CN107203987A (en) * | 2017-06-07 | 2017-09-26 | 云南师范大学 | A kind of infrared image and low-light (level) image real time fusion system |
| CN107886523A (en) * | 2017-11-01 | 2018-04-06 | 武汉大学 | Vehicle target movement velocity detection method based on unmanned plane multi-source image |
| CN109951617A (en) * | 2017-12-21 | 2019-06-28 | 中国科学院深圳先进技术研究院 | A kind of double spectrum high speed cameras based on fpga |
| CN108154494B (en) * | 2017-12-25 | 2019-05-14 | 北京航空航天大学 | A kind of image fusion system based on low-light and infrared sensor |
| CN109064501A (en) * | 2018-09-14 | 2018-12-21 | 南京理工技术转移中心有限公司 | A kind of working method of sewage treatment monitoring system |
| CN109270853A (en) * | 2018-09-14 | 2019-01-25 | 南京理工技术转移中心有限公司 | A kind of working method of intelligent ecological Water Management System |
| CN109345541A (en) * | 2018-09-14 | 2019-02-15 | 南京理工技术转移中心有限公司 | A kind of working method of intelligent monitor system |
| CN109191454A (en) * | 2018-09-14 | 2019-01-11 | 南京理工技术转移中心有限公司 | A kind of working method of electric power monitoring system |
| CN109308702A (en) * | 2018-09-14 | 2019-02-05 | 南京理工技术转移中心有限公司 | A kind of real-time recognition positioning method of target |
| CN109319619A (en) * | 2018-09-14 | 2019-02-12 | 南京理工技术转移中心有限公司 | A kind of working method of elevator intelligent monitoring system |
| CN109151287A (en) * | 2018-10-30 | 2019-01-04 | 泰州市华润纺织品有限公司 | A kind of military vehicle-mounted auxiliary night vision device system |
| CN109788170A (en) * | 2018-12-25 | 2019-05-21 | 合肥芯福传感器技术有限公司 | It is a kind of based on infrared with video image processing system and method for visible light |
| CN110210292A (en) * | 2019-04-23 | 2019-09-06 | 江西理工大学 | A kind of target identification method based on deep learning |
| CN110620885B (en) * | 2019-10-18 | 2022-04-26 | 中国科学院深圳先进技术研究院 | Infrared low-light-level image fusion system and method and electronic equipment |
| CN113610695A (en) * | 2021-05-07 | 2021-11-05 | 浙江兆晟科技股份有限公司 | Infrared telescope full-frame imaging output method and system |
| CN113301233B (en) * | 2021-05-21 | 2023-02-03 | 南阳格瑞光电科技股份有限公司 | Double-spectrum imaging system |
| CN113609942B (en) * | 2021-07-27 | 2022-11-22 | 数量级(上海)信息技术有限公司 | Road intelligent monitoring system based on multi-view and multi-spectral fusion |
| CN114815217A (en) * | 2022-05-06 | 2022-07-29 | 山东北方光学电子有限公司 | Synchronous imaging control circuit for dim light and display device of multifunctional helmet glasses and using method |
| CN114757854B (en) * | 2022-06-15 | 2022-09-02 | 深圳市安星数字系统有限公司 | Night vision image quality improving method, device and equipment based on multispectral analysis |
| CN115144828B (en) * | 2022-07-05 | 2024-04-12 | 同济大学 | Automatic online calibration method for intelligent automobile multi-sensor space-time fusion |
| CN115914792A (en) * | 2022-12-22 | 2023-04-04 | 长春理工大学 | Real-time multidimensional imaging self-adaptive adjustment system and method based on deep learning |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102567979A (en) * | 2012-01-20 | 2012-07-11 | 南京航空航天大学 | Vehicle-mounted infrared night vision system and multi-source images fusing method thereof |
| CN202541437U (en) * | 2012-01-20 | 2012-11-21 | 南京航空航天大学 | Vehicle-mounted infrared night view driving device |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE10126683A1 (en) * | 2001-06-01 | 2003-02-27 | Hella Kg Hueck & Co | Night vision device for vehicles |
| US7786898B2 (en) * | 2006-05-31 | 2010-08-31 | Mobileye Technologies Ltd. | Fusion of far infrared and visible images in enhanced obstacle detection in automotive applications |
| CN201240344Y (en) * | 2008-05-14 | 2009-05-20 | 合肥工业大学 | Embedded integrated vision auxiliary driving safety system |
-
2013
- 2013-07-29 CN CN201310323416.2A patent/CN103390281B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102567979A (en) * | 2012-01-20 | 2012-07-11 | 南京航空航天大学 | Vehicle-mounted infrared night vision system and multi-source images fusing method thereof |
| CN202541437U (en) * | 2012-01-20 | 2012-11-21 | 南京航空航天大学 | Vehicle-mounted infrared night view driving device |
Non-Patent Citations (1)
| Title |
|---|
| 微光与红外视频融合系统的图像处理器研究;李露曦;《南京理工大学硕士学位论文》;20090501;第28-31页，图3.1，图3.6 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103390281A (en) | 2013-11-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103390281B (en) | A kind of two spectrum night vision device onboard systems and two spectrum Combined design method | |
| CA3157194C (en) | Systems and methods for augmentation of sensor systems and imaging systems with polarization | |
| Zhu et al. | The multivehicle stereo event camera dataset: An event camera dataset for 3D perception | |
| CN107292860B (en) | Image processing method and device | |
| CN104243959B (en) | Based on the Intelligent Composite glasses of polarization orientation and integrated positioning | |
| CN103885573B (en) | The auto-correction method of automobile-used display system and its system | |
| CN103794056B (en) | Based on the vehicle precise classification system and method for real-time two-way video stream | |
| WO2018032457A1 (en) | Systems and methods for augmented stereoscopic display | |
| CN104601953B (en) | A kind of video image fusion processing system | |
| JP5299026B2 (en) | Vehicle display device | |
| CN102982518A (en) | Fusion method of infrared image and visible light dynamic image and fusion device of infrared image and visible light dynamic image | |
| CN107850944A (en) | The method of service data glasses and the system with data glasses in the motor vehicle | |
| CN208240087U (en) | A kind of infrared and visible light emerging system and image fusion device | |
| CN107453811B (en) | A method of the unmanned plane based on photopic vision communication cooperates with SLAM | |
| US20180188004A1 (en) | Photoelectric sighting system and calibration method thereof | |
| CN109618087A (en) | A kind of infrared and low-light fusion night vision device having precision target positioning function | |
| CN116137902A (en) | Computer vision camera for infrared light detection | |
| CN113609942B (en) | Road intelligent monitoring system based on multi-view and multi-spectral fusion | |
| CN105070204A (en) | Miniature AMOLED optical display | |
| CN110412605A (en) | A kind of auxiliary aiming laser rangefinder and auxiliary aim at distance measuring method | |
| Toet et al. | The TRICLOBS dynamic multi-band image data set for the development and evaluation of image fusion methods | |
| CN104980727A (en) | Image synthesis method based on virtual reality technology and headphone display device | |
| CN106156764B (en) | Realize the optical tracking system and its control method of high speed tracking | |
| CN110572576A (en) | Method and system for shooting visible light and thermal imaging overlay image and electronic equipment | |
| CN110221403A (en) | A kind of big visual field multiband stereoscopic vision auxiliary pilot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Qin Die Inventor before: Qin Die Inventor before: Lu Qiang |
|
| COR | Change of bibliographic data | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20160413 Termination date: 20190729 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |