CN104662891A

CN104662891A - Correction of image distortion in ir imaging

Info

Publication number: CN104662891A
Application number: CN201380038189.7A
Authority: CN
Inventors: 凯特琳·斯特兰德马尔; 亨里克·约恩松
Original assignee: Flir Systems AB
Current assignee: Flir Systems AB
Priority date: 2012-07-16
Filing date: 2013-07-16
Publication date: 2015-05-27
Also published as: EP2873229A1; WO2014012946A1

Abstract

There is disclosed a method, arrangement and computer program product for correcting distortion present in an image captured using an infrared (IR) arrangement, the method for an embodiment comprising: capturing a first image using a first imaging system comprised in said IR arrangement; and correcting image distortion of the first image based on a pre-determined distortion relationship. According to embodiments, the method may further comprise capturing a second image using a second imaging system comprised in said IR arrangement, wherein: said distortion relationship represents distortion caused by said first and/or second imaging systems of said IR arrangement; and said correcting of image distortion of the first image comprises correcting image distortion with relation to the second image based on said pre-determined distortion relationship, wherein the distortion relationship represents distortion caused by said first and/or second imaging systems.

Description

The correction of image fault in infrared imaging

The cross reference of related application

Present patent application advocates that application number is 61/672,153, and the applying date is rights and interests and the priority of the U.S. Patent application on July 16th, 2012, and it comprises in this application by reference and in full at this.

The application is one section of application number is 13/437, 645, the applying date is on April 2nd, 2012, patent name is the extendible portion of the U.S. Patent application of " infrared resolution and contrast strengthen and fusion ", this U.S. Patent application 13/437, 645 are another section of application numbers is 13/105, 765, the applying date is on May 11st, 2011, patent name is the extendible portion of the U.S. Patent application of " infrared resolution and contrast strengthen and fusion ", this section of U.S. Patent application 13/105, 765 be again another section of application number is PCT/EP2011/056432, the applying date is on April 21st, 2011, patent name is the continuity of the international patent application of " infrared resolution and contrast strengthen and fusion ", patent applications all above comprises in this application by reference and all.

International Patent Application PCT/EP2011/056432 advocates that one section of application number is 61/473, the rights and interests of the U.S. Provisional Patent Application that 207, to be April 8, patent name in 2011 be the applying date " infrared resolution and contrast strengthen and fusion ", it comprises in this application by reference and all.

International Patent Application PCT/EP2011/056432 is one section of application number is 12/766, the continuity of the U.S. Patent application that 739, to be April 23, patent name in 2010 be the applying date " infrared resolution and contrast strengthen and fusion ", it comprises in this application by reference and all.

Technical field

Usually, embodiments of the invention relate to the correction using infrared facility to carry out infrared imaging.

More particularly, the different embodiments of the application relate to the correction of distortion in infrared imaging, wherein such as pass through the physics aspect of at least one imaging system or parts, be included in the image of the seizure in infrared facility and introduce distortion, wherein this at least one imaging system is used to catch image, for example, such as such as infrared view and visible images.

Background technology

Infrared ray product or infrared facility, such as infrared pick-up head, nowadays usually with require that bottom line keeps production cost to link together.

Because the cost of the optics of infrared ray or infrared facility is just becoming a part increasing in whole IR imaging device cost, optics is just becoming the field that manufacturer wants to seek more cheap solution.For example, this can by reducing the optical element that be included in optical system, the quantity of such as lens, or use cheap lens instead of costliness, higher-quality lens realize.

But nowadays many IR imaging devices with low-cost optical system also have short focus, cause introducing distortion in the infrared view caught when the use of infrared facility.More the lens of high cost can be designed to reduce distortion, but on the other hand, the price of imaging device can be higher.In addition, the traditional images treatment technology of correcting distortion does not generally solve and is present at infrared imaging and/or the problem of dtmf distortion DTMF under the background of the image processing and tracking unit of infrared view and visible light (VL) image.

Therefore, in infrared facility, the needs realizing distortion correction still can be there are, to be suitable for the particular problem occurred in such a device.In addition, still exist with a kind of needs having cost-efficient method to realize such distortion correction.

Prior art

See U.S. Patent application, invent artificial Kawasaki (Kawasaki), publication number is US2009/208136A1, an example of disclosed prior art, discloses the view data that process has distortion.

Another example of prior art is disclosed in the document " the real-time live programmable gate array (FPGA) of the antibarreling algorithm of use bilinear interpolation realizes " of people (et.al.) works such as Ge Liben (K.T Gribbon).The method is suitable for correcting the barrel-shaped distortion about described scene.

See another example of the prior art of the international monopoly WO2009/008812 of people (et.al.) such as invention artificial Lennart Strand Ma Er (Strandemar) etc., the distance disclosed based on the scene with image conversion carries out the method for the collimation of image to be mixed.

But, these publication neither ones relate to the problem of dtmf distortion DTMF be present in infrared imaging and/or under the image processing and tracking unit background of infrared view and visible images (VL), or relate to any trial using the method for similar embodiments of the invention to solve this kind of problem.

Summary of the invention

Embodiments of the invention eliminate above-described problem or at least make it to minimize.This is realized by the equipment according to additional claim, method and apparatus.

According to one or more embodiment, be disclosed for the system and method using infrared facility to correct infrared imaging.For example, for one or more embodiment, system and method can realize the distortion correction in the infrared view caught when infrared facility uses.

According to one or more embodiment of the present invention, with system and method disclosed herein for form, in order to correct the distortion be presented in the image using infrared facility seizure, this system and method is performed by using the first imaging system be included in described infrared facility to catch the first image; And the image fault of in described image first is corrected based on a default distortion relation.

According to one or more embodiment, seizure image comprises use first imaging system and catches the first image.

According to one or more embodiment, correcting image distorsion comprises the image fault corrected based on described default distortion relation in the first image relevant with the outdoor scene observed.

According to one or more embodiment, described first imaging system is infrared imaging system and described first image is the infrared view using described infrared imaging system acquires.

According to one or more embodiment, described distortion relation table is shown in the distortion in described first image caused by described first imaging system of described infrared facility.

According to one or more embodiment, catch image and comprise seizure first image.

According to one or more embodiment, use the second imaging system to catch the second image and associate the first and second images.

According to one or more embodiment, described first image using the first imaging system to catch is the infrared view using infrared imaging system acquires, and described second image using the second picture system to catch is the visible images using Visible imaging system to catch.

According to one or more embodiment, correcting image distorsion comprises and goes to correct and the image fault in second image-related the first image based on described default distortion relation.

According to one or more embodiment, described distortion relation table is shown in the distortion in described first image caused by described first imaging system, distortion in described second image caused by described second imaging system, and the distortion in described first image caused by described first imaging system and the relation between the distortion in described second image caused by described second imaging system.

According to one or more embodiment, comprise further and use the second imaging system be included in described infrared facility to catch the second image, wherein: described distortion relation represents the distortion caused by the described first and/or second imaging system of described infrared facility; And the correction of the image fault of described first image comprises and corrects the image fault image-related with second based on described default distortion relation.

According to one or more embodiment, comprise the image fault correcting second image image-related with first based on described default distortion relation further.

According to one or more embodiment, the first imaging system is infrared imaging system, and the first image is infrared view accordingly, and the second imaging system is Visible imaging system, and the second image is visible images accordingly; First imaging system be Visible imaging system accordingly the first image to be visible images and the second imaging system be infrared imaging system accordingly the second image be infrared view; First and second imaging systems are two different infrared imaging systems and the first and second images is infrared views that use first and second infrared imaging system catches respectively; Or the first and second imaging systems are two different Visible imaging systems and the first and second images is visible images that use first and second Visible imaging system catches respectively.

According to one or more embodiment, the first imaging system be infrared imaging system accordingly the first image be infrared view, and the second imaging system be Visible imaging system accordingly the second image be visible images; First imaging system be Visible imaging system accordingly the first image to be visible images and the second imaging system be infrared imaging system accordingly the second image be infrared view; First and second imaging systems are two different infrared imaging systems and the first and second images is infrared views that use first and second infrared imaging system catches respectively; Or the first and second imaging systems are two different Visible imaging systems and the first and second images is visible images that use first and second Visible imaging system catches respectively.

According to one or more embodiment, described default distortion relation represents with the form of distortion map or look-up table.

According to one or more embodiment, distortion map or look-up table have the model of distortion behavior based on one or more.

According to one or more embodiment, the correction of described distortion comprises and in x direction and y direction, the pixel coordinate of input picture is mapped to the pixel coordinate of corrected output image respectively.

According to one or more embodiment, the distortion that it is form that these mapping relations calculated depend on rotating deviation and/or shifting deviation at least partly.

According to one or more embodiment, the method comprises further and being bonded in composograph by described first and second images.

According to one or more embodiment, composograph is the version of the contrast strengthen of the infrared view that with the addition of visible images data.

According to one or more embodiment, the method comprises further and obtains composograph by aiming at infrared view and visible images, and determines that visible images resolution value is identical in fact with infrared view resolution value and synthesize infrared view and visible images.

According to one or more embodiment, synthesize described first and second images and comprise further and process visible images by extracting the high spatial frequency content of visible images.

According to one or more embodiment, synthesize described first and second images and comprise process infrared view further to reduce in infrared view and/or the noise making infrared view fuzzy.

According to one or more embodiment, synthesize described first and second images and comprise further high-resolution noise is added in composograph.

According to one or more embodiment, synthesize described first and second images and comprise further the high spatial frequency content of the visible images of seizure and the extraction of infrared view is blended in composograph.

According to one or more embodiment, the method comprises communication data further, and these data comprise via data communication interface associated images to outer member.

According to one or more embodiment, the method is included in the image display that is integrated in or is coupled in thermal imaging device showing association further.

According to one or more embodiment, the method can within hardware, such as, in a field programmable gate array (FPGA), be implemented.According to embodiment of the method, distortion correction maps can be preset and be placed in look-up table (LUT).By using the interpolation of this look-up table (LUT) and pixel value, compared to the numerical value calculated operationally, when significantly not reducing precision, the complexity of hardware designs can be reduced.

According to an embodiment, provide a kind of for catching image and infrared ray (IR) device for correcting the distortion be presented in described image, this device comprises: at least one is for catching the infrared imaging system of infrared view and/or at least one is for catching the Visible imaging system of visible images; For storing the memory of the distortion function preset of the distortion represented caused by one or more imaging system of described infrared facility; And be configured to the processing unit receiving the described default distortion relation of retrieval in the running of described infrared facility from described memory, wherein this processing unit is configured in described infrared facility running, use described default distortion relation to go the distortion of one or more image correcting described seizure further, this processor is configured to further: use the imaging system be included in described infrared facility to catch image, and based on the image fault that the distortion relation preset corrects in described image.

According to one or more embodiment, this processing unit is suitable for performing various methods that are all or part disclosed herein.

By the beneficial effect that embodiment described herein obtains, being use the optical system of infrared facility or infrared pick-up head to have lower cost to make, is because some distortions are allowed to existence.Normally, less lens element can be used to reduce production cost widely.Embodiments of the invention can also use signal-lens solution to improve Output rusults widely.According to the embodiment wherein reducing optical element quantity, by can high image quality be obtained on the contrary according to the image procossing of embodiment described herein; Or in the running of infrared facility or infrared pick-up head, or in the subsequent processes of the image using such infrared facility or infrared pick-up head to catch.Accordingly, the further outstanding effect of embodiment disclosed herein is included in imaging system, especially in infrared imaging system, the cost of optics can be reduced, maintain simultaneously or strengthen output image quality, or alternatively, when not increasing optics cost, strengthen picture quality.

Containing using different imaging systems, such as infrared imaging system and Visible imaging system, the composograph of the image of seizure, such as, merge, infrared imaging in the specific problem that occurs be that these images must be aligned to make synthesis result meet visual interpretation and measure correlation.Inventor has recognized by saving the step that performs about the distortion correction of outdoor scene or outside reference to reduce computational complexity, but the distortion correction of the image be relative to each other is performed herein according to the different embodiments presented, can distortion correction be performed in the mode of resource efficiency more, meet output quality.In other words, according to embodiment described herein, distortion correction need not further about outdoor scene or outside reference " perfect ".Therefore, compared to scheme costly on previous operation, distortion correction can have the mode of cost and resource efficiency to perform with one.

Can realize further giving prominence to effect by embodiments of the invention, be the improvement collimation realizing image to be synthesized, also play up higher-quality image in post synthesis accordingly, such as image more clearly.

In addition, because the cost of involved calculating in computing is not expensive, in the operating process of infrared facility, embodiments of the invention can perform in real time.In addition, embodiments of the invention can use limited or functional professional treatment unit execution of field programmable gate array (FPGA) or other types.

Normally, infrared view has the resolution lower than visible images, and therefore has the calculating of the pixel value distortion being compared to correction more not expensive in visible images computing.Therefore, this can be the benefit of the distortion correcting the infrared view relevant to visible images.But depend on the imaging system of use, also may be set up conversely for some embodiments.In addition, because infrared view such as, compared to, visible images, usually fuzzyyer, or the lower contrast that in other words to comprise with border and profile be form, down sample and use the value of interpolation can be used to infrared view when there is not any visible light catalytic.Just as used herein presented, embodiments of the invention, the image wherein obtained from two different imaging systems is mentioned, and also can relate to the partial correction of the first and second images about other images.

Any suitable interpolation method known in the prior art, if depended in the situation of such as such as focus in quality cost or computing cost, can be used to interpolation according to embodiments of the invention.

As described herein, the embodiment of method and apparatus solves the problem of the distortion that correction is form with the rotation caused by the respective imaging system be included in infrared facility and/or translation further.

According to embodiment, provide the computer system with the processor being suitable for the various embodiments performing all or part of method disclosed herein.

According to embodiment, providing computer readable medium, it storing the non-transitory information of the processor being suitable for the various embodiments controlled for performing all or part of method disclosed herein.

Scope of the present invention define by the claims, the part becoming this summary of the invention is for reference.By considering the detailed description of one or more embodiment hereafter, the enlightenment of the more complete understanding of embodiments of the invention and wherein additional advantage will be given to those those skilled in the art.Reference is can be used as by the additional pictures described first briefly.

Accompanying drawing explanation

Now by reference to accompanying drawing, in further detail embodiments of the invention will be described, wherein:

Fig. 1 is the schematic diagram of infrared ray (IR) device according to embodiments of the invention;

Fig. 2 a and Fig. 2 b shows the example of the image distortion correction according to embodiment;

Fig. 3 a and Fig. 3 b shows the flow chart of the aberration emendation method according to embodiment;

Fig. 4 shows the flowchart illustrative of the distortion correction according to embodiment;

Fig. 5 is the flow chart of the method according to embodiment;

Fig. 6 shows according to described in disclosed embodiment, removes the flow chart of the method obtaining composograph from infrared view and visible light (VL) image;

Fig. 7 shows according to described in disclosed embodiment, comprises the interactive display of such as touch screen and so on, image display position and enables user carry out the example of the input equipment of the control device inputted;

Fig. 8 a illustrates according to described in disclosed embodiment, when not enabling angle of visual field following function, and the angle of visual field of Visible imaging system and infrared imaging system;

Fig. 8 b describes according to described in disclosed embodiment, and when enabling angle of visual field following function, the example of treated visible images and treated infrared view, describes or represent the substantially the same subset catching view;

Fig. 9 shows according to described in disclosed embodiment, the display electronic equipment of display image data and information, and view data and packets of information are containing infrared view, visible images or the composograph associating infrared data and visible data;

Figure 10 a describes according to described in disclosed embodiment, when not having distortion correction, synthesizes the method for the first distorted image and the second distorted image;

Figure 10 b describes according to disclosed embodiment, when enabling distortion correction function, synthesizes the method for the first distorted image and the second distorted image;

Embodiments of the invention and their advantage can be understood the most fully by referring to detailed description hereafter.It is to be understood that mark identical element with identical reference numeral in the accompanying drawings.

Embodiment

Introduce

Following present the embodiment for the method for distortion correction, infrared facility, infrared pick-up head and computer readable medium.

Image fault is also referred to as distortion

When catching image by imaging system, the various mistake being form with image fault or distortion or deviation may exist, and such as, can cause the distortion of the acutance of the object in observer's outdoor scene relevant to the expression of the same object caught in the picture.Give an example, the straight line in scene can't be still straight in the picture.Various types of image fault exists but is not limited to such as barrel-shaped distortion, pincushion distortion, beard distortion or complicated image distortion, parallax error in pointing, parallax range error, Resolution Error or parallax rotation error.

Image fault also may relate to the difference that the object between the first image caught by the first imaging system and the second image caught by the second imaging system in viewed outdoor scene is represented.

Due to such a fact, how being different from the second imaging system is rotate around its second optical axis, first imaging system may rotate around its primary optic axis, rotate the introducing that distortion may exist such as angle [alpha], rotate distortion also can be called as parallax rotation error, radial distortion or rotate distortion/skew, they are replaceable use in this article.Because each imaging system has an angle of visual field (FOV), the angle of visual field is exactly the degree in the observable world that any given time can be seen, the angle of visual field of the first and second imaging systems can be different.Due to such a fact, first imaging system can be located thus its primary optic axis about the second imaging system second optical axis by translation, such as, by separated by a distance fixing, introduce displacement images distortion and be introduced into, also can be called as parallax range error, they are replaceable use in this article.Due to such a fact, the first imaging system can be located thus its primary optic axis and the second imaging system second optical axis not parallel, error in pointing image fault may be introduced into, and also can be called as parallax error in pointing, and they are replaceable use in this article.

Due to the pixel resolution value of the first imaging system, the quantity of the original paper namely in imageing sensor, and it is different in the relation between the image that the pixel resolution value of the second imaging system may catch first and second, this can cause the another kind of form of image fault, also can be called as Resolution Error.

System architecture

Fig. 1 shows the schematic diagram of infrared facility/infrared pick-up head or thermal imaging device 1, comprise one or more infrared ray (IR) imaging system 11, each has an infrared ray sensor 20, such as, many pixels pyroscan of any type, such as focal plane array, for catching infrared view data, such as Still image data and/or video data, the outdoor scene observed of presentation video.In one or more embodiment, one or more infrared ray sensor 20 of one or more infrared imaging system 11 provides and by the pictorial data representation of seizure is, such as be converted to, numerical data, such as via be included in as infrared ray sensor 20 a part or be separated the analog to digital converter (analog-to-digitalconverter) of the part as infrared facility 1 with infrared ray sensor 20.

According to embodiment, infrared facility 1 can comprise one or more visible ray/visible light (VL) imaging system 12 further, each has visible ray (VL) transducer 16, many pixels visible light detector of such as any type, for catching visible light view data, such as Still image data and/or video data, the outdoor scene observed of presentation video.In one or more embodiment, one or more visible light transducer 16 of one or more Visible imaging system 12 provides and by the pictorial data representation of seizure is, such as be converted to, numerical data, such as by be included in as infrared ray sensor 20 a part or be separated the analog to digital converter (analog-to-digital converter) of the part as infrared facility 1 with infrared ray sensor 20.

For illustrative purposes, the device comprising an infrared imaging system 11 and a Visible imaging system 12 has been shown in Fig. 1.

In one or more embodiment, infrared imaging system 1 can represent IR imaging device, such as infrared pick-up head, to catch and to process the image of outdoor scene, and such as continuous print picture frame or video frame image.

In one or more embodiment, infrared facility 1 can comprise infrared pick-up head or the infrared imaging system of any type, be configured to detection infrared radiation or provide represent data and information, for example be the infrared view data of scene or the temperature correlation infrared view data of scene, be expressed as different color-values, any other be applicable to, gray value that the expression of visually explainable image can be provided.

In a schematic embodiment, device 1 can be expressed as an infrared pick-up head, point to near-infrared, in infrared and/or far-infrared spectrum.

In one or more embodiment, device 1 or infrared pick-up head, visible ray (VL) camera or Visible imaging system can be comprised further, be suitable for detection visible radiation and provide representing data and information, for example as the visible images data of scene.In one or more embodiment, device 1 or infrared pick-up head, can comprise portable or hand-held equipment.

Hereinafter, term infrared facility can with reference to being physically separated but the system of the unit of coupling, IR imaging device or camera, wherein be integrated with all unit, or system or equipment, wherein within one device integrated the and remaining unit of unit more described below is coupled in integrated equipment or is configured to for sending data to this integrated equipment and transmit data from this integrated equipment.

According to embodiment, infrared facility 1 comprises at least one memory 15 further, or is communicatively coupled at least one external memory storage (not shown in FIG.).

In one or more embodiment, according to embodiment, infrared facility can comprise control assembly 3 and/or display 4/ display unit 4.

In one or more embodiment, control assembly 3 comprises user's input and/or interface equipment, such as rotatable knob, such as potentiometer, button, slide bar, keyboard, soft key, touch controllable function, interactive display, control lever and/or recording/push button.In one or more embodiment, control assembly 3 is suitable for generating user's input control signal.

In one or more embodiment, when responding input instruction and/or user's input control signal, processor 2 controls the function of the different piece of thermal imaging device 1.

In one or more embodiment, processing unit 2 or external processing unit, can be adapted to pass through this control assembly 3 and detect the control inputs signal from user and respond any user's input control signal detected from user received.

In one or more embodiment, processing unit 2, or external processing unit, according to the mode that one or more can be readily appreciated by one skilled in the art, can be suitable for the value explaining this user's input control signal.

In one embodiment, control assembly 3 can comprise user's Input Control Element, such as, there is the wired of user's input block or wireless handheld control unit, such as button, soft key or other suitable parts that user can be made to input, be suitable for coordinating with user and receive user and input, determine user's input control value and sending controling unit signal generates user's input control signal to control assembly with trigging control parts.

In one or more embodiment, the user's input block being included in the control unit in control assembly 3 can be used to the various functions controlling infrared facility 1, and such as auto-focusing, Menu realization and selection, the angle of visual field (FOV) following function, leveling (level), space generate the feature that (span), noise filtering, high-pass filtering, low-pass filtering, fusion (fusion), temperature measurement function, distortion correction setting, rotation correction setting and/or other those skilled in the art various understand.

In one or more embodiment, one or more user's input block can be used to provide user input values, such as adjust parameter, be such as chosen as and comprise marginal information, characteristic etc. the expectation percentage for the pixel of image stabilization algorithm (image stabilization algorithm).

In a schematic embodiment, one or more user's input block can be used to regulate in infrared view that is that caught by infrared facility 1 and/or process, low pass and/or high-pass filtering characteristic, and/or for the threshold value of edge detection.

According to an embodiment, " angle of visual field tracking " function, or the angle of visual field of infrared view of in other words mating the corresponding visible images angle of visual field are the patterns that can be unlocked and close.Open and close angle of visual field tracing mode, the setting based on thermal imaging device can be performed automatically, or is performed manually by user and one or more input equipment interaction.

In one or more embodiment, at least one first and second image, such as visible images and/or infrared view, the angle of visual field that processed thus visible images represents is the angle of visual field that represents of corresponding infrared view substantially.In one or more embodiment, process the selection that at least one first and second image comprises following computing: the cutting of the part of at least one image or at least one image; Windowing; Convergent-divergent; Displacement; And rotate.

Fig. 8 a shows an example, and for Visible imaging system 820, what kind of the view caught without angle of visual field following function is, for infrared imaging system 830, what kind of the view caught without angle of visual field following function is.Fig. 8 a also show the schematic subset 840 of the view of the complete outdoor scene of being surrounded by the infrared imaging system angle of visual field and the Visible imaging system angle of visual field captured.Fig. 8 a shows the outdoor scene 810 observed in addition.

Fig. 8 b shows an example, and when angle of visual field following function is activated, the first treated image, such as infrared view, the second treated image, such as visible images is the substantially the same subset how describing or represent the view captured

In one or more embodiment, display unit 4 comprises image display, the known video display of such as liquid crystal display (LCD), or various other types or monitor.Processing unit 2 or external processing unit, can be suitable for display image data and information on display 4.Processing unit 2 or external processing unit, can be suitable for retrieve image data and information from memory cell 15 or external storage component, and on display 4, show any view data of retrieving and information.

In one or more embodiment, display 4 can comprise display electronic equipment, its can be processed unit 2 or external processing unit utilized, with display image data and information, for example, infrared ray (IR) image, visible ray (VL) image or infrared ray (IR) view data of association and the composograph of visible ray (VL) view data, such as, with composograph, such as merge, mixing or the form of pip image.Fig. 9 shows a schematic embodiment.

In one or more embodiment, display unit 4 can be suitable for directly receiving view data and information from imaging system 11,12 via processing unit 2 or external processing unit, or can via processing unit 2 or external processing unit from memory 15 or external storage component transmit image data and information.

Figure 7 illustrates the schematic embodiment of control assembly 3, comprise user's Input Control Element with user's input block.Control assembly 3 comprises interactive display 770, such as touch screen, image display position 760 and enable user carry out the input block 710-750 inputted.

According to an embodiment, input equipment comprises controller, enables the function that user performs:

-select distortion correction target pattern 710, such as relevant to the outdoor scene observed or with about the relevant correcting distortion of the first and second images aligned with one another.

-select the distortion correction operator scheme 720 of distortion correction based on distortion relation, such as correct the distortion of the first image, correct the distortion of the second image, based on the distortion of the second image rectification first image or the distortion based on the first image rectification second image.

-activate 730 or do not activate " angle of visual field tracking " function, the coupling of the angle of visual field that the infrared view namely associated represents and the angle of visual field of the visible images associated.

-select 740 obtaining/or display image, the infrared view such as associated, the visible images of association or based on the infrared view of association and the composograph of the visible images associated.This is described in further detail in fig .9, show interactive display 970 (implementing in the mode similar with display 770) and image display position 960 (implementing in the mode similar with image display position 760), display visible images 910, infrared view 920 or depend on the synthesizing visible light/infrared view 930 of selection 740.

-store or preserve image 750 to memory (15) or retrieving images from memory.

According to different embodiments, all parts of infrared facility 1, as described herein, can be included in, be located at outward but be communicatively coupled to or be integrated in thermal imaging device/equipment, such as such as infrared pick-up head.

In one or more embodiment, the seizure of infrared view is performed by least one the infrared imaging system 11 be included in infrared facility 1.

In one or more embodiment, visible light image is also captured by least one Visible imaging system 12 be included in infrared facility 1.

In one or more embodiment, one or more image of seizure can be transmitted/be sent to the processing unit 2 being included in infrared facility 1, being configured to carries out image process computing.

In one or more embodiment, the image of seizure also directly can be transmitted or is transmitted together with intermediate, is stored to the processing unit (not shown in picture) separated with imaging device or in imaging device outside.

In one or more embodiment, processing unit 2 in thermal imaging device 1 or the processing unit separated can be provided or can comprise specially designed program code segments or code section, are suitable for controlled processing unit to perform step and the function of the embodiment of inventive method described herein.

In one or more embodiment, processing unit 2 and/or external processing unit can be processors, such as general or special processing engine, for example, microprocessor, microcontroller or other control logics, comprise code segment or code section, be suitable for controlled processing unit and/or external processing unit to perform step and the function of the embodiment of inventive method described herein, wherein said code segment or code section are stored in computer readable storage medium.

In one or more embodiment, described code segment is fixed for performing a certain task/function.In one or more embodiment, described code segment is variable, other sections of being stored in variable code segment on computer-readable storing media in use.

In one or more embodiment, described variable code segment can comprise and be used as processing unit 2 and/or external processing unit, the calibration of all infrared facilities as understood by the skilled person 1, operator scheme, distortion relation or other infrared facility functions for the sample rate of the space filtering of image or filter, distortion correction, the parameter of various task/functions.

In one or more embodiment, can be integrated in, be coupled to or be configured to transmit data to infrared facility 1 more than a processing unit or transmit data from infrared facility 1.

In one or more embodiment, processing unit 2 and/or external processing unit are configured to the infrared view data processing infrared ray (IR) transducer 20 describing the outdoor scene observed from one or more.

In one or more embodiment, processing unit 2 and/or external processing unit are configured to perform any or all method steps disclosed herein or function.

In one or more embodiment, provide and be configured to catch image and infrared ray (IR) device 1 calibrating the distortion presented in described image, wherein this device comprises: the first imaging system, and it is configured to seizure first image; Memory, 15 or 8, it is configured to store the distortion relation preset, and the function such as preset, represents the distortion caused by described first imaging system of the described infrared facility in described first image; And processing unit 2, it is configured to reception or retrieves the described default distortion relation from described memory in the operating process of described infrared facility, and wherein processing unit is configured to use described default distortion relation to correct the distortion of the first image captured described in the operating process of infrared facility further.

In one or more embodiment, described default distortion relation precalculates and is stored in memory.

In one or more embodiment, described default distortion relation is dynamically calculated by described processing unit 2 and is stored in memory 15 within the time interval of presetting.

In a typical embodiment, when the parameter be stored in memory, the measurement relevant to image is compared exceed default threshold value or prior to each distortion correction time, default distortion relation be infrared facility 1 startup stage dynamically calculated.

According to an embodiment, default distortion relation represents the distortion caused by the first imaging system of the described infrared facility in described first image.

According to an embodiment, infrared ray (IR) device 1 can comprise infrared pick-up head, and the first imaging system, one or more in memory and processing unit, and is integrated in described infrared pick-up head.

According to an embodiment, the first imaging system is infrared imaging system, and the first image is the infrared view using described infrared imaging system acquires.

According to an embodiment, infrared facility comprises the second imaging system further, is configured to seizure second image.

In one or more embodiment, described distortion relation represents the distortion caused by the described first and/or second imaging system of described infrared facility; And described processing unit is configured to the described image fault correcting the first image based on described default distortion relation by correcting the image fault image-related with second.

In one or more embodiment, described default distortion relation represents the distortion caused by the first imaging system described in described first image, by the distortion caused by the second imaging system described in described second image, the distortion caused by the first imaging system described in described first image and the relation between the distortion caused by the second imaging system described in described second image; And described processing unit 2 is configured to carry out correcting image distorsion based on described default distortion relation.

In one or more embodiment, described processing unit 2 is configured to the image fault corrected based on described default distortion relation in the first image relevant with the outdoor scene observed.In one or more embodiment, described processing unit 2 is configured to correct image fault in the first image image-related with second based on described default distortion relation.In one or more embodiment, described processing unit 2 is configured to the image fault corrected based on described default distortion relation in the second image relevant with the outdoor scene observed.In one or more embodiment, wherein said processing unit 2 is configured to the image fault corrected based on described default distortion relation in the second image image-related with first.

In one or more embodiment, the distortion caused by the first imaging system described in described first image and the described relation between the distortion caused by the second imaging system described in described second image are expressed by the difference of distortion between described first imaging system and described second imaging system.

In one or more embodiment, the distortion caused by the first imaging system described in described first image and the described relation between the distortion caused by the second imaging system described in described second image are expressed by the difference being all included in the distortion between infrared imaging system in infrared facility 1 and Visible imaging system.

In one or more embodiment, the distortion caused by the first imaging system described in described first image and the described relation between the distortion caused by the second imaging system described in described second image are expressed by the parallactic error between described first and described second imaging system in the difference for describing the angle of visual field (FOV) or the different described infrared facility of the view of outdoor scene that caught by the first imaging system and described second imaging system.

According to an embodiment, processing unit 2 or external processing unit, be further configured to the image fault correcting second image image-related with first based on the distortion relation preset.

According to an embodiment, the configurable practical hardware description language (HDL) of processing unit, external processing unit.

According to an embodiment, processing unit 2 and/or external processing unit are field programmable gate array (FPGA), an i.e. integrated circuit, it is designed to be configured by client or designer and hardware description language (HDL) can be used to configure.Accordingly, embodiments of the invention comprise configuration data, are configured to control step and the function that field programmable gate array (FPGA) performs embodiment of the method described herein.

In various embodiments, processing unit 2 and/or external processing unit comprise processor, such as one or more microprocessor, a polycaryon processor, a microcontroller, a logical device, such as be configured to the programmable logic device (PLD) performing processing capacity, Digital Signal Processing (DSP) equipment, Graphics Processing Unit (GPU), application-specific integrated circuit (ASIC) (ASIC) etc.

In one or more embodiment, the enforcement of the step of some or all aberration emendation methods or algorithm in field programmable gate array FPGA is feasible, and this is because the method or algorithm do not comprise the calculating process of the cost in complicated or computing.

In this text, term " computer program " and " computer-readable storing media " generally can be used to refer to media, the storing media of such as memory 15/8 or processing unit 2 or outside storing media.

The computer readable storage medium of these and other forms can be used to be supplied to the instruction of processing unit 2 for carrying out.This instruction, be generally known as " computer program code " (its can computer program or other grouping form be grouped), when being implemented, make infrared facility 1 can perform feature or the function of the embodiment of current techniques.Further, as used herein, " logic " can comprise hardware, software, firmware or their synthesis.

In one or more embodiment, processing unit 2 and/or external processing unit are communicatively coupled to and communicate to connect memory 15 and/or external memory storage 8, wherein, parameter prepares processed unit 2 at any time and/or external processing unit uses, if and user expects, the image that processed unit 2 is processing can be occurred wherein by storage.

In one or more embodiment, one or more memory 15 can comprise selects hard disk drive, floppy disk, tape drive, CD drive, CD or DVD driver (recordable (R) or can rewritable format (RW)), or other moveable or fixing media drives.

According to embodiment, processing unit 2 and/or external processing unit can be suitable for interface and the miscellaneous part of the infrared facility 1 that communicates to perform treatment step described herein and/or running.

In one or more embodiment, processing unit 2 and/or external processing unit can comprise distortion correction module (not shown), be suitable for implementing aberration emendation method or algorithm, it is such that such as aberration emendation method or algorithm such as consult that Fig. 2 to Fig. 4 discusses.

In one or more embodiment, processing unit 2 and/or external processing unit can be suitable for performing other image procossing runnings various, comprise translation/displacement, or can by the contrast of translation and/or the image of rotation, the rotation of image and image or other data sets, or as aberration emendation method embodiment a part or separate with it.

In one or more embodiment, distortion correction module can be integrated in the part as processing unit 2 and/or external processing unit in software and/or hardware, its with code, such as software, firmware or configuration data, for such as memory 15 and/or outside and the distortion correction module stored in accessible memory.

In one or more embodiment, aberration emendation method as disclosed herein, by the computer readable medium separated, such as memory, such as hard disk driver, laser disc, digital vidio disc or flash memory, store, in order to pass through computer, such as logic system or the system based on processor, carry out and perform various method disclosed herein and computing.

In one or more embodiment, computer readable medium can be portable and/or installs with device 1 is independent, by the aberration emendation method of storage, algorithm, mapping or look-up table (LUT), from computer scale medium, aberration emendation method, algorithm, mapping or look-up table (LUT) is downloaded to device 1 and/or by device 1 by coupled computers readable medium, such as by wired connection and/or wireless connections, be supplied to device 1.

In one or more embodiment, memory 15 or external memory unit comprise one or more memory device, are suitable for storage data and information, comprise infrared data and information.In one or more embodiment, memory 15 or external memory unit can comprise one or more various types of memory device, comprise volatibility with non-volatile memory device, such as RAM (random access memory), ROM (read-only memory), EEPROM (electrically erasable read-only memory), flash memory etc.

In one or more embodiment, processing unit 2 or external processing unit can be suitable for carrying out the software in storage generation memory 15 or external memory unit, thus perform method described herein and process step described herein and/or computing.

In various embodiments, the parts of infrared facility 1 can be synthesized and/or whether be implemented, and are as required or depend on purposes and demand, and this device 1 represents the various functional blocks of related system.In a typical embodiment, processing unit 2 can combine with memory unit 15, imaging system 11,12 and/or display 4.

In another typical embodiment, processing unit 2 combines with one of imaging system 11,12, this imaging system 11,12 is by the circuit in one of described imaging system 11,12, such as processor, microprocessor, logical device, microcontroller etc., only perform certain function of processing unit 2.

In one or more embodiment, the various parts of infrared facility 11 away from each other, such as one or more imaging system 11,12 can comprise the remote sensor with processing unit 2 or external processing unit etc., represents the computer that can or can not communicate to connect with one or more imaging system 11,12.

Embodiment of the method

In Figure 5, for correcting distortion, also can be called as image fault, embodiment of the method, represent as shown in the flowchart use infrared ray (IR) device catch image in, the method comprises:

In step 510: use the first imaging system be included in described infrared facility to catch the first image; And

In step 520: the image fault correcting the first image based on the image fault relation preset.

In one or more embodiment, the image fault relation preset is expressed with the form of distortion map or look-up table (LUT).

In one or more embodiment, distortion map or look-up table are for distortion behavior type (distortion behavior type) based on one or more, such as barrel-shaped distortion, pincushion distortion or beard/babble itself are the technology that can be readily appreciated by one skilled in the art.

In one or more embodiment, exemplary type of distortion represents radial distortion, such as, cause straight line to become different types of non-directional distortion.

According to an embodiment, the correction of distortion is included in the pixel coordinate that the pixel coordinate of input picture is mapped to corrected output image by x direction and y direction respectively.

According to an embodiment, the image fault relation preset is image fault relation as calculated, it depends on the image fault rotated and/or shifting deviation is form at least partly, characterize input picture 300 pixel coordinate and correct after output image pixel coordinate between one-one relationship.

According to an embodiment, the first imaging system is infrared imaging system and the first image is the infrared view using described infrared imaging system acquires.

According to an embodiment, described image fault relation represents the image fault in described first image caused by described first imaging system of described infrared facility.

According to embodiment, step 510 can comprise use the second imaging system be included in described infrared facility further and catch the second image, wherein: described image fault relation represents the image fault caused by the described first and/or second imaging system of described infrared facility; And the image fault of described correction first image comprises and corrects the image fault image-related with second based on described default image fault relation.

According to an embodiment, the method comprises the image fault correcting second image image-related with first based on described default image fault relation further.

In an optional embodiment, for the method representation of correcting image distorsion in the image using infrared ray (IR) device table, the method comprises:

In step 510: use the imaging system be included in described infrared facility to catch image; And

In step 520: the image fault correcting described image based on image fault relation.

In one or more embodiment, catch image in step 510 and comprise use first imaging system and catch the first image.

In one or more embodiment, described first image wherein using the first imaging system to catch is the infrared view that captures and described first imaging system is infrared imaging system.

In one or more embodiment, described first image using the first imaging system to catch is visible ray (VL) image that captures and described first imaging system is Visible imaging system.

In one or more embodiment, seizure image comprises use second imaging system further and catches the second image.

In one or more embodiment, described second image wherein using the second imaging system to catch is the infrared view that captures and described second imaging system is infrared imaging system.

In one or more embodiment, described second image using the second imaging system to catch is visible ray (VL) image that captures and described second imaging system is Visible imaging system.

In one or more embodiment, catch image and associate the first and second images further.

In an example, association is obtained by the common data structure of described first and second images by generating wherein to store.

In the example of an indefiniteness, using the imaging system be included in described infrared facility to catch image is one of following situation: use the first imaging system to catch infrared view; The first imaging system is used to catch visible images; Use the first imaging system to catch infrared view and use the second imaging system to catch visible images; Use the first imaging system to catch visible images and use the second imaging system to catch infrared view; Use the first imaging system to catch infrared view and use the second imaging system to catch infrared view; Or use the first imaging system to catch visible images and use the second imaging system to catch visible images.

Correcting image distorsion

In one or more embodiment, correcting image distorsion comprises the image fault correcting first image relevant with the outdoor scene observed based on described default image fault relation.

In the example of an indefiniteness, described default image fault relation can obtain when the design of described infrared ray (IR) device 1 or when producing.This is by the image acquisition of the object in the outdoor scene observed described in catching, the flat surface such as configured together with grid pattern, analyze relevant with the described outdoor scene observed described in image fault in the image that captures determine that required corrected value is to correct, to minimize or reduce the image fault of first image relevant with the outdoor scene observed and store described corrected value as the image fault relation preset.Described default image fault relation may be determined distance limited between described infrared facility 1 and the described outdoor scene observed.

In one or more embodiment, correcting image distorsion comprises and goes to correct and the image fault in second image-related the first image based on described default image fault relation.

In the example of an indefiniteness, described default image fault relation can obtain when the design of described infrared ray (IR) device 1 or when producing.This is by the first and second image acquisition of the object in the outdoor scene observed described in catching, the flat surface such as configured together with grid pattern, analyze with described capture second image-related described in image fault in the first image of capturing determine that required corrected value is to correct, to minimize or reduce the image fault of first image image-related with second and store described corrected value as default image fault relation.Described default image fault relation may be determined distance limited between described infrared facility 1 and the described outdoor scene observed.

In one or more embodiment, correct to comprise and go to correct the image fault in the second image relevant with the outdoor scene observed based on described default image fault relation.

In one or more embodiment, correct to comprise and go to correct the image fault in the second image image-related with first based on described default image fault relation.

In one or more embodiment, described image fault relation comprises: by the image fault caused by the first imaging system described in described first image; By the image fault caused by the second imaging system described in described second image; And the image fault caused by the first imaging system described in described first image and the relation between the image fault caused by the second imaging system described in described second image.

In one or more embodiment, the image fault caused by the first imaging system described in described first image and the described relation between the image fault caused by the second imaging system described in described second image comprise the different image fault between described first imaging system and described second imaging system.

In one or more embodiment, the image fault caused by the first imaging system described in described first image and the described relation between the image fault caused by the second imaging system described in described second image comprise image fault different between infrared imaging system and visible ray (VL) imaging system.

In one or more embodiment, image fault caused by the first imaging system described in described first image and the described relation between the image fault caused by the second imaging system described in described second image comprise further: the parallax range error between described first and described second imaging system of described infrared facility, wherein parallax range error describes displacement images distortion, depends on the translation of the optical axis of first imaging system relevant with described second imaging system optical axis; Parallax error in pointing, wherein error in pointing image fault describes the deviation that the orientation coming from described primary optic axis is parallel to the second imaging system second optical axis; Parallax rotation error, radial distortion or rotation distortion/deviation, wherein parallax rotation error describes and rotates the image rotating distortion of relevant described first imaging system around its optical axis with described second imaging system around its optical axis; And pixel resolution value error, the distortion of pixel resolution value error image describes depends on pixel resolution value, i.e. the quantity of element and the pixel resolution value of the second imaging system in the imageing sensor of the first imaging system, pixel resolution image fault.

In one or more embodiment, first and second images are caught in step 510, it is composograph that the method can comprise described first and second image combinings further, for example by perform capture the fusion of image, mixing or picture-in-picture computing.

According to another embodiment, the first imaging system is infrared imaging system, and therefore the first image is infrared view, and the second imaging system is Visible imaging system, and therefore the second image is visible images.

According to another embodiment, the first imaging system is Visible imaging system, and therefore the first image is visible images, and the second imaging system is infrared imaging system, and therefore the second image is infrared view.

According to another embodiment, the first and second imaging systems are two different infrared imaging systems, and the first and second images are the infrared views using the first and second infrared imaging system acquires respectively.

According to another embodiment, the first and second imaging systems are two different Visible imaging systems, and the first and second images are the visible images using the first and second Visible imaging systems to catch respectively.

In one or more embodiment, the image fault relation preset represents and is all included in image fault different between infrared imaging system in infrared facility 1 and Visible imaging system.According to this embodiment, use described image fault relation to go the image fault of one or more image captured described in correction, following selections can be comprised:

The infrared view that the use infrared imaging system acquires relevant with the visible images using Visible imaging system to capture arrive is corrected based on the image fault relation preset;

Based on the image fault relation preset correct with use infrared imaging system acquires to the visible images that captures of the relevant use Visible imaging system of infrared view; Or

Based on the image fault relation preset all process use infrared imaging system acquires to infrared view and the visible images that uses Visible imaging system to capture, thus treated image is the image fault of mutual correction.

In fig. 2 a, the image 200 of the distortion after distortion correction and corresponding image 210 is shown.The image 200 of distortion shows a type of image fault, is called as barrel-shaped distortion, one in multiple types of the image fault that those skilled in the art know.Several examples of other type of distortion are pincushion distortion and beard or babble.

According to embodiment, the distortion correction of image 200 becomes image 210 can pass through to use default performing in real time with the distortion relation mapped or look-up table (LUT) is form.According to this execution mode, the dissimilar of distortion behavior can when without any reprogramming or introduce new algorithm or code and be corrected to when the processing unit of field programmable gate array (FPGA) or other general execution distortion corrections.

According to other embodiments, be that image 210 is by using with the function preset by the distortion correction of image 200, such as transfer function, equation, algorithm or other parameter Sum fanction of distortion relation is described, for form, the distortion relation preset that can be readily appreciated by one skilled in the art performs in real time.

In one or more embodiment, preset distortion relation by calculate determine, wherein calculate comprise obtain default function value and store results to memory 15.

According to embodiment, by be used to catch image imaging system caused by distortion relevant preset distortion relation, just may be determined when producing calibration, maintenance calibration, using the input of control assembly 3 described in conjunction with Figure 1 by user, or use the self-calibration function of infrared facility or infrared pick-up head just to be determined.

In Fig. 3 a and Fig. 3 b, show the flow chart of the aberration emendation method according to embodiment.In Fig. 3 a and Fig. 3 b, the image 300 of distortion is in the step 310 processed into the image 330 of correction.

According to an embodiment, processing unit 2 or external processing unit are communicatively coupled in infrared facility 1, are suitable for processing image 300 according to the embodiment of the method for Fig. 3 a.

The embodiment described in Fig. 3 b, wherein needs the distortion correction function performing distortion correction according to embodiment to be implemented in step 340.

In one or more embodiment, step 340 is performed once usually, for example when the product calibration being used as the infrared facility of distortion correction, infrared pick-up head or other processing units or upgrading.

After this, according to embodiment, based on the distortion relation taking the distortion map of the one-one relationship between the pixel coordinate showing input picture 300 and the output image 330 of correction or look-up table (LUT) 360 as form, distortion correction is performed by the mapping of pixel coordinate.

In one or more embodiment, among the external memory storage that the processing unit that distortion map or look-up table (LUT) 360 memory 15 can be occurred, perform the memory of the processing unit of distortion correction or pass through to perform distortion correction by storage obtains.

Comprise distortion map or look-up table (LUT), 360 embodiment demand memory space, but comprise low computing cost, for the processing unit performing virtual borderlines in the step 310.Therefore, the mapping of step 310 can under imaging system or the frame rate of system catching image, and for example the frame rate of 30 hertz, is performed according to embodiment.

Reduce computation complexity

According to a schematic embodiment, distortion map or look-up table (LUT) can represent the distortion map of the down sample version being used as the image caught, and for example associate 32 × 24 pixel images to substitute 320 × 240 pixel images.According to this schematic embodiment, be accessed in the sub-fraction time needed for situation that memory can only map wherein or look-up table (LUT) represents the pixel of the image of all seizure storing distortion map or look-up table (LUT), therefore eliminate a large amount of operands.Fig. 3 b depicts an embodiment, wherein by 350 and 370 mapping of down samples illustrated or look-up table (LUT) embodiments, wherein 350 describe to map before the mapping of down sample or look-up table (LUT) and step 370 are included in the process that step 310 performs or look-up table (LUT) to up-sampling.Such as when the defect that distortion to be corrected is low spatial frequency, such as example rotation defect, time, the mapping of down sample or look-up table (LUT) can be convenient to use.

According to embodiment, all aberration emendation methods can comprise the interpolation of pixel value, and at least one subset of pixel in image, this image waits the image being " positioned " to obtain correction.The interpolation of any suitable type known in the art can be used, such as arest neighbors interpolation, linear interpolation, bilinear interpolation, spline interpolation or polynomial interopolation.According to an embodiment, weighted interpolation can be used.

In one or more embodiment, processing procedure 310 comprises and processes image 300 based on distortion relation that is known or that preset, is for example determined in the production or calibration process of infrared facility.Known distortion relation can such as be expressed with function, transfer function, equation, algorithm, mapping, look-up table (LUT) or another form organizing parameter Sum fanction.

According to an embodiment, one or more parameter and/or rule 320 are used to the process of step 310.According to embodiment, due to the mechanical tolerance of special infrared device, this one or more parameter and/or rule can be the system parameters decided at the higher level but not officially announced (defaultparameters) determined in the design process of infrared facility, or producing calibration, self calibration, use calibration or safeguarding that calibration associates the independent propagation of distortion, for example associate distortion, rotate the center of distortion and/or translation distortion, process in the parameter determined and/or rule.

The distortion relation that this one or more parameter 320 can be preset according to embodiment association, it can such as be expressed with function, transfer function, equation, algorithm, mapping, look-up table (LUT) or another form organizing parameter Sum fanction.

According to an embodiment, parameter and/or rule 320 are occurred, in memory 15 or external memory unit, to may have access to the processing unit of carries out image process in the step 310 by storage.According to this embodiment, the processing unit of carries out image process is in the step 310 configured to receive or retrieval come from addressable memory cell, be stored in one or more parameter described wherein and/or rule.According to embodiment, the process of step 310, for example, can be performed in programmable gate array (FPGA) or general processing unit at the scene.

According to an embodiment, the process 310 described in Fig. 3 a is performed for each frame in sequence of image frames.As being understood easily by those skilled in the art, depend on various situation, the execution of such as infrared facility or operational capability and/or can be used for the bandwidth of image transmitting, process is performed so not frequent.According to embodiment, wherein the process of step 310 be performed for all frames selection subsets or be in other words the down sample collection of sequence of the picture frame caught, interpolation is used to the picture frame generating middle distortion correction.According to another embodiment, if the imaging system for catching image has low frame rate, the interpolation of the image of distortion correction can be used.

According to other embodiments, the process of step 310 is performed the subset for pixels all in picture frame, or is in other words the picture frame of down sample.According to these embodiments, picture element interpolation is used.

Normally, infrared view there is the resolution lower than visible images and the computing of the pixel value of therefore distortion correction than for the cost had in lower computing of visible images.Therefore, it can be conducive to the distortion correction infrared view relevant with visible images.But depend on the imaging system of use, for some embodiments, vice versa.And, due to infrared view usually than visible ray head portrait more " fuzzy ", or in other words comprise, for example, with the lower contrast that border and profile are form, the down sample of the value of interpolation and use can be used for depositing infrared view in case without any visible light catalytic by use.

According to embodiments of the invention, such as, depend on and focus on quality or computing cost, any suitable interpolation method known in the art can be used for interpolation by use.

In one or more embodiment, image to be corrected can be infrared view or visible light image.

In one or more embodiment, distortion correction can refer to and correct infrared view, visible light image or two kinds of earlier figures pictures, with the outdoor scene observed that correspondence catches in the picture, or referring to correction treats by the image of " perfect ", or similar to outside reference with degree large as far as possible, the outdoor scene such as described or reference picture.

In one or more embodiment, distortion correction can refer to and correct infrared view with the visible light image similarity with the identical outdoor scene of description, correct light image with similar to infrared view, or correct/process infrared view and visible light image, thus make them similar each other.Correspondingly, the distortion relation preset can describe by the first imaging system, for example can be infrared imaging system or visible light imaging system, caused distortion, by the second imaging system, for example can be infrared imaging system or visible light imaging system, caused distortion, or the distortion relation between the first imaging system and the second imaging system.

Containing the different imaging system of use, for example infrared imaging system and Visible imaging system, the synthesis of the image of seizure, for example merge, infrared imaging in the particular problem that occurs, be that image must collimate to synthesize result, to meet visual interpretation and to measure correlation.

With the advantage of the correction of second image-related the first image

Inventor recognizes, by omitting the step performing the distortion correction relevant with the scene of image conversion or outside reference, but the distortion correction of the image of cross-correlation is performed herein according to the different embodiment presented, reduce computational complexity, distortion correction can be performed in the mode of a resource efficiency more, to meet output quality accordingly.

Figure 10 a shows the schematic embodiment of the method for the image of image and second distortion of synthesizing the first distortion when not having distortion correction.In this special embodiment, the composograph 1040 of contrast strengthen is generate from visible images 1010 and infrared view 1020 1030.Seen in Figure 10 a, the border of object is not well aimed at.

Figure 10 b shows the schematic embodiment of the method for the image of image and second distortion of synthesizing the first distortion when there being distortion calibration function.In this special embodiment, the composograph 1070 of contrast strengthen is generate from visible images 1050 and infrared view 1060 1030.Seen in Figure 10 b, the aligning of the profile of object uses the contrast strengthened to play up image more clearly and is enhanced.In a schematic embodiment, composograph is the version of the contrast strengthen of the infrared view that with the addition of visible images data, and it is synthesized after distortion correction, obtains the composograph of the contrast strengthen of improvement whereby.

Therefore, according to embodiment, distortion correction do not need as much as possible about outdoor scene or other outside references correct.Purport is after calibration on the contrary, from the image of corresponding imaging system geometric representation can as much as possible each other similar or image can aim at as far as possible well.

In order to make two images similar and/or aim at better each other

In one or more embodiment, one of image has been added into distortion instead of has reduced it in other image, for example, in the application of the inventive concept in computing when the solution of more efficiency.

Accordingly, for example, the fusion of the enforcement of the field programmable gate array (FPGA) of distortion correction and/or the image of enforcement image distortion correction and distortion correction is feasible.The further outstanding effect realized by embodiments of the invention is that the collimation of the improvement of image to be synthesized can be implemented, and can also play up image more clearly in post synthesis accordingly.

The embodiment of the method presented herein can be used to aim at the image using different imaging systems to catch, for example, when image by merge or other synthetic methods in time being synthesized, because embodiment of the method provides the image after distortion correction, and accordingly better about aiming at each other and/or about the scene described.

In one or more embodiment, the object of reference that distortion relation between first imaging system and the second imaging system can show the intermediate releases of the distortion caused by the first imaging system and the second imaging system is form, and the image using two imaging systems to catch is waited to be mated or adapt to.Certainly, the image of any type using different imaging systems to catch, such as use the visible images that different infrared rays is liked the infrared view that your imaging system catches or used different Visible imaging systems to catch, use embodiment of the method described herein, about the scene described and/or can be corrected each other.

In a schematic embodiment, the infrared view using infrared imaging transducer to catch can be corrected to mate the visible images using visual light imaging transducer to catch, the visible images using visual light imaging transducer to catch can be corrected the infrared view mating the seizure of use infrared imaging transducer, or the visible images of the infrared view using infrared imaging transducer to catch and the seizure of use visual light imaging transducer can partly be corrected to match each other.Accordingly, the image of the distortion correction of mating as much as possible is each other acquired.

Projecting apparatus collimates

According to embodiment, the infrared view using infrared imaging system or Visible imaging system to catch respectively or visible images, can about the image of the imaging system projection by taking projecting apparatus as form by distortion correction, for example visible light projector project visible light is to treating in scene that is observed and/or that describe.As in above-described embodiment, the image of seizure can be corrected about this projected image, and projected image can be corrected about the image caught, or two kinds of images can about being partly corrected each other.According to all these embodiments, the image of seizure can aim at the projection of projector better.

According to different embodiments, its objective is similar between scene or outside reference realizing image and image conversion, or similar between two images.If similar between image is object, this may mean, but might not mean, image seems correct compared to the outdoor scene observed depicted.The more important thing is, for example, if image is synthesized, for example, if they are merged, mixed or use picture-in-picture techniques be synthesized, by providing infrared view about distortion correction each other and visible images, being synthesized and if the image of distortion correction aims at image well, good visual effect can be realized.User can analyze and illustrate in composograph what shows accordingly, and even composograph still has more or less distortion compared to the outdoor scene described.In addition, due to be suitable for when image coupling outdoor scene or other, " perfect ", outside reference time, distortion correction must be so accurate, therefore computing spend cheap distortion correction can be implemented.This means, for example, according to embodiments of the invention, the field programmable gate array (FPGA) of the fusion of the distortion correction of distortion correction and/or realtime graphic and the image of distortion correction is feasible.

In order to meet by processing applied performance constraint in real time, according to embodiment, algorithm can be implemented within hardware, for example, at the scene in programmable gate array (FPGA).But according to embodiment, any embodiment that processing unit 2 or external processing unit can present according to the device of composition graphs 1 is used to perform any or all method steps described herein or function.

Rotate and/or translation distortion (inspection error)

According to embodiment, the processing unit for distortion correction is configured to compensate with the distortion rotated and/or translation is form.In one or more embodiment, two different imaging systems are wherein used to catch and about distortion correction two images each other, such as the first and second images, for the difference in parallax rotation error, parallax range error and parallax error in pointing that the rotation that compensates and translation distortion can describe between two imaging systems.

According to an embodiment, for parallax rotation error two imaging systems between parallactic error compensate, poor corresponding to the curl around the optical axis of each imaging system, and/or translation, due to parallax range error and parallax error in pointing, be represented as displacement in the x and y direction, can be used in distortion relation by compensation, the parameter Sum fanction of the distortion relation default with the distortion map preset, look-up table (LUT), function, transfer function, algorithm or other formation such as added is the distortion relation preset of form.

Rotate and/or translation is synthesis for wherein image, such as merge, mix or picture-in-picture, the embodiment that can be performed needs the important factor taken in.Because rotation error and/or translation error are constant in the length of life of imaging device, these errors can be determined between the production of infrared facility 1 or alignment epoch.According to an embodiment, rotation and/or translation distortion use control assembly 3 described in conjunction with Figure 1 to be transfused to by user.

In figure 2b, the image 220 of the distortion after distortion correction and corresponding image 230 is shown.The image 220 of distortion shows an image, and the rotation distortion of angle [alpha] has been introduced into this image.According to the embodiment for illustrating in figure, this image can be changed into or additionally by translation distortion by distortion.If Fig. 2 b is by a dotted line illustrated by boundary line, according to an embodiment, the part that the image after distortion correction can comprise clipping correction and convergent-divergent cut out is with the size of matching area 240 and/or resolution.This region 240 may correspond to the part selected by display unit or display unit, it shows the image 230 after correcting.In order to zoomed image is to meet different resolution, the compensation of any suitable species known in the art all can be used.

According to embodiment, the distortion correction of image 220 becomes image 230 to perform in real time by the use of default distortion map or look-up table (LUT).According to this execution mode, dissimilar distortion behavior can when without any reprogramming or new algorithm or code be introduced into the processing unit of field programmable gate array (FPGA) or other general execution distortion corrections be corrected.

According to other embodiments, the distortion correction of image 220 becomes image 230 to perform in real time by default function, transfer function, equation, algorithm or other uses describing the parameter Sum fanction group of distortion relation.

According to embodiment, the distortion preset caused by the imaging system for catching image, may be determined during the production calibration being used by user control assembly 3 described in conjunction with Figure 1 to input, maintenance calibration, or use the self-calibration function of infrared facility or infrared pick-up head to be determined.

According to an embodiment, rotation and/or translation compensation are explained in the distortion relation preset.Accordingly, the rotation of synthesis, translation and distortion correction can operationally between be implemented based on the distortion relation preset.

Owing to being apparent for those skilled in the art, the image fault of any kind caused by the imaging system in order to catch image, causes the displacement of pixel in the image caught, and the embodiment presented can be used herein to be corrected.

According to an embodiment, the method that one's own department or unit presents can be used to change the angle of visual field of image further, for example illustrated by Fig. 2 b, plays up a less angle of visual field.By this modified angle of visual field of convergent-divergent, amplification effect can be obtained or reduce effect.In addition, according to embodiment, wherein use two images that different imaging systems catches, fusion by way of example, mixing or picture-in-picture are waited to be synthesized, and the angle of visual field of one or two images was adjustable to obtain better images match or collimation before the synthesis of image.

With the image of the synthesis of contrast strengthen

According to embodiment, the image of the association that user is accessed for showing further comprises the image of the synthesis that can demonstrate the image depending on association.

According to an embodiment, the image of synthesis is the version of the contrast strengthen of the infrared view that with the addition of visible images data.

According to an embodiment, the method for the image obtaining synthesis comprises the step of aligning, determines that visible images resolution value is substantially equal with infrared view resolution value and synthesizes infrared view and visible images.The flow chart of the method with reference to disclosed embodiment has been shown in Fig. 6.

Catch

In one or more embodiment, the thermal imaging device being form with infrared pick-up head or imaging device are equipped with visible light imaging system to catch visible images, infrared imaging system is used for catching infrared view, processor is suitable for processing the infrared view and the visible images of seizure that catch, thus as the image synthesized on they can be displayed on thermal imaging device together display.The advantage of this synthesis is to identify the changeability used from temperature in the object of the infrared data of infrared view, meanwhile shows and uses imaging device with the location and identification that simplify object in result images for user from the enough data in visible images.

In the region of image procossing, describe the infrared view including the outdoor scene of one or more object, be enhanced by the image information combined from the visible images describing described outdoor scene, described synthesis is called as fusion.

In one embodiment, infrared view that describe outdoor scene, that comprise one or more object is by synthesizing itself and the image information from the visible images describing described outdoor scene thus strengthen contrast.This inventive concept is described below.

Aim at

Because the seizure of infrared ray (IR) image and visible light (VL) image are generally performed by the different imaging system of the imaging device installed in mode that is biased, directed and that rotate, optical axis is different.Optical axis between imaging system can at a distance from each other and the optical phenomena being called as parallax range error just there will be.Optical axis between imaging system can about being directed angularly in relation to one another, and the optical phenomena being called as parallax error in pointing just there will be.Imaging system rotates around the optical axis of their correspondences, and the optical phenomena being called as parallax rotation error just there will be.Due to the view of the seizure of these parallactic error outdoor scenes, be referred to as between the angle of visual field (FOV) infrared imaging system and Visible imaging system and can have difference.

Because the seizure of infrared ray (IR) image and the seizure of visible light (VL) image are generally passed through with different characteristics, such as magnification ratio, the different imaging system of imaging device of different optical systems be performed, the view of the seizure of outdoor scene, is referred to as the angle of visual field (FOV) and can has difference between imaging system.Infrared view and visible images can by with different optical characteristics, such as magnification ratios, different optical systems obtain, result is different with the size of the angle of visual field that visible light sensor catches (FOV) by infrared ray sensor.

In order to the visible images of the infrared ray and seizure that synthesize seizure, these images must be applicable thus the infrared view be applicable to obtaining the same section representing the outdoor scene observed and the visible images be applicable to, in other words, different parallactic errors and angle of visual field size is compensated.This treatment step is called as registration or the collimation of infrared view and visible images.Registration or collimation can according to being performed by the suitable technology that those skilled in the art understands.

Determine that visible images resolution value and infrared view resolution value are substantially the same

In one embodiment, infrared view and visible images can obtain by different resolution, i.e. the varying number of the sensor element of imaging system.In order to make pixel comparison operation feasible on infrared ray and visible images, they need to be resampled to a general resolution.Resampling can be performed in technical staff's any method always according to ability.

In one embodiment, infrared view is resampled to first resolution, visible images is resampled to second resolution, the wherein multiple of first resolution to be the multiple of 2 times of second resolution or second resolution be 2 times of first resolution, make accordingly by consider each infrared view or visible images 2 ⁿthe instant resampling of individual pixel is feasible.

Synthesis infrared view and visible images

In one or more embodiment, synthesize infrared view and visible images, to produce the composograph of contrast strengthen by being combined with the high spatial frequency content of the visible images aimed at by the infrared view aimed at.By this synthesis of Overlapped Execution of the high spatial frequency content of infrared view and visible images, or alternatively by overlapping for the infrared view high spatial frequency content to visible images.Result is exactly, and the contrast of visible images can be inserted into and illustrate in the infrared view of temperature variability, accordingly when losing definition and the interpretation of result composograph, in conjunction with the advantage of two image types.

According to an embodiment, the method for the composograph obtaining contrast strengthen comprises the steps:

Step 610: catch visible images

In a schematic embodiment, catch visible images and comprise the visible images using the Visible imaging system being furnished with optical system and sensor element to catch the outdoor scene that description is observed, the visible images wherein caught comprises the visible ray pixel of the visual representation of the visible light image of seizure.Catch visible images to be performed according to any method well known by persons skilled in the art.

Step 620: catch infrared view

In a schematic embodiment, catch infrared view and comprise the infrared view using the infrared imaging system acquires being furnished with optical system and sensor element to describe the outdoor scene observed, the infrared view wherein caught comprises the infrared ray pixel of the association of the visual representation of the temperature value of the infrared data value of the seizure of the infrared radiation sent from the outdoor scene observed and the infrared data value of expression seizure.Catch infrared view to be performed according to any method well known by persons skilled in the art.

In a schematic embodiment, step 610 and 620 performs simultaneously or performs successively.In a schematic embodiment, image can or be captured within the time difference small as far as possible simultaneously, can reduce because catch the risk of the operating collimation difference of the imaging device unit of visible light and infrared view due to this.It should be apparent to those skilled in the art that the image of seizure can be used in the time instance separated further.

In a schematic embodiment, the sensor element of infrared imaging system and the sensor element of visible images system are substantially the same, such as, have substantially the same resolution.

In a schematic embodiment, infrared view can be captured with the IR imaging device of ultra-low resolution, the such as low resolution to 64 × 64 or 32 × 32 pixels, but as readily understood by the skilled person, other resolution much can be applied equally.Inventor has been found that if edge and border (high spatial frequency) information are added in composograph from visible images, the use of ultra-low resolution infrared view still can play up composograph, and wherein user can the clearly object of resolved depiction and temperature or other infrared ray information relevant with them.Catch infrared view to be performed in any method known to the skilled according to ability.

Step 630: aim at infrared view and visible images

In a schematic embodiment, parallactic error comprise the layout of the transducer due to the imaging system for catching described infrared view and described visible images difference and between optical axis the general parallax range error occurred, the parallax error in pointing angle of causing between these optical axises due to the mechanical tolerance of their complete parallel installations generally can be hindered, and due to the parallax rotation error of the mechanical tolerance that them generally can be hindered to install with identical curl with the optical axis of visible images system around infrared ray completely.

In a schematic embodiment, by being furnished with, there is heterogeneity, such as magnification ratio, the different imaging system of imaging device of different optical systems be performed the seizure of infrared ray (IR) image and the seizure of visible light (VL) image, outdoor scene catches the scope at visual angle, and the size being referred to as the angle of visual field (FOV) may be different.

According to any method known to those skilled in the art, by the size of compensation for parallax disparity error and the angle of visual field (FOV), can perform and aim at infrared view, to obtain, there is the infrared view of the aligning of the substantially the same angle of visual field and the visible images of aligning.

Step 690: determine the resolution value of infrared imaging system and the resolution value of Visible imaging system, the resolution of the visible images that the resolution of infrared view of the wherein resolution value correspondence seizure of infrared imaging system and the resolution value correspondence of Visible imaging system catch.

In a schematic embodiment, resolution value represents the quantity of the series of pixel of the image of seizure and the quantity of a row pixel.In a schematic embodiment, the resolution of imaging system is determined.

According to any method well known by persons skilled in the art, can perform and determine the resolution value of infrared imaging system and the resolution value of Visible imaging system, the resolution of the visible images that the resolution of infrared view of the wherein resolution value correspondence seizure of infrared imaging system and the resolution value correspondence of Visible imaging system catch.

Step 600: determine that visible images resolution value is substantially the same with infrared view resolution value

If in step 600, determine that visible images resolution value and infrared view resolution value are not substantially the same, this method can relate to optional step 640 further, resampling at least one receive image thus the result visible images resolution value obtained after resampling is substantially the same with result infrared view resolution value.

In a schematic embodiment, resampling comprises resolution to up-sampling infrared view to the resolution of visible images determined in step 690.In a schematic embodiment, resampling comprises resolution to up-sampling visible images to the resolution of infrared view determined in step 690.In a schematic embodiment, resampling comprises the resolution of resolution to the visible images determined in step 690 of down sample infrared view.In a schematic embodiment, resampling comprises the resolution of resolution to the infrared view determined in step 690 of down sample visible images.

In a schematic embodiment, resampling comprises the resolution of the infrared view that resampling is determined in step 690 with the resolution of visible images to the mid-resolution different from the visible images resolution of the infrared view resolution caught and seizure.

In a schematic embodiment, mid-resolution is determined based on the resolution of the display unit of thermal imaging device or imaging device.

According to a schematic embodiment, a position of infrared view and the corresponding position of visible images perform this method step.According to an embodiment, the corresponding position of visible images is the position describing the outdoor scene part observed identical with this position of infrared view.In this embodiment, high spatial frequency content extracts from the position of visible ray, and the position of infrared view is combined with the high spatial frequency content at the position of the visible images of extraction, to generate composograph, in the position of wherein infrared view, contrast and/or resolution improve compared to the resolution of the initial infrared view caught.

According to different embodiments, the described position of infrared view can be the sub-position of whole infrared view or whole infrared view, and the corresponding position of described visible images can be the sub-position at whole visible images or whole visible ray position.That is, according to an embodiment, if corresponding infrared ray and visible light imaging system have the different angles of visual field, these positions are corresponding positions of whole infrared view and visible images, can be the subdivisions of whole visible images or visible images.

According to ability in known to the skilled any method, determine that visible images resolution value is substantially the same with infrared view resolution value and be performed.

Step 650: by extracting the high spatial frequency content process visible images of visible images

According to a schematic embodiment, performed the high spatial frequency content extracting visible images by usage space filter high-pass filtering visible images.

According to a schematic embodiment, the high spatial frequency content extracting visible images is performed by the difference (being commonly referred to as different images) extracted between two images describing same scene, wherein the first image is captured on very first time example, second image is captured in the second time instance, and the second time instance is preferably in time near very first time example.In sequence of image frames, these two images can be two continuous print picture frames usually.Unless the scene of image conversion is completely constant from very first time example to the second time instance, and imaging sensor keeps completely static, represents that in scene, the edge of object and the high spatial frequency content on border can manifest in different images.For example, due to the change of the motion of the object of the light in the scene of image conversion or description, this scene may change to next frame from this frame.And in almost each situation, imaging device and thermal imaging system all can not keep static completely.

In order to the high spatial frequency content in abstract image, perform high-pass filtering, in other words, contrast district, location, namely shows a large amount of difference, such as burr, the value of neighborhood pixels.Through the result images of high-pass filtering, by calculating pixel by pixel, the image that to deduct from original image through low-pass filtering realizes.

By extracting the high spatial frequency content process visible images of visible images, can be performed according to any method well known by persons skilled in the art.

Step 660: process infrared view is to reduce the noise of in infrared view and/or fuzzy infrared view.Step 660 is optional steps.

According to a schematic embodiment, process infrared view comprises making for performing the noise reducing in line image and/or fuzzy infrared view by low pass spatial filtering.Low-pass filtering, by placing cored slab (spatial core) in each pixel of image, is that described pixel calculates a new value by using the coefficient of the value in contiguous pixel and described cored slab.

The object performing low-pass filtering in optional step 660 is that it is uneven to eliminate from the noise presented in the protoplasm infrared view caught in step 620 in infrared view.Being removed in filtering process due to the burr in original infrared view and visible noise or being at least weakened, improving visibility in result images further by reducing the ancipital risk exposed in composograph that the filtering of infrared view and infrared view and visible images do not aim at.

Process infrared view can be performed according to any method well known by persons skilled in the art with the noise reducing in infrared view and/or fuzzy infrared view.

Step 670: the high spatial frequency content of the extraction of the visible images of seizure and the infrared view that processed alternatively are bonded to composograph.

In a schematic embodiment, the high spatial frequency content of the extraction of the visible images of seizure and the infrared view that processes alternatively are bonded to composograph and comprise the luminance component Y only used from the visible images processed.

In a schematic embodiment, the high spatial frequency content of the extraction of the visible images of seizure and the infrared view that processes alternatively are bonded to composograph and comprise the luminance component of the luminance component of the high spatial frequency content of the extraction of the visible images of seizure with the infrared view to process alternatively is combined.As a result, the color of infrared view or gray scale inconvenience and the character of original infrared ray colour table remain stationary, meanwhile add the contrast of expectation in composograph.Liking your information due to radiancy or other relevant infrared rays can all the time in the process of composograph be kept in explaining, is useful, can facilitates user accordingly by all stages of processing and show to maintain infrared ray colour table.

In a schematic embodiment, the high spatial frequency content of the extraction of the visible images of seizure and the infrared view that processes alternatively are bonded to composograph comprise, when adding luminance component, be combined with the luminance component of infrared view by the luminance component of visible images, usage factor Alpha (alpha) is with the balance between the luminance component determining two images.This factor Alpha is determined by imaging device or imaging system self, use suitable parameter for determining that from visible images required border level is to create satisfied image, but also provide by user and input to imaging device or imaging system decides.This factor also can be changed by follow-up phase, such as when image is stored up in generation systems or be conditioned to adapt to any requirement from user in personal computer (PC) or in analog.

In a schematic embodiment, the high spatial frequency content of the extraction of the visible images of seizure and the infrared view that processes alternatively being bonded to composograph uses colour table to go maps colors to the temperature value of infrared view, such as according to the color space of YCbCr race, wherein Y-component (i.e. colour table luminance component) can be selected as the constant on whole colour table.In an example, Y-component can select 0.5 times of the high-high brightness of composograph, visible images or infrared view.Result is, when being combined with visible images by infrared view according to selected colour table, the Y-component of the visible images processed can be added to the infrared view 305 that processed and produce desired contrast when the color of the infrared view processed can not change.Maintain in the processing procedure of original infrared view accordingly the specific nuance of color meaning.

When calculating color component, following equation can be used to determine the Y-component of composograph, Cr component and Cb component, and composograph has from processing, such as through high-pass filtering, the Y-component of visible images and Cr and the Cb component from infrared view

hp_y_vis＝highpass(y_vis)

(y_ir,cr_ir,cb_ir)＝colored(lowpass(ir_signal_linear))

Can write by another kind of representation:

hp _yvis＝highpass(y _vis)

(y _ir,cr _ir,cb _ir)＝colored(lowpass(ir _{signal linear}))

Other are not the color spaces of YCbCr, certainly can be used in existing disclosed embodiment yet.Use different color spaces, such as example RGB, YCbCr, HSV, CIE1931XYZ or CIELab, the conversion between color space well known to the skilled person.Such as when using rgb color model, brightness can be calculated by as all color components, and calculate brightness in color space ground one by one by transfer equation, for determining that one of brightness new expression formula will be determined each color space.

Step 680: add high-resolution noise to composograph

According to a schematic embodiment, high-resolution noise is high-resolution noise in time domain.High-resolution noise can be added into composograph in order to more clearly rendering result image is to observer, and in order to reduce stain or the similar marking, it can be presented due to the noise in original infrared view, and it is retained and in the optional low-pass filtering process of described infrared view.

According to an embodiment, processor 2 is configured to manner of execution step 610-680.A user interface can be provided here, make user can be with, such as, on display 4, the data interaction demonstrated.This user interface can comprise the possibility of selectable option or input, allows user to switch between different views, amplify region-of-interest etc.In order to mutual with display, user can use one or more input equipment of control assembly 3 to provide input.

According to an embodiment, before image storage or display, user can in a manner known in the art, mutual with the convergent-divergent of one of carries out image or scaling with thermal imaging device 1.If user performs convergent-divergent or scaling action on infrared ray or visible images, the angle of visual field of associated images can be conditioned according to the various embodiments (namely in act 630) with reference to the said method described by Fig. 6.Thus, the angle of visual field of associated images is often with in real time or close to on-the-spot or mate with user being stored in the view data for retrieving after a while in real time.

Distortion correction mapping or look-up table

According to an embodiment, default distortion relation describes the distortion map of distortion caused by different imaging systems, and it can be preset and operationally for distortion correction.

According to other embodiment of the method, distortion relation value is default and is placed in look-up table (LUT).By using the interpolation of look-up table and pixel value, compared to the calculated value when computing, the complexity of hardware designs is lowered when not having huge precision to lose.

In distortion relationship description imaging system, compared to describe scene or compared to external reference distortion or two imaging system distortions relative to each other, distortion.According to embodiment, distortion relation can be calibrated in the production of infrared facility, safeguard calibration or self-alignment process in determined, wherein be integrated with imaging system or problem system, or described system be communicatively coupled or be configured to transmit image data to this infrared facility and/or from this infrared facility transmit image data.According to an embodiment, distortion relation uses control assembly 3 be transfused to or change by user.

Distortion relation can be used to and correction relevant between scene, outside reference or the different imaging systems that use in the running of infrared facility.As mentioned above, the image correcting and caught by an imaging system can be related to according to embodiment distortion correction, it is compared with the image caught by another imaging system, similar each other to make, or with correct/process from one or more imaging system image with to describe scene or outside reference similar.

According to an embodiment, distortion relation can be occurred, in memory 15 or in another internal storage or external memory storage, to may have access to the processing unit 2 of infrared facility 1 in running by storage, or accesses external processing unit when subsequent treatment.

Virtual borderlines

In the diagram, the embodiment that use is the distortion correction of proper mapping or look-up table is described.

In the diagram, provide distortion map or look-up table (LUT) 400, the mapping of each pixel (x, y) in the image wherein caught, be for example represented as displacement (Δ x, Δ y).According to embodiment, the precision of shift value, such as, in units of decimal, can be different in different applications, it depends on the balance of quality requirement and computing cost.

Shift value (Δ x, Δ y) obtains from distortion relation, depends on the optics in the imaging system comprised or imaging system.As referred to herein, determined in the process that distortion relation can be calibrated in the production calibration of one or more imaging device comprised, self calibration or maintenance.

According to an embodiment, perform the processing unit of distortion correction at the shift value running or calculate in subsequent processes each pixel, thus generate distortion map in real time or in running.In other words, distortion map is environmentally calculated for each pixel or the subdivision for pixel.According to an embodiment, the processing unit performing distortion correction is field programmable gate array (FPGA).In running, the calculation requirement of distortion map value or shift value calculates frequently, thus has larger operation blocks in field programmable gate array embodiment, but on the other hand, the quantity of required memory access has been lowered.The one side kept firmly in mind is if the equation for displacement calculating value or distortion map value changes, field programmable gate array execution mode requires each field programmable gate array of reprogrammed, and contrary predefine distortion map or look-up table (LUT) embodiment only require the self adaptation of product code.

According to any embodiment presented herein, the operand required by distortion correction and the amount of distortion grow proportionately.For example, if the situation less compared to shift value, if shift value is very large, and long-range pixel is had to, " being obtained ", for distortion correction, performs the processing unit of distortion correction has the memory always can accessing it more pixels by having to.

Shift value (Δ x, Δ y) is used to the pixel value of image 410 distortion correcting the seizure representing detector pixel, alternatively by interpolation procedure 420, to the picture frame 430 of distortion correction.

Shift value, has optional interpolation in one or more decimal instead of integer and/or step 420, can be used to reduce the introducing of pseudomorphism in the image 430 corrected.

According to embodiment, all aberration emendation methods can comprise the interpolation 420 of pixel value, and at least one subset of pixel, it is treated " being substituted " to obtain the image of correction.Any suitable type of interpolation known in the art can be used, such as arest neighbors interpolation, linear interpolation, bilinear interpolation, spline interpolation or polynomial interopolation.According to an embodiment, weighted interpolation can be used.

Distortion correction calculates in real time

According to embodiment, the distortion correction of forward part perform by using to calculate in real time instead of map.According to these embodiments, function, transfer function, algorithm or other parameter Sum fanction groups are described between imaging system or between one or more imaging system and scene or another outside reference, for example produce or calibration process in by the distortion relation determined.Accordingly, for each image caught or image pair of distortion to be corrected, the calculating of distortion relation can be performed in real time.Compared to the embodiment wherein using distortion map or look-up table (LUT), the memory capacity that real-time operation method demand is less, but perform distortion correction processing unit in the more logic of demand.According to embodiment, processing unit can be the processing unit of any type, device in its composition graphs 1 is described, for example be a kind of processor being integrated in, being connected to or being located at the general type of infrared facility 1 outward, or a kind of processor of special configuration, such as field programmable gate array (FPGA).

Map as also described above, distortion correction function or function can be generated in the design of infrared facility 1, production or calibration.According to embodiment, be there is memory 15 by storage in distortion map or look-up table (LUT), be integrated in field programmable gate array (FPGA) memory in infrared facility or another is connected to or may have access in the memory cell of the processing unit performing distortion correction.In running, the calculating of distortion correction parameter and being performed by processing unit 2 or the external processing unit that is coupled to infrared facility 1 communicatedly according to the correction of calculated value.According to an embodiment, processing unit 2 is field programmable gate array (FPGA), and the calculating of distortion correction parameter is also the correction performed by field programmable gate array (FPGA) according to calculated value.

The use of application program and use case

The embodiment of the method presented herein can be used for merging collimation; Owing to using the image of different imaging system seizure to be by distortion correction about each other and/or about the scene described.Accordingly, image can be similar each other to a great extent.In other words, by providing infrared view and the visible light image of relative to each other distortion correction, if image is synthesized, if for example they are merged or mix, good visual effect can be realized.User can analyze and explain in composograph what shows accordingly, and even composograph still has more or less distortion compared to the outdoor scene described.In addition, computing spend low distortion correction be implemented.Accordingly, for example, the fusion of field programmable gate array (FPGA) execution mode of distortion correction and/or the image of realtime graphic distortion correction and distortion correction is feasible.According to embodiment, for example, therefore operating personnel can be used in the distortion correction function in hand-held infrared pick-up head, comprise the processing logic of field programmable gate array (FPGA) logic or other any suitable type, and according to the frame rate of infrared pick-up head, obtain real-time update, distortion correction and merge or mixing image.

The embodiment of the method presented herein can be used to length or area measure on-site support further.According to a schematic embodiment, the operating personnel of infrared imaging system can use infrared imaging system to go investigate building surface to identify the region that there be moisture damage danger according to presented embodiment above.If operating personnel find such region on the surface of investigation, namely operating personnel can see on the display of IR imaging device, region that the mode known with operating personnel marks, that represent wet area, operating personnel may want to find out this region to be had much.Therefore, operating personnel use be included in infrared imaging system, the physical length of scene of computed image or the measurement function of actual area, consider and the angle of visual field of compensating distortion for example by calculating, based on the distance obtained and/or the angle of visual field, convergent-divergent length or region to the size of display, and show length and/or territory element over the display.Operating personnel can see that in fact the region of identification has much accordingly.This information, i.e. image, also can be stored for later retrieval and analysis.Because IR imaging device comprises distortion correction, if length and/or area information certainly can be more correct than the situation not performing overcorrect.If do not perform distortion correction, image central authorities are subject to the region of barrel-shaped distortion, for example, compared to the length cell shown in image and/or territory element, originally will than it, in fact size be larger with presenting, image outer peripheral areas originally will be with presenting in fact less than it simultaneously.

By using the distortion correction embodiment presented herein, the measurement in length or region in the scene of combining image, the calculating of such as power/performance information and visual, for example with every square metre of watt (w/m2) for form, can to present in conjunction with the image of display or the image of integrating capture is stored.

Further embodiment

According to an embodiment, any or all method steps described herein or function can be performed in the subsequent treatment of the view data stored, for example use personal computer (PC) or other suitable processing units, processing unit can utilize default distortion relation to provide data.

According to an embodiment, provide a kind of computer system, it has processing unit, and it is configured to any step or the function that perform any or all embodiments of the method disclosed herein.

According to an embodiment, provide a kind of computer readable medium, it stores non-transitory information, it is configured to controlled processing unit to perform any step or the function of any or all embodiments of the method disclosed herein.

According to an embodiment, provide a kind of computer program, comprise code section, it is configured to control processor to perform any step or the function of any or all embodiments of the method disclosed herein.

According to an embodiment, provide computer program, comprise configuration data, be suitable for configuring field programmable gate array (FPGA) to perform any step or the function of any or all embodiments of the method disclosed herein.

According to an embodiment, provide a kind of computer program, comprise code section, its Ei is configured to controlled processing unit to perform any step or the function of any or all embodiments of the method disclosed herein.

According to an embodiment, provide a kind of computer program, comprise configuration data, be suitable for configuring field programmable gate array (FPGA) to perform any step or the function of any or all embodiments of the method disclosed herein.

Further advantage

The outstanding effect obtained by embodiment described herein uses the optical system of infrared facility or infrared pick-up head can to make with lower cost, is owing to allowing to there are some distortions.Normally, use less lens element greatly to reduce production cost.Even signal-lens scheme is also possible.According to the embodiment of wherein optical element quantity reduction, no matter in the operating process of infrared facility or infrared pick-up head, or in the subsequent treatment of the image using this infrared facility or infrared pick-up head to catch, by can high image quality be obtained on the contrary according to the image procossing of embodiment described herein.Accordingly, the outstanding further effect of embodiment disclosed herein is, comprise in imaging systems, especially in infrared imaging system, the cost of optics can be lowered, maintain simultaneously or enhance output image quality, or strengthening picture quality when not increasing optics cost alternatively.

Although in conjunction with only there being the embodiment of minority to describe the present invention in detail, should being understood that easily and the invention is not restricted to embodiment disclosed in these.On the contrary, the present invention can modify in conjunction with any amount of distortion, remodeling, substitute or the equivalent arrangements that do not describe up to now, but these amendments match with the spirit and scope of the invention.In addition, although described various embodiment of the present invention, should be understood that, many-side of the present invention only can comprise some execution mode described.Correspondingly, the present invention should not be considered described above limit, but only to be limited by the scope of appended claim.

Claims

1., for correcting a method for the distortion presented in the image using infrared facility to catch, the method comprises:

The first imaging system be included in described infrared facility is used to catch the first image; And

The image fault of this first image is corrected based on the distortion relation preset.

2. the method for claim 1, wherein this first imaging system is infrared imaging system, and this first image uses described infrared imaging system acquires to arrive.

3. the method for claim 1, wherein described distortion relation represents the distortion caused by described first imaging system of described infrared facility.

4. the method for claim 1, comprises further and uses the second imaging system be included in described infrared facility to catch the second image, wherein,

Described distortion relation represents the distortion caused by the described first and/or second imaging system of described infrared facility; And

The correction of the image fault of described first image comprises and correcting and this second image-related image fault based on described default distortion relation.

5. method as claimed in claim 4, comprises further and corrects the image fault with this first image-related this second image based on described default distortion relation.

6. method as claimed in claim 4, wherein:

This first imaging system is infrared imaging system, and therefore this first image is infrared view, and this second imaging system is Visible imaging system, and therefore this second image is visible images;

This first imaging system is Visible imaging system, and therefore this first image is visible images, and this second imaging system is infrared imaging system, and therefore this second image is infrared view;

This first and second imaging system is two different infrared imaging systems, and this first and second image is the infrared view using this first and second infrared imagings system acquires to arrive respectively; Or

This first and second imaging system is two different Visible imaging systems, and this first and second image is the visible images using this first and second Visible imaging system to capture respectively.

7. method as claimed in claim 5, wherein:

This first imaging system is infrared imaging system, and therefore this first image is infrared view, and this second imaging system is Visible imaging system, and therefore this second image is visible images.

8. method as claimed in claim 5, wherein:

This first imaging system is Visible imaging system, and therefore this first image is visible images, and this second imaging system is infrared imaging system, and therefore this second image is infrared view.

9. method as claimed in claim 5, wherein:

This first and this second imaging system be two different infrared imaging systems, and this first and second image is the infrared view using this first and second infrared imagings system acquires to arrive respectively.

10. method as claimed in claim 5, wherein:

This first and this second imaging system be two different Visible imaging systems, and this first and second image is the visible images using this first and second Visible imaging system to capture respectively.

11. methods as claimed in claim 4, wherein said default distortion relation represents with the form of distortion map or look-up table.

12. methods as claimed in claim 11, wherein this distortion map or look-up table are the models having distorted characteristic based on one or more.

13. methods as claimed in claim 11, the correction of wherein said distortion comprises the mapping of the coordinate respectively on x direction and y direction.

14. methods as claimed in claim 4, the distortion that wherein it is form that these mapping relations calculated depend on rotating deviation and/or shifting deviation at least partly.

15. methods as claimed in claim 4, comprise further and described first and second images are combined into composograph.

16. 1 kinds of infrared facilities, it is configured to catch image and corrects the distortion presented in described image, and this device comprises:

Be configured to the first imaging system catching image;

Be configured to the memory of the default distortion function of storage one, this distortion function represents the distortion caused by described first imaging system of described infrared facility; And

Be configured to the processing unit receiving or retrieve described default distortion relation when described infrared facility runs from described memory, wherein this processing unit is configured to use described default distortion relation to correct the distortion of the image of described seizure when this infrared facility runs further.

17. infrared facilities as claimed in claim 13, comprise infrared pick-up head further, and by this first imaging system, and one or more in memory and processing unit is integrated in described infrared pick-up head.

18. infrared facilities as claimed in claim 13, wherein this first imaging system is infrared imaging system, and this first image is the infrared view using described infrared imaging system acquires.

19. infrared facilities as claimed in claim 13, wherein said distortion relation represents the distortion caused by described first imaging system of described infrared facility.

20. infrared facilities as claimed in claim 13, comprise the second imaging system being configured to seizure second image further, wherein:

Described processing unit is configured to based on described default distortion relation by correcting the image fault correcting described first image with this second image-related image fault.

21. infrared facilities as claimed in claim 17, wherein said processing unit is configured to correct the image fault with this first image-related the second image based on described default distortion relation further.

22. infrared facilities as claimed in claim 13, the wherein configurable use hardware description language (HDL) of this processing unit.

23. infrared facilities as claimed in claim 18, wherein this processing unit is field programmable gate array (FPGA).

24. infrared facilities as claimed in claim 13, this processing unit is configured to perform the method for claim 1 further.

25. 1 kinds of computer systems, it has the processor being suitable for performing the method for claim 1.

26. 1 kinds of non-transitory computer readable mediums, it store non-transitory information, are suitable for control processor to perform the method for claim 1.