CN205157056U

CN205157056U - System for be used for keeping watch on driving system

Info

Publication number: CN205157056U
Application number: CN201390001108.1U
Authority: CN
Inventors: J·D·弗兰克; S·杰普森; M·纳斯迈耶; E·A·库尔特; N·霍根斯特恩; T·R·赫尔特; K·斯特兰德玛; P·布朗热; B·夏普; D·杜姆珀特; A·C·泰希
Original assignee: Flir Systems Inc
Current assignee: Teledyne Flir LLC
Priority date: 2012-12-21
Filing date: 2013-12-19
Publication date: 2016-04-13
Anticipated expiration: 2023-12-19
Also published as: WO2014100507A2; WO2014100507A3

Abstract

A technique that is used for using little form factor infrared imaging module to keep watch on the system in the aspect of the driving system is disclosed. System for be used for keeping watch on driving system can include the module of one or more infrared imaging module, treater, memory, display, communication module and control driving system's part. The mountable of infrared imaging module is installed in the driving system who has the one or more driving system part on the driving system who has the one or more driving system part, or combines together with the driving system who has the one or more driving system part with other modes. The infrared imaging module can dispose the heat picture for the part of catching driving system. Can carry out various hot image analysis and performance analysis to the heat picture of catching to confirm the operating condition and the temperature of driving system's part. Can produce the monitor message based on state and the temperature of confirming, then present to driving system's user.

Description

For monitoring the system of power system

The cross reference of related application

This application claims the application number proposed on Dec 21st, 2012 is 61/745,373, be entitled as the rights and interests of the U.S. Provisional Patent Application of " SYSTEMSANDMETHODSFORMONITORINGPOWERSYSTEMS ", it can be used as entirety to be herein incorporated by way of reference.

The application is the application number proposed on Dec 9th, 2013 is 14/101,245, be entitled as the continuation-in-part application of the U.S. Patent application of " LOWPOWERANDSMALLFORMFACTORINFRAREDIMAGING ", it can be used as entirety to be herein incorporated by way of reference.

The application is the application number proposed on Dec 6th, 2013 is 14/099,818, be entitled as the continuation-in-part application of the U.S. Patent application of " NON-UNIFORMITYCORRECTIONTECHNIQUESFORINFRAREDIMAGINGDEVI CES ", it can be used as entirety to be herein incorporated by way of reference.

The application is the application number proposed on Dec 9th, 2013 is 14/101,258, is entitled as the continuation-in-part application of the U.S. Patent application of " INFRAREDCAMERASYSTEMARCHITECTURES ", it can be used as entirety to be herein incorporated by way of reference.

This application claims the application number proposed on Dec 31st, 2012 is 61/748,018, is entitled as the rights and interests of the U.S. Provisional Patent Application of " COMPACTMULTI-SPECTRUMIMAGINGWITHFUSION ", it can be used as entirety to be herein incorporated by way of reference.

This application claims the application number proposed on March 15th, 2013 is 61/792,582, is entitled as the rights and interests of the U.S. Provisional Patent Application of " TIMESPACEDINFRAREDIMAGEENHANCEMENT ", it can be used as entirety to be herein incorporated by way of reference.

This application claims the application number proposed on March 15th, 2013 is 61/793,952, is entitled as the rights and interests of the U.S. Provisional Patent Application of " INFRAREDIMAGINGENHANCEMENTWITHFUSION ", it can be used as entirety to be herein incorporated by way of reference.

This application claims the application number proposed on Dec 26th, 2012 is 61/746,069, is entitled as the rights and interests of the U.S. Provisional Patent Application of " TIMESPACEDINFRAREDIMAGEENHANCEMENT ", it can be used as entirety to be herein incorporated by way of reference.

This application claims the application number proposed on Dec 26th, 2012 is 61/746,074, is entitled as the rights and interests of the U.S. Provisional Patent Application of " INFRAREDIMAGINGENHANCEMENTWITHFUSION ", it can be used as entirety to be herein incorporated by way of reference.

Technical field

One or more embodiment of the present utility model is usually directed to thermal imaging device, particularly relates to and such as uses thermal imaging to carry out monitoring and controlling power system.

Background technology

Nearly all modern technologies all depend on electric powertrain and to unify mechanical dynamic system or directly supported by it.Electric propulsion system generates, store and distribute the electricity of a lot of variety classes electron devices of driving use every day.Mechanical dynamic system to all utilizing their mechanical dominance from the everything being transported to lawn-care, and they form the basis of nearly all type building.

For monitoring the conventional sensors of the operation of such power system, because its active sensor surface of needs only contacts with monitored object intimate physical, thus install difficulty often with costliness.Partly owing to forcing their close to monitored object, so such conventional sensors is also often more fragile and easily wear and tear and tear.In addition, conventional thermal imaging sensor provides from single-point (such as usually, point sensor) and/or sensing data that is coarse or time delay, to such an extent as to hamper the cost efficient to modern power system, accurate and/or real-time supervision.

Utility model content

Disclose for using little form factor infrared imaging module to monitor the technology of the system of power system aspect.In one embodiment, surveillance can comprise the module of the parts of one or more infrared imaging module, processor, storer, display, communication module and control power system.Infrared imaging module can be arranged close to the power system with one or more power system components, be arranged on there is one or more power system components power system on, be arranged in the power system with one or more power system components, or otherwise combine with the power system with one or more power system components.Infrared imaging module can be configured to the heat picture of the part of catching power system.Various heat picture analysis and distribution plan can be performed cut open and paint the heat picture of catching, to determine duty and the temperature of the part of power system.Monitor message can being produced based on the state detected and temperature, then presenting to user and/or for controlling power system.

In another embodiment, for monitoring that the system of power system comprises the infrared imaging module being at least partially arranged as the parts observing power system, wherein this infrared imaging module comprises the focal plane arrays (FPA) (FPA) of the heat picture of the part being configured to capturing means; And processor, it is to infrared imaging module communication and be configured to process heat picture to produce the monitor message relevant with power system.

Scope of the present utility model is defined by the claims, and this part is herein incorporated by way of reference.By detailed description to one or more embodiment below considering, the realization of more complete understanding to the utility model embodiment and wherein additional advantage will be provided to those skilled in the art.Below with reference to the accompanying drawing that first can briefly describe.

Accompanying drawing explanation

Fig. 1 shows the infrared imaging module being configured to realize in the host device according to disclosure embodiment.

Fig. 2 shows the infrared imaging module after according to the assembling of disclosure embodiment.

Fig. 3 show according to disclosure embodiment and be placed in the exploded view of the infrared imaging module on socket.

Fig. 4 shows the block diagram comprising the infrared sensor package of infrared array sensor according to disclosure embodiment.

Fig. 5 shows the process flow diagram of the various operations of determination nonuniformity correction (NUC) item according to disclosure embodiment.

Difference between Fig. 6 shows according to the neighbor of disclosure embodiment.

Fig. 7 shows the flat field correction technology according to disclosure embodiment.

Fig. 8 shows according to the various image processing techniques of Fig. 5 of disclosure embodiment and other operations of being applied in image processing pipeline.

Fig. 9 shows the noise in time domain reduction step according to disclosure embodiment.

Figure 10 shows the concrete implementation detail of several steps of the image processing pipeline of the Fig. 8 according to disclosure embodiment.

Figure 11 shows the fixed pattern noise (FPN) according to the space correlation in the neighborhood of pixels of disclosure embodiment.

Figure 12 shows the block diagram comprising another implementation of the infrared sensor package of infrared array sensor and low-dropout regulator according to disclosure embodiment.

Figure 13 shows the circuit diagram of a part for the infrared sensor package of the Figure 12 according to disclosure embodiment.

Figure 14 shows the block diagram of the system of the parts for monitoring power system according to disclosure embodiment.

Figure 15 shows the block diagram of the various parts of the power system according to disclosure embodiment.

Figure 16 shows the configuration of the system of the parts for monitoring power system according to disclosure embodiment.

Figure 17 shows the vehicles panel board comprising the display of surveillance according to disclosure embodiment.

Figure 18 shows the process of the parts of the supervision power system according to disclosure embodiment.

By reference to detailed description below, will better understand embodiment of the present utility model and advantage thereof.Should be understood that, identical reference number is for representing the similar elements shown in a pair or several accompanying drawings.

Embodiment

Fig. 1 shows the infrared imaging module 100 (such as, thermal camera or infreared imaging device) being configured to realize in host apparatus 102 according to disclosure embodiment.For one or more embodiment, according to Wafer level packaging or other encapsulation technologies, the infrared imaging module 100 of little form factor can be realized.

In one embodiment, infrared imaging module 100 can be configured to realize in small-sized portable host apparatus 102, such as, mobile phone, tablet computing device, laptop devices, personal digital assistant, visible light camera, music player or any other suitable mobile device.With regard to this respect, infrared imaging module 100 can be used for providing infrared imaging function to host apparatus 102.Such as, infrared imaging module 100 can be configured to catch, process and/or manage infrared image, and this infrared image is supplied to host apparatus 102, host apparatus 102 can use this infrared image (such as, be further processed this infrared image, be stored in storer, show, used by the various application programs operated in host apparatus 102, output to other devices or other application) in any desired way.

In various embodiments, infrared imaging module 100 can be configured to work in low voltage level and wide temperature range.Such as, in one embodiment, infrared imaging module 100 can use the power work of about 2.4 volts, 2.5 volts, 2.8 volts or lower voltage, and can work (such as, providing suitable dynamic range and performance in the ambient temperature range of about 80 DEG C) in the temperature range of about-20 DEG C to about+60 DEG C.In one embodiment, by making infrared imaging module 100 work under low voltage level, compared with the infreared imaging device of other types, the heat that infrared imaging module 100 self produces is less.Therefore, infrared imaging module 100 operationally, can utilize the measure of simplification to compensate this heat self produced.

As shown in Figure 1, host apparatus 102 can comprise socket 104, shutter 105, motion sensor 194, processor 195, storer 196, display 197 and/or miscellaneous part 198.Socket 104 can be configured to reception infrared imaging module 100 as shown by an arrow 101.With regard to this respect, Fig. 2 shows the infrared imaging module 100 be assemblied in socket 104 according to disclosure embodiment.

Other suitable devices of motion of host apparatus 102 can be detected to realize motion sensor 194 by one or more accelerometer, gyroscope or can be used for.Processing module 160 or processor 195 can monitor motion sensor 194 and motion sensor 194 provides information, to detect motion to processing module 160 or processor 195.In various embodiments, motion sensor 194 can be embodied as a part (as shown in Figure 1) for host apparatus 102, the part of other devices that also can be embodied as infrared imaging module 100 or be connected to host apparatus 102 or contact with host apparatus 102.

Processor 195 can be embodied as any suitable treating apparatus (such as, logical unit, microcontroller, processor, special IC (ASIC) or other devices), host apparatus 102 can use above-mentioned treating apparatus to perform suitable instruction, such as, the software instruction in storer 196 is stored in.Display 197 can be used for display capture and/or process after infrared image and/or other images, data and information.Miscellaneous part 198 can be used for any function realizing host apparatus 102, as the various application (such as, clock, temperature sensor, visible light camera or miscellaneous part) that may expect.In addition, machine readable media 193 can be used for storing non-transitory instruction, can will be performed by processor 195 in this non-transitory instruction load to storer 196.

In various embodiments, can produce infrared imaging module 100 and socket 104 in a large number, to promote their widespread use, such as, it can be applicable in mobile phone or other devices (such as, needing the device of little form factor).In one embodiment, when infrared image-forming module 100 is installed in socket 104, the overall dimensions gone out shown by the combination of infrared imaging module 100 and socket 104 is approximately 8.5mm × 8.5mm × 5.9mm.

Fig. 3 show according to disclosure embodiment and be placed in the exploded view of the infrared imaging module 100 on socket 104.Infrared imaging module 100 can comprise lens barrel 110, shell 120, infrared sensor package 128, circuit board 170, pedestal 150 and processing module 160.

Lens barrel 110 can be at least part of loading optical element 180 (such as, lens), the hole 112 in scioptics lens barrel 110, described optical element 180 in figure 3 part visible.Lens barrel 110 can comprise roughly cylindrical prolongation 114, and it can be used for lens barrel 110 is contacted with the hole 122 in shell 120.

Such as, infrared sensor package 128 can be realized by the cap 130 (such as, lid) be arranged on substrate 140.Infrared sensor package 128 can comprise by row or other modes be arranged on the multiple infrared sensors 132 (such as, infrared eye) covered on substrate 140 and by cap 130.Such as, in one embodiment, infrared sensor package 128 can be embodied as focal plane arrays (FPA) (FPA).This focal plane arrays (FPA) can be embodied as the assembly (such as, being sealed by cap 130 and substrate 140) of such as Vacuum Package.In one embodiment, infrared sensor package 128 can be embodied as wafer-class encapsulation (such as, infrared sensor package 128 can be and be arranged on the monolithic that on wafer, one group of vacuum packaging assembly is separated).In one embodiment, the power supply that infrared sensor package 128 can be embodied as use about 2.4 volts, 2.5 volts, 2.8 volts or similar voltage carrys out work.

Infrared sensor 132 can be configured to the infrared radiation of detection target scene (such as, infrared energy), described target scene comprises: such as medium-wave infrared wave band (MWIR), long wave infrared region (LWIR) and/or as other desired in a particular application thermal imaging wave bands.In one embodiment, infrared sensor package 128 can be provided according to wafer-class encapsulation technology.

Infrared sensor 132 can be embodied as such as micro-metering bolometer, or is configured to the thermal imaging infrared sensor of the other types providing multiple pixel with the array pattern of any desired.In one embodiment, infrared sensor 132 can be embodied as vanadium oxide (VOx) detector with 17 micron pixel spacing.In various embodiments, the infrared sensor 132 of the infrared sensor 132 of about 32 × 32 arrays, about 64 × 64 arrays, the infrared sensor 132 of about 80 × 64 arrays or the array of other sizes can be used.

Substrate 140 can comprise various circuit, and comprising such as reading integrated circuit (ROIC), in one embodiment, the size of this reading integrated circuit (ROIC) is less than about 5.5mm × 5.5mm.Substrate 140 also can comprise bond pad 142, and it can be used for when assembling infrared imaging module 100 as shown in Figure 3, contacts with the complementary tie point on the inside surface being placed on shell 120.In one embodiment, the low-dropout regulator (LDO) performing voltage-regulation can be utilized to realize ROIC, to reduce the noise be incorporated in infrared sensor package 128, thus provide the Power Supply Rejection Ratio (PSRR) of improvement.In addition, by realizing having the LDO (such as, in wafer-level packaging) of ROIC, less die area can be consumed and the discrete tube core (or chip) needed is less.

Fig. 4 shows the block diagram comprising the infrared sensor package 128 of infrared sensor 132 array according to disclosure embodiment.In the embodiment as shown, infrared sensor 132 is as a part for the elementary cell array of ROIC402.ROIC402 comprises bias voltage and produces and timing control circuit 404, column amplifier 405, row multiplexer 406, row multiplexer 408 and output amplifier 410.The picture frame (that is, heat picture) of being caught by infrared sensor 132 by output amplifier 410 is supplied to processing module 160, processor 195 and/or any other suitable parts, to perform various treatment technology described herein.Although shown in Fig. 4 be 8 × 8 array, the array configurations of any expectation all can be used in other embodiments.ROIC and infrared sensor (such as, micro-metering bolometer circuit) further describe the U.S. Patent No. 6,028 can issued on February 22nd, 2000, find in 309, it can be used as entirety to be herein incorporated by way of reference.

Infrared array sensor 128 can catch image (such as, picture frame), and provides this image with various speed from its ROIC.Processing module 160 can be used for performing suitable process to the infrared image of catching, and can realize this processing module 160 according to any suitable structure.In one embodiment, processing module 160 can be embodied as ASIC.With regard to this respect, this ASIC can be configured to high performance and/or high efficiency execution image procossing.In another embodiment, general Central Processing Unit (CPU) can be utilized to realize processing module 160, described CPU can be configured to perform suitable software instruction, to carry out image procossing, adjustment and to carry out image procossing, mutual and/or other operations of working in coordination between processing module 160 and host apparatus 102 by various image processing block.In another embodiment, field programmable gate array (FPGA) can be utilized to realize processing module 160.In other embodiments, as understood by those skilled in the art, the process of other types and/or logical circuit can be utilized to realize processing module 160.

In these and other embodiments, processing module 160 also can realize by the parts suitable with other, such as, volatile memory, nonvolatile memory and/or one or more interface are (such as, infrared detector interface, internal integrated circuit (I2C) interface, mobile Industry Processor Interface (MIPI), JTAG (JTAG) interface (such as, IEEE1149.1 standard test access port and boundary-scan architecture) and/or other interfaces).

In some embodiments, infrared imaging module 100 can comprise one or more actuator 199 further, and it can be used for the focus adjusting the infrared image frame that infrared sensor package 128 is caught.Such as, the miscellaneous part that actuator 199 can be used for mobile optical element 180, infrared sensor 132 and/or is relative to each other, optionally to focus on according to technology described herein and to defocus infrared image frame.Actuator 199 can be realized according to the motional induction equipment of any type or device, and actuator 199 can be placed on the inner or outside any position of infrared imaging module 100, to adapt to different application.

After infrared imaging module 100 being assembled, infrared sensor package 128, pedestal 150 and processing module 160 can seal by shell 120 subsequently completely.Shell 120 can be convenient to the connection of the various parts of infrared imaging module 100.Such as, in one embodiment, shell 120 can be provided for the electric connecting part 126 connecting various parts, will be described in greater detail below.

When infrared imaging module 100 being assembled, electric connecting part 126 (such as, the electric connecting part of conductive path, electrical trace or other types) can be electrically connected with bond pad 142.In various embodiments, can electric connecting part 126 be embedded in shell 120, be arranged on the inside surface of shell 120 and/or by shell 120 described electric connecting part 126 is provided.As shown in Figure 3, electric connecting part 126 can end in the link 124 of the basal surface protruding from shell 120.When infrared imaging module 100 being assembled, link 124 can be connected with circuit board 170 (such as, in various embodiments, shell 120 can be placed in circuit board 170 top).Processing module 160 is electrically connected with circuit board 170 by suitable electric connecting part.Therefore, infrared sensor package 128 can be such as electrically connected with processing module 160 by conductive path, and described conductive path can be provided by the electric connecting part 126 of the complementary tie point in bond pad 142, shell 120 interior surface, shell 120, link 124 and circuit board 170.Advantageously, the realization of this layout can without the need to arranging bonding wire between infrared sensor package 128 and processing module 160.

In various embodiments, the material of any expectation (such as, copper or any other suitable conductive material) can be used to manufacture electric connecting part 126 in shell 120.In one embodiment, the heat that electric connecting part 126 can contribute to infrared imaging module 100 produces dispels the heat.

Other connections can be used in other embodiments.Such as, in one embodiment, sensor module 128 is connected to processing module 160 by ceramic wafer, and described ceramic wafer is connected to sensor module 128 by bonding wire and is connected to processing module 160 by ball grid array (BGA).In another embodiment, sensor module 128 directly can be installed on hard and soft plate and to be electrically connected with bonding wire, and bonding wire or BGA can be utilized processing module 160 to be installed and are connected to hard and soft plate.

The various application of infrared imaging module 100 described in this paper and host apparatus 102 are just in order to illustrate, instead of restriction.With regard to this respect, any one in various technology described herein all may be used on any infrared camera system, infrared imaging device or other devices for carrying out infrared/thermal imaging.

The substrate 140 of infrared sensor package 128 can be installed on pedestal 150.In various embodiments, pedestal 150 (such as, base) such as by the copper production formed by metal injection moulding (MIM), and can carry out black oxidation process or nickel coating process to described pedestal 150.In various embodiments, pedestal 150 can by the material manufacture of any expectation, such as, can according to application-specific, by such as zinc, aluminium or magnesium manufacture, and, pedestal 150 is formed by the application flow of any expectation, such as, according to application-specific, such as, can be formed by the quick cast of aluminium casting, MIM or zinc.In various embodiments, pedestal 150 can be used for providing support structure, various circuit paths, heat radiator performance and other suitable functions.In one embodiment, pedestal 150 can be the sandwich construction using stupalith to realize at least partly.

In various embodiments, circuit board 170 can hold shell 120, thus can support the various parts of infrared imaging module 100 physically.In various embodiments, circuit board 170 can be embodied as printed circuit board (PCB) (such as, the circuit board of FR4 circuit board or other types), the interconnect equipment (such as, the interconnect equipment of interconnection belt or other types) of rigidity or flexibility, flexible circuit board, flexible plastic substrates or other suitable structures.In various embodiments, pedestal 150 can be embodied as various function and the attribute of the circuit board 170 with description, and vice versa.

Socket 104 can comprise the cavity 106 being configured to hold infrared imaging module 100 (view such as, after assembling as shown in Figure 2).Infrared imaging module 100 and/or socket 104 can comprise suitable card, arm, pin, securing member or any other suitable attachment, described attachment can be used for, by friction, tension force, adhesion and/or any other suitable mode, infrared imaging module 100 is fixed to socket 104, or it is inner infrared imaging module 100 to be fixed to socket 104.Socket 104 can comprise attachment 107, and it can when being inserted in the cavity 106 of socket 104 when infrared image-forming module 100, the surface 109 of splice closure 120.The attachment of other types can be used in other embodiments.

Infrared imaging module 100 is electrically connected with socket 104 by suitable electric connecting part (such as, contact, pin, electric wire or any other suitable link).Such as, socket 104 can comprise electric connecting part 108, it can contact to the corresponding electric connecting part of infrared imaging module 100 (such as, interconnect pad, contact or other electric connecting parts on circuit board 170 profile or basal surface, engage other electric connecting parts on keyboard 142 or pedestal 150 or other links).Electric connecting part 108 can be manufactured by the material of any expectation (such as, copper or any other suitable conductive material).In one embodiment, electric connecting part 108 can by the flattening of machinery, can against the electric connecting part of infrared imaging module 100 when infrared image-forming module 100 is inserted in the cavity 106 of socket 104.In one embodiment, what electric connecting part 108 can be at least part of is fixed to infrared imaging module 100 in socket 104.The electric connecting part of other types can be used in other embodiments.

Socket 104 is electrically connected with main frame 102 by the electric connecting part of similar type.Such as, in one embodiment, main frame 102 can comprise the electric connecting part (such as, be welded to connect, buckle type connects or other connect) be connected with electric connecting part 108 through hole 190.In various embodiments, this electric connecting part can be placed in profile and/or the bottom of socket 104.

Realize the various parts of infrared imaging module 100 by flip chip technology (fct), described flip chip technology (fct) can be used for parts to be directly installed on circuit board, and without the need to being generally used for the extra gap that bonding wire connects.Flip-chip connects the overall dimensions being such as used in and reducing infrared imaging module 100 in compact little form factor application.Such as, in one embodiment, can use flip-chip link that processing module 160 is installed to circuit board 170.Such as, this flip-chip arrangement can be used to realize infrared imaging module 100.

In various embodiments, can be 12/844 according to such as application number, 124, the applying date is the U.S. Patent application on July 27th, 2010 and application number is 61/469,651, the various technology of the applying date described in the U.S. Provisional Patent Application on March 30th, 2011 (such as, justifying brilliant level encapsulation technology), realize infrared imaging module 100 and/or relevant parts, it can be used as entirety to be herein incorporated by way of reference.In addition, according to one or more embodiment, the various technology can recorded according to document as described below realize, correct, test and/or the parts using infrared imaging module 100 and/or be correlated with, described document is such as: if publication number is 7, 470, 902, publication date is the United States Patent (USP) on Dec 30th, 2008, publication number is 6, 028, 309, publication date is the United States Patent (USP) on February 22nd, 2000, publication number is 6, 812, 465, publication date is the United States Patent (USP) on November 2nd, 2004, publication number is 7, 034, 301, publication date is the United States Patent (USP) on April 25th, 2006, publication number is 7, 679, 048, publication date is the United States Patent (USP) on March 16th, 2010, publication number is 7, 470, 904, publication date is the United States Patent (USP) on Dec 30th, 2008, application number is 12/202, 880, the applying date is the U.S. Patent application on September 2nd, 2008 and application number is 12/202, 896, the applying date is the U.S. Patent application on September 2nd, 2008, by way of reference above-mentioned document is herein incorporated as a whole.

In some embodiments, host apparatus 102 can comprise miscellaneous part 198, such as non-thermal video camera (such as, the non-thermographic device of visible light camera or other types).Non-thermal video camera can be little form factor image-forming module or imaging device, and in some embodiments, can realize in the mode similar with the various embodiments of infrared imaging module 100 disclosed herein, wherein one or more sensors and/or sensor array respond the radiation (radiation such as, in visible wavelength, ultraviolet wavelength and/or other non-thermal wavelengths) in non-thermal frequency spectrum.Such as, in some embodiments, non-thermal video camera charge available coupled apparatus (CCD) sensor, electron multiplication CCD (EMCCD) sensor, complementary metal oxide semiconductor (CMOS) (CMOS) sensor, Scientific Grade CMOS (sCMOS) sensor or other wave filters and/or sensor realize.

In some embodiments, non-thermal video camera can be stayed and be oriented to the visual field making at least part of overlapping infrared imaging module 100 in the visual field of non-thermal video camera (FOV) altogether with infrared imaging module 100.In an example, according to be incorporated to by way of reference herein, application number that on Dec 31st, 2012 proposes is 61/748, the U.S. Provisional Patent Application of 018 describes various technology, and infrared imaging module 100 and non-thermal video camera can be implemented as the dual sensor module of shared public substrate.

For the embodiment with this non-thermo-optical video camera, various parts (such as, processor 195, processing module 160 and/or other processing element) superposition can be configured to, merge, mixing or otherwise combine be no matter the substantially identical time or the substantially different time (such as, time interval exceedance hour, a couple of days, relative night on daytime etc.), the infrared image of being caught by infrared imaging module 100 (such as, comprise heat picture) and the non-thermographic (such as, comprising visible images) of being caught by non-thermal video camera.

In some embodiments, heat can be processed with non-thermographic to produce the image (one or more such as, in some embodiments, execution this image process) combined.Such as, the NUC process (just as further described herein) based on scene can be performed, euchroic process can be performed, and/or high-contrast process can be performed.

For euchroic process, mixed with non-thermographic by heat picture by such as the radiation detection component of heat picture and the respective components of non-thermographic being mixed according to hybrid parameter, hybrid parameter can be adjusted by machine by user and/or in some embodiments.Such as, brightness and the chromatic component of heat and non-thermographic is may be combined with according to hybrid parameter.In one embodiment, such hybrid technology can be called as euchroic infrared imaging.Such as, by day during imaging, the image of mixing can comprise the non-thermal coloured image containing luminance component and chromatic component, and wherein its brightness value is replaced by the brightness value from heat picture.To the use of the brightness data from heat picture, the object-based temperature of intensity of real non-thermal coloured image is brightened or dimmed.So, these hybrid technologies are that daytime or visible images provide thermal imaging.

For high-contrast process, high spatial frequency content can obtain from one or more heat and non-thermographic (such as, by performing high-pass filtering, Difference Imaging and/or other technologies).Combination image can comprise radiation detection component and the mixed components of heat picture, mixed components comprise according to hybrid parameter be mixed with the scene of high spatial frequency content infrared (such as, heat) feature, hybrid parameter can be adjusted by user and/or machine in some embodiments.In some embodiments, by high spatial frequency content is added on heat picture, high spatial frequency content from non-thermographic can be mixed with heat picture, wherein high spatial frequency content replace or cover heating image corresponding to these part heat pictures that there is high spatial frequency content.Such as, high spatial frequency content can be included in the edge of the object described in scene image, but can be present in the inside of these objects this.In this embodiment, blended image data simply can comprise high spatial frequency content, and high spatial frequency content can be encoded into the content of the high spatial frequency of one or more components of combination image subsequently.

Such as, the radiation components of heat picture measures the chromatic component component that component can be this heat picture, and high spatial frequency content can obtain from the brightness and/or chromatic component component being divided into non-thermographic.In this embodiment, the radiation components that combination image can comprise the chromatic component component being encoded into combination image measures component (such as, the chromatic component component of heat picture), the high spatial frequency content of the luminance components component of combination image is become with direct coding (such as, as blended image data but the blended image data not having heat picture to contribute).By doing like this, the radiation components that can retain heat picture measures the radiation detection standard calibration of component.In similar embodiment, blended image data can comprise the high spatial frequency content adding the luminance components component being added to heat picture, and is encoded into the result blended data as a result of luminance components component of result combination image as a result.

Such as, any technology disclosed in following application can be used in various embodiments: the application number that on June 3rd, 2009 proposes is the U.S. Patent application of 12/477,828; The application number that on April 23rd, 2010 proposes is the U.S. Patent application of 12/766,739; The application number that on May 11st, 2011 proposes is the U.S. Patent application of 13/105,765; The application number that on April 2nd, 2012 proposes is the U.S. Patent application of 13/437,645; The application number that on April 8th, 2011 proposes is the U.S. Provisional Patent Application of 61/473,207; The application number that on Dec 26th, 2012 proposes is the U.S. Provisional Patent Application of 61/746,069; The application number that on Dec 26th, 2012 proposes is the U.S. Provisional Patent Application of 61/746,074; The application number that on Dec 31st, 2012 proposes is the U.S. Provisional Patent Application of 61/748,018; The application number that on March 15th, 2013 proposes is the U.S. Provisional Patent Application of 61/792,582; The application number that on March 15th, 2013 proposes is the U.S. Provisional Patent Application of 61/793,952; The application number of proposition was the international patent application of PCT/EP2011/056432 with on April 21st, 2011, and all these applications are incorporated to herein all by way of reference as a whole.Other any technology of applying for or describing in patent described herein or quote herein, all can be applicable to any thermal described herein, non-thermal, and in application.

Refer again to Fig. 1, in various embodiments, host apparatus 102 can comprise shutter 105.With regard to this respect, when infrared imaging module 100 is arranged in socket, shutter 105 optionally can be placed on (such as, direction as determined in arrow 103) on socket 104.With regard to this respect, shutter 105 is such as used in when infrared imaging module 100 does not use and protects it.Shutter 105 also can be used as temperature reference, as those skilled in the art be to be understood that, described temperature reference is as a part for the trimming process (such as, Nonuniformity Correction (NUC) process or other trimming processes) of infrared imaging module 100.

In various embodiments, shutter 105 can be manufactured by various material, such as, and polymkeric substance, glass, aluminium (such as, japanning or after anodized) or other materials.In various embodiments, shutter 105 can comprise one or more coating (such as, uniform black matrix coating or reflexive gold coatings), and it is for the various optical properties of optionally filter electromagnetic radiation and/or adjustment shutter 105.

In another embodiment, shutter 105 can be fixed on appropriate location with round-the-clock protection infrared imaging module 100.In this case, the part of shutter 105 or shutter 105 can by substantially can not filter out needs Infrared wavelength suitable material (such as, polymkeric substance, or the infrared transmission material of such as silicon, germanium, zinc selenide or chalcogenide glass) manufacture.As those skilled in the art be to be understood that, in another embodiment, shutter can be embodied as a part for infrared imaging module 100 (such as, in the miscellaneous part of lens barrel or infrared imaging module 100, or as the part of the miscellaneous part of lens barrel or infrared imaging module 100).

Alternatively, in another embodiment, without the need to providing shutter (such as, the outside of shutter 105 or other types or inner shutter), but the technology without shutter can be used to carry out the correction of NUC step or other types.In another embodiment, use and can carry out with the combine with technique based on shutter without the NUC step of fast gate technique or the correction of other types.

Any one in the various technology can recorded according to following document realizes infrared imaging module 100 and host apparatus 102, and described document is: application number is 61/495,873, and the applying date is the U.S. Provisional Patent Application on June 10th, 2011; Application number is 61/495,879, and the applying date is the U.S. Provisional Patent Application on June 10th, 2011; And application number is 61/495,888, the applying date is the U.S. Provisional Patent Application on June 10th, 2011, is herein incorporated as a whole by above-mentioned document by way of reference.

In various embodiments, the parts of host apparatus 102 and/or infrared imaging module 100 can be embodied as local system, or are embodied as between parts and carry out by wired and/or wireless network the distributed system that communicates.Therefore, according to the needs of particular implementation, the various operations mentioned by the disclosure can be performed by local and/or remote units.

Fig. 5 shows the process flow diagram of the various operations of the determination NUC item according to disclosure embodiment.In some embodiments, the processing module 160 that can be processed by the picture frame of catching infrared sensor 132 or processor 195 (the two usually also finger processor) perform the operation of Fig. 5.

At block 505, infrared sensor 132 starts the picture frame of capturing scenes.Usually, scene will be the current true environment be in of host apparatus 102.With regard to this respect, shutter 105 (if optionally providing) can be opened to allow infrared imaging module to receive infrared radiation from scene.During all operations shown in Fig. 5, infrared sensor 132 catches picture frame serially.With regard to this respect, catch picture frame continuously and can be used for various operation as further discussed.In one embodiment, time-domain filtering can be carried out (such as to the picture frame of catching, step according to block 826 carries out time-domain filtering to the picture frame of catching, to be described further according to Fig. 8 herein), and before described picture frame is used to the operation shown in Fig. 5, by other (such as, factory's gain term 812, factory's shift term 816, the NUC item 817 previously determined, row FPN item 820 and row FPN item 824, will be described further it according to Fig. 8 herein) they are processed.

At block 510, the startup event of NUC step detected.In one embodiment, NUC step can start in response to the physics of host apparatus 102 moves.Such as, this movement can be detected by by the motion sensor 194 of processor poll.In one example in which, for mobile host device 102 may be carried out in a particular manner, such as, by the host apparatus 102 that moves around of having a mind to, host apparatus 102 is made to do " elimination " or " bang " motion.With regard to this respect, user can according to predetermined speed and direction (speed), such as, is carried out mobile host device 102 by the motion of upper and lower, left and right or other types thus started NUC step.In this example, the use of this movement can allow user's operating host device 102 intuitively, to simulate the noise " elimination " to the picture frame of catching.

In another example, if detect that motion exceedes threshold value (such as, motion has exceeded the normal use expected), then can start NUC step by host apparatus 102.Can be expected that, the spatial displacement of the type of any expectation of host apparatus 102 all can be used for starting NUC step.

In another example again, if since the NUC step previously performed, pass by minimum time, then can start NUC step by host apparatus 102.In another example, if since the NUC step previously performed, infrared imaging module 100 experienced by minimum temperature change, then can start NUC step by host apparatus 102.In other example, start serially and repeat NUC step.

At block 515, after NUC step startup event being detected, determine whether perform NUC step veritably.With regard to this respect, whether can meet based on one or more subsidiary condition, optionally start NUC step.Such as, in one embodiment, unless since the NUC step previously performed, pass by minimum time, otherwise NUC step can not have been performed.In another embodiment, unless since the NUC step previously performed, infrared imaging module 100 experienced by minimum temperature variation, otherwise can not perform NUC step.Other standards or condition can be used in other embodiments.If met suitable standard or condition, process flow diagram will proceed to block 520.Otherwise process flow diagram turns back to block 505.

In NUC step, fuzzy graph picture frame can be used for determining NUC item, and the picture frame that described NUC item can be applicable to catch is to correct FPN.As discussed, in one embodiment, the multiple picture frames (picture frame of such as, catching when scene and/or thermal imaging system are in the state of motion) by cumulative moving scene obtain fuzzy graph picture frame.In another embodiment, defocus by the optical element or miscellaneous part making thermal imaging system, obtain fuzzy graph picture frame.

Therefore, block 520 provides the selection of two kinds of methods.If use based drive method, then process flow diagram proceeds to block 525.If used based on the method defocused, then process flow diagram proceeds to block 530.

With reference now to based drive method, at block 525, motion detected.Such as, in one embodiment, the picture frame can caught based on infrared sensor 132 detects motion.With regard to this respect, suitable motion detection step (such as, image registration step, frame are to the mathematic interpolation of frame or other suitable steps) can be applicable to the picture frame of catching, to determine whether there is motion (such as, whether having captured picture frame that is static or motion).Such as, in one embodiment, can determine that the quantity that the pixel of the surrounding of the pixel of successive image frame or region change has exceeded user-defined quantity (such as, number percent and/or threshold value).If the pixel of at least given number percent has changed and the quantity of the pixel changed is at least user-defined quantity, then that can affirm very much has detected motion, thus process flow diagram forwards block 535 to.

In another embodiment, can determine motion on the basis of each pixel, wherein, only cumulative those demonstrate the pixel of significant change, to provide fuzzy graph picture frame.Such as, can arrange counter for each pixel, described counter is identical for ensureing the quantity of the pixel value that each pixel adds up, or is averaged pixel value for the quantity of the pixel value in fact added up according to each pixel.The motion based on image that can perform other types detects, and such as, performs and draws east (Radon) to convert.

In another embodiment, the data that can provide based on motion sensor 194 detect motion.In one embodiment, whether this motion detects to comprise and detects host apparatus 102 and move along relative to straight track in space.Such as, if host apparatus 102 just moves along relative to straight track, so following situation is possible: occur that some object in scene after imaging may fuzzy not (object such as, in scene be aimed at straight track or substantially moved along the direction being parallel to described straight track).Therefore, in this embodiment, when only having host apparatus 102 demonstrate motion or do not demonstrate motion but move along particular track, motion sensor 194 just can detect motion.

In another embodiment again, both motion detection step and motion sensor 194 can be used.Therefore, use any one in these various embodiments, can determine scene at least partially and host apparatus 102 relative to each other between motion while (such as, this can be moved relative to scene by host apparatus 102, scene move relative to host apparatus 102 at least partially or above-mentioned two situations cause), whether capture each picture frame.

Can be expected that, detect the picture frame of motion can demonstrate some of the scene of catching secondary fuzzy (such as, the fuzzy thermographic image data relevant to scene), described secondary fuzzy be cause alternately because thermal time constant (such as, micro-metering bolometer thermal time constant) and the scene of infrared sensor 132 moves.

At block 535, to detecting that the picture frame of motion adds up.Such as, if the motion of continuous print a series of images frame detected, then can add up to series of drawing picture frame.As another one example, if the motion of some picture frame only detected, then can neglect and not have the picture frame moved not add up to the picture frame that these do not move.Therefore, can, based on the motion detected, continuous print or discontinuous a series of images frame be selected to add up.

At block 540, be averaged to provide fuzzy graph picture frame to cumulative picture frame.Because cumulative picture frame during movement captures, so scene information actual between our desired image frame will be different, thus cause fuzzy after picture frame in scene information by further fuzzy (block 545).

In contrast, during movement, within least short time and at least limited change of scene radiation time, FPN (such as, being caused by one or more parts of infrared imaging module 100) remains unchanged.As a result, picture frame close on the Time and place during movement captured will suffer identical or at least similar FPN.Therefore, although the scene information in successive image frame may change, FPN will keep substantially constant.By being averaged to the multiple picture frames captured between moving period, described multiple picture frame will fuzzy scene information, but can not fuzzy FPN.As a result, compared with scene information, FPN keeps in the fuzzy graph picture frame provided at block 545 clearly.

In one embodiment, in block 535 and 540, cumulative sum carried out to 32 or more picture frames average.But the picture frame of any desired amt is all in other embodiments available, just along with the minimizing of the quantity of frame, correction accuracy can reduce usually.

With reference now to based on the method defocused, at block 530, carry out defocusing operations and defocus with the picture frame making infrared sensor 132 wittingly and catch.Such as, in one embodiment, one or more actuator 199 can be used for adjusting, the miscellaneous part of mobile or translation optical element 180, infrared sensor package 128 and/or infrared imaging module 100, to make fuzzy (such as, not focusing on) picture frame of infrared sensor 132 capturing scenes.Also can consider to use other not make infrared image frame defocus wittingly based on the technology of actuator, such as, as artificial (such as, user starts) defocuses.

Although the scene in picture frame may occur fuzzy, by defocusing operations, FPN (such as, being caused by one or more parts of infrared imaging module 100) will remain unaffected.As a result, the fuzzy graph picture frame of scene (block 545) will have FPN, and compared with scene information, described FPN keeps in described blurred picture clearly.

In superincumbent discussion, what described is relevant with single picture frame of catching based on the method defocused.In another embodiment, can comprise based on the method defocused and when infrared image-forming module 100 is defocused, multiple picture frame being added up, and the picture frame defocused is averaged to eliminate the impact of noise in time domain and provides fuzzy graph picture frame at block 545.

Therefore, be understandable that, both by based drive method also by providing fuzzy picture frame based on the method defocused at block 545.Because motion, to defocus or said two devices all can make a lot of scene informations fuzzy, so in fact fuzzy graph picture frame can be thought the low-pass filtering version of the picture frame of original relevant scene information of catching.

At block 550, the FPN item of the row and column determining to upgrade is processed (such as to fuzzy graph picture frame, if do not determine the FPN item of row and column before, the FPN item of the row and column so upgraded can be block 550 first time the new row and column in iteration FPN item).As the disclosure use, according to the direction of the miscellaneous part of infrared sensor 132 and/or infrared imaging module 100, the interchangeable use of term row and column.

In one embodiment, block 550 comprises determines that often row fuzzy graph picture frame (such as, often row fuzzy graph picture frame can have the space FPN correction term of himself) space FPN correction term, and also determine the space FPN correction term of often row fuzzy graph picture frame (such as, often row fuzzy graph picture frame can have the space FPN correction term of himself).This process can be used for reducing space and the slow change (1/f) reducing the intrinsic row and column FPN of thermal imaging system, this slow change case is caused by the 1/f noise feature of the amplifier in ROIC402 in this way, and described 1/f noise feature can show as the vertical and horizontal bar in picture frame.

Advantageously, by the FPN utilizing fuzzy graph picture frame to determine space row and column, can reduce and the vertical and horizontal in the scene of actual imaging be thought by mistake be the risk (such as, real scene content is fuzzy, and FPN maintenance is not fuzzy) of row and column noise.

In one embodiment, by consider fuzzy graph picture frame neighbor between difference determine row and column FPN item.Such as, Fig. 6 to show according to the neighbor of disclosure embodiment between difference.Particularly, in figure 6, pixel 610 and 8 horizontal adjacent pixels near it are compared: d0-d3 is in side, and d4-d7 is at opposite side.Difference between neighbor can be averaged, to obtain the estimated value of the offset error of the pixel groups illustrated.All can calculate the offset error of each pixel in row or row, and the mean value obtained can be used for correcting whole row or row.

In order to prevent that real contextual data is interpreted as noise, can SC service ceiling threshold value and lower threshold (thPix and-thPix).The pixel value (in this example embodiment, being pixel d1 and d4) fallen into outside this threshold range is not used in acquisition offset error.In addition, these threshold values can limit the maximum that row and column FPN corrects.

Application number is 12/396,340, and the applying date is that the U.S. Patent application on March 2nd, 2009 describes the technology more specifically performing space row and column FPN correction process, it can be used as entirety to be herein incorporated by way of reference.

Refer again to Fig. 5, the row and column FPN item of the renewal determined at block 550 is carried out storing (block 552) and is applied to the fuzzy graph picture frame that (block 555) block 545 provides.After applying these, the FPN of some the space row and columns in fuzzy graph picture frame can be reduced.Such as, but because these are applied to row and column usually, so additional FPN can keep, skew or the other reasons of space-independent FPN and pixel to pixel are relevant.With single row and column may not be directly related, the neighborhood of the FPN of space correlation also can remain unchanged.Therefore, can be further processed to determine NUC item, will be described below.

At block 560, determine local contrast value in fuzzy graph picture frame (the gradient edge value such as, between neighbor or small group of pixels or absolute value).If the scene information in fuzzy graph picture frame comprises also not by obviously fuzzy contrast region (such as, the high-contrast edge in Raw scene data), so these features can be identified by the contrast determining step of block 560.

Such as, the local contrast value in fuzzy graph picture frame can be calculated, or the edge detecting step of any other type can be applicable to identify as local contrast region a part, some pixel in blurred picture.Can think that the pixel marked by this way comprises the scene information of very high spatial frequency, the scene information of this very high spatial frequency can be interpreted as FPN (such as, this region may correspond in also not by the part of fully fuzzy scene).Therefore, these pixels can be got rid of outside the process being used for determining further NUC item.In one embodiment, this contrast check processing can be dependent on higher than the expectation contrast value relevant to FPN threshold value (such as, can think that the contrast value that demonstrates is scene information higher than the pixel of threshold value, and think that those pixels lower than threshold value are display FPN).

In one embodiment, after row and column FPN item has been applied to fuzzy graph picture frame, can determine (such as, as shown in Figure 5) the contrast of fuzzy graph picture frame execution block 560.In another embodiment, can before block 550 execution block 560, to determine contrast (such as, to prevent contrast based on scene for determining that this has impact) before determining row and column FPN item.

After block 560, can be expected that, any high spatial frequency component remained in fuzzy graph picture frame can be general owing to space-independent FPN.With regard to this respect, after block 560, other noises a lot of or the real information based on scene needed are removed or got rid of outside fuzzy graph picture frame, this is because: to fuzzy wittingly (such as, by from the motion of block 520 to 545 or defocus) of picture frame, the application (block 555) of row and column FPN item and the determination (block 560) of contrast.

Therefore, it is expected to, after block 560, any residual high spatial frequency component (such as, being shown as the contrast in fuzzy graph picture frame or distinct regions) is all attributable to space-independent FPN.Therefore, at block 565, high-pass filtering is carried out to fuzzy graph picture frame.In one embodiment, this can comprise application Hi-pass filter to extract high spatial frequency component from fuzzy graph picture frame.In another embodiment, this can comprise fuzzy graph picture frame application of low-pass filters, and the difference extracted between the picture frame after low-pass filtering and the picture frame not having filtering is to obtain high spatial frequency component.According to various embodiment of the present disclosure, realize Hi-pass filter by the mean difference between calculating sensor signal (such as, pixel value) and its adjacent signals.

At block 570, flat field correction process is carried out to the fuzzy graph picture frame after high-pass filtering, to determine the NUC item (such as, if previously do not carry out NUC step, the NUC item so upgraded can be the new NUC item of first time in iteration of block 570) upgraded.

Such as, Fig. 7 shows the flat field correction technology 700 according to disclosure embodiment.In the figure 7, the NUC item by using the value of neighbor 712 to 726 of pixel 710 to determine each pixel 710 of fuzzy graph picture frame.For each pixel 710, several gradient can be determined based on the absolute difference between the value of various neighbor.Such as, the absolute difference between following pixel can be determined: between pixel 712 and 714 between (diagonal angle gradient from left to right), pixel 716 and 718 between (VG (vertical gradient) from top to bottom), pixel 720 and 722 between (diagonal angle gradient from right to left) and pixel 724 and 726 (horizontal gradient from left to right).

Can sue for peace to these absolute differences, to provide the summation gradient of pixel 710.Can determine the weighted value of pixel 710, described weighted value is inversely proportional to summation gradient.This step can be performed, until provide weighted value for each pixel 710 to whole pixels 710 of fuzzy graph picture frame.For the region (such as, by fuzzy region or the region with low contrast) with low gradient, weighted value will close to 1.On the contrary, for the region with high gradient, weighted value will be 0 or close to 0.Updated value as the NUC item estimated by Hi-pass filter is multiplied with weighted value.

In one embodiment, by a certain amount of time decay is applied to NUC item determining step, risk scene information being incorporated into NUC item can be reduced further.Such as, the time decay factor λ between 0 and 1 can be selected, the new NUC item (NUC stored like this _nEW) be old NUC item (NUC _oLD) and the NUC item (NUC of renewal that estimates _uPDATE) average weighted value.In one embodiment, this can be expressed as: NUC _nEW=λ NUC _oLD+ (1-λ) (NUC _oLD+ NUC _uPDATE).

Determine NUC item although described according to gradient, local contrast value time suitable, also can be used to replace gradient.Also other technologies can be used, such as, standard deviation calculation.The flat field correction step that can perform other types, to determine NUC item, comprising: the such as patent No. is 6,028,309, and the day for announcing is the United States Patent (USP) on February 22nd, 2000; The patent No. is 6,812,465, and the day for announcing is the United States Patent (USP) on November 2nd, 2004; And application number is 12/114,865, the various steps of the applying date described in the U.S. Patent application on May 5th, 2008.By way of reference above-mentioned document is herein incorporated as a whole.

Refer again to Fig. 5, block 570 can comprise the additional treatments to NUC item.Such as, in one embodiment, in order to retain the mean value of scene signals, by the mean value that deducts NUC item from each NUC item by whole NUC item and normalize to 0.Of the same race at block 570, in order to avoid row and column noise effect NUC item, the mean value of every row and column can be deducted from the NUC item of every row and column.Result is, the row and column FPN wave filter being used in the row and column FPN item that block 550 is determined after can filtering out better and NUC item being applied to the image of catching (such as, in the step that block 580 carries out, to be further described this herein) further iteration in the row and column noise of (such as, as Fig. 8 be shown specifically).With regard to this respect, row and column FPN wave filter can use more data to calculate often row the and often deviation ratio that arranges is (such as usually, the FPN item of row and column), and with carry out the NUC item of incoherent noise on capture space based on Hi-pass filter compared with, can thus provide more reliably, for reducing the option of the FPN of space correlation.

At block 571-573, can perform additional high-pass filtering to the NUC item upgraded alternatively and further determine that process is with the FPN eliminating space correlation, the FPN of described space correlation has the spatial frequency lower than the previous spatial frequency eliminated by row and column FPN item.With regard to this respect, some changes of the miscellaneous part of infrared sensor 132 or infrared imaging module 100 can produce the FPN noise of space correlation, can not easily by the FPN noise modeling of produced space correlation for row or row noise.The FPN of this space correlation can comprise the transmitted fluorescence on such as sensor module or infrared sensor 132 groups, and described infrared sensor 132 groups is compared with adjacent infrared sensor 132, and it responds different radiancy.In one embodiment, offset correction can be used to reduce the FPN of this space correlation.If the quantity of the FPN of this space correlation is a lot, then also noise can be detected in fuzzy graph picture frame.Because such noise can affect neighbor, the Hi-pass filter with very little kernel may not detect that FPN in neighbor (such as, whole values that Hi-pass filter uses can from affected pixel near pixel extract, thus described whole value can by offset errors effect of the same race).Such as, if use the high-pass filtering of little kernel execution block 565 (such as, only consider the pixel of the direct neighbor of the environs of the pixel falling into the FPN impact being subject to space correlation), then the FPN of the space correlation of extensively distribution may not be detected.

Such as, Figure 11 shows the FPN according to the space correlation in pixel near disclosure embodiment.As collected seed shown in picture frame 1100, pixel near pixel 1110 can show the FPN of space correlation, inaccurate and the single row and column of FPN of described space correlation is relevant, and be distributed in neighbouring multiple pixels (such as, in this example embodiment, neighbouring pixel is about the pixel of 4 × 4).Seed collecting picture frame 1100 also comprises one group of pixel 1120 and one group of pixel 1130, and described pixel 1120 shows does not have substantially responding uniformly of use in filtering calculates, and described pixel 1130 is for estimating the low-pass value of the pixel near pixel 1110.In one embodiment, pixel 1130 can be can be divided into multiple pixels of 2, so that effective calculating of hardware or software.

Refer again to Fig. 5, at block 571-573, can optionally perform additional high-pass filtering to the NUC item upgraded and further determine process, to eliminate the FPN of space correlation, such as, the FPN of the space correlation that pixel 1110 shows.At block 571, the NUC item of the renewal determined at block 570 is applied to fuzzy graph picture frame.Therefore, now, fuzzy graph picture frame will for the FPN (such as, by applying the row and column FPN item upgraded at block 555) of preliminary corrections space correlation, and also for the space-independent FPN of preliminary corrections (such as, by applying the NUC item upgraded at block 571).

At block 572, further apply Hi-pass filter, the core of this Hi-pass filter is larger than the core of the Hi-pass filter used in block 565, and can determine the NUC item of renewal further at block 573.Such as, in order to detect the FPN of the space correlation existed in pixel 1110, the data of the enough large adjacent area from pixel can be comprised at the Hi-pass filter of block 572 application, thus can determine there is no affected pixel (such as, pixel 1120) and affected pixel (such as, pixel 1110) between difference.Such as, the low-pass filter (such as, the N × N kernel much larger than 3 × 3 pixels) with macronucleus can be used, and the result that obtains can be deducted to carry out suitable high-pass filtering.

In one embodiment, in order to counting yield, sparse kernel can be used, thus only use the neighbor of the lesser amt in N × N near zone.For Hi-pass filter operation (such as, there is the Hi-pass filter of macronucleus) of any given use neighbor far away, there is the risk (may be fuzzy) scene information of reality being modeled as the FPN of space correlation.Therefore, in one embodiment, the time decay factor λ of the NUC item being used for the renewal determined at block 573 can be set to close to 1.

In various embodiments, can repeatable block 571-573 (such as, cascade), high-pass filtering is performed iteratively to utilize the core size increased progressively, thus the NUC item upgraded further is provided, the NUC item of described further renewal is used for the FPN of the space correlation correcting the adjacent size area needed further.In one embodiment, the NUC item of the renewal that can obtain according to the prior operation by block 571-573, whether by elimination real for the FPN of space correlation, determines the decision performing this iteration.

After block 571-573 completes, make the decision (block 574) whether the NUC item of renewal being applied to the picture frame of catching.Such as, if the mean value of the absolute value of the NUC item of whole picture frame is less than minimum threshold value, or be greater than maximum threshold value, then can think that this NUC item is false or can not provides significant correction.Optionally, threshold criteria can be applied to each pixel, to determine which pixel-by-pixel basis receives the NUC item of renewal.In one embodiment, threshold value may correspond to the difference between the NUC item and the NUC item previously calculated of new calculating.In another embodiment, threshold value can independent of the NUC item previously calculated.Other tests (such as, spatial coherence test) can be applied to determine whether to apply this NUC item.

If think that NUC item is false or can not provides significant correction, then process flow diagram turns back to block 505.Otherwise, store the up-to-date NUC item (block 575) determined to substitute previous NUC item (such as, being determined by the iteration previously performed in Fig. 5), and the described up-to-date NUC item determined be applied to the picture frame that (block 580) catch.

Fig. 8 shows and operates with other according to the various image processing techniques being applied in the Fig. 5 in image processing pipeline 800 of disclosure embodiment.With regard to this respect, streamline 800 identifies when the processing scheme of the whole iterative images for correcting the picture frame that infrared imaging module 100 provides, the various operations of Fig. 5.In some embodiments, streamline 800 can be provided by the processing module 160 operated the picture frame of being caught by infrared sensor 132 or processor 195 (the two usually also finger processor).

The picture frame that infrared sensor 132 is caught can be supplied to frame averager 804, described frame averager 804 asks the integration of multiple picture frame to provide the picture frame 802 of the signal to noise ratio (S/N ratio) with improvement.By infrared sensor 132, ROIC402 and be embodied as and support that other assemblies that hi-vision catches the infrared sensor package 128 of speed provide frame averager 804 effectively.Such as, in one embodiment, infrared sensor package 128 can catch infrared image frame with the frame rate of 240Hz (such as, per second catch 240 width images).In this embodiment, such as by making infrared sensor package 128 be operated in relatively low voltage (such as, compatible mutually with the voltage of mobile phone), and by using relatively little infrared sensor 132 array (such as, in one embodiment, be the infrared array sensor of 64 × 64), realize frame rate high like this.

In one embodiment, with high frame rate (such as, 240Hz or other frame rate), this infrared image frame from infrared sensor package 128 can be supplied to processing module 160.In another embodiment, infrared sensor package 128 can carry out integration in longer time period or multiple time period, thus with lower frame rate (such as, 30Hz, 9Hz or other frame rate) (after such as, being averaged) the infrared image frame after integration is supplied to processing module 160.About can be used for providing compared with hi-vision catch the implementation of speed details can it can be used as by way of reference entirety be incorporated to herein, application number that on June 10th, 2011 proposes is find in the U.S. Provisional Patent Application of 61/495,879.

The picture frame 802 processed by streamline 800, for determining various adjustment item and gain compensation, wherein, is adjusted described picture frame 802 by various item, time-domain filtering.

At block 810 and 814, factory's gain term 812 and factory's shift term 816 are applied to picture frame 802, with the gain between the miscellaneous part compensating determined various infrared sensor 132 and/or infrared imaging module 100 during Computer-Assisted Design, Manufacture And Test respectively and offset deviation.

At block 580, NUC item 817 is applied to picture frame 802, to correct FPN as above.In one embodiment, if also do not determine NUC item 817 (such as, before starting NUC step), then may can not execution block 580, or initial value can be used for the NUC item 817 (such as, the off-set value of each pixel will equal 0) that view data can not be caused to change.

At block 818 to 822, respectively row FPN item 820 and row FPN item 824 are applied to picture frame 802.Row FPN item 820 and row FPN item 824 can be determined as mentioned above according to block 550.In one embodiment, if also do not determine row FPN item 820 and row FPN item 824 (such as, before starting NUC step), then may can not execution block 818 and 822, or the row FPN item 820 that initial value can be used for causing view data to change and row FPN item 824 (such as, the off-set value of each pixel will equal 0).

At block 826, according to noise in time domain abatement (TNR) step, time-domain filtering is performed to picture frame 802.Fig. 9 shows the TNR step according to disclosure embodiment.In fig .9, to the picture frame 802b process after the picture frame 802a be currently received and previous time-domain filtering to determine the picture frame 802e after new time-domain filtering.Picture frame 802a and 802b comprises local neighbor 803a and 803b respectively centered by pixel 805a and 805b.Neighbor 803a and 803b corresponds to the same position in picture frame 802a and 802b, and is the subset of the whole pixel of picture frame 802a and 802b.In the embodiment as shown, neighbor 803a and 803b comprises the region of 5 × 5 pixels.The neighbor of other sizes can be used in other embodiments.

Determine the difference of the pixel that neighbor 803a and 803b is corresponding and it is averaging, thinking that the position corresponding to pixel 805a and 805b provides average increment value 805c.Average increment value 805c is used in block 807 and determines weighted value, to apply it to pixel 805a and the 805b of picture frame 802a and 802b.

In one embodiment, as shown in curve 809, the weighted value determined at block 807 can be inversely proportional to average increment value 805c, and during to make that difference is larger between neighbor 803a and 803b, weighted value is reduced to 0 rapidly.With regard to this respect, between neighbor 803a and 803b, bigger difference can represent in scene and there occurs change (such as, the change occurred due to motion), and in one embodiment, suitable weighting can be carried out, to avoid to run into frame fuzzy to introducing during the scene change of frame to pixel 802a and 802b.Other associations between weighted value and average increment size 805c can be used in other embodiments.

The weighted value determined at block 807 can be used for pixel 805a and 805b, to determine the value (block 811) of the respective pixel 805e of picture frame 802e.With regard to this respect, pixel 805e can have according to the average increment value 805c determined at block 807 and weighted value the value after pixel 805a and 805b weighted mean (or other combinations).

Such as, the pixel 805e of the picture frame 802e after time-domain filtering may be the pixel 805a of picture frame 802a and 802b and the weighted sum of 805b.If the average difference between pixel 805a and 805b causes due to noise, so can be expected that, the change of the mean value between neighbor 805a and 805b will close to 0 (such as, corresponding to the mean value of incoherent change).In this case, can be expected that, the difference between neighbor 805a and 805b and will close to 0.In this case, suitable weighting can be carried out to the pixel 805a of picture frame 802a, to contribute to the value generating pixel 805e.

But, if this difference and be not 0 (such as, in one embodiment, even very close to 0), so can by change interpretation for being by kinetic, instead of caused by noise.Therefore, the change of the mean value that can show based on neighbor 805a and 805b detects motion.In this case, larger weight can be applied to the pixel 805a of picture frame 802a, and less weight is applied to the pixel 805b of picture frame 802b.

Other embodiments are also admissible.Such as, although what describe is determine average increment value 805c according to neighbor 805a and 805b, but in other embodiments, average increment value 805c can be determined according to the standard of any expectation (such as, according to the pixel groups be made up of a series of pixel of single pixel or other types).

In superincumbent embodiment, picture frame 802a is described as the picture frame be currently received, and picture frame 802b is described as the picture frame previously after time-domain filtering.In another embodiment, picture frame 802a and 802b can be infrared imaging module 100 capture also not through the first and second picture frames of time-domain filtering.

Figure 10 shows the detailed implementation detail relevant with the TNR step performed by block 826.As shown in Figure 10, respectively picture frame 802a and 802b is read into line buffer 1010a and 1010b, and before picture frame 802b (such as, previous image frames) is read into line buffer 1010b, can be stored in frame buffer 1020.In one embodiment, one piece of random access memory (RAM) that can be provided by any suitable parts of infrared imaging module 100 and/or host apparatus 102 realizes line buffer 1010a-b and frame buffer 1020.

Refer again to Fig. 8, picture frame 802e can be sent to automatic gain compensation block 828, it is further processed picture frame 802e, the result images frame 830 that can use as required to provide host apparatus 102.

Fig. 8 further illustrates the various operations for determining as discussed performed by row and column FPN item and NUC item.In one embodiment, these operations can use picture frame 802e as shown in Figure 8.Because carried out time-domain filtering to picture frame 802e, so at least some noise in time domain can be eliminated, thus can not casual impact to the determination of row and column FPN item 824 and 820 and NUC item 817.In another embodiment, can use not through the picture frame 802 of time-domain filtering.

In fig. 8, the block 510,515 of Fig. 5 is together with 520 expressions of concentrating.As discussed, can event be started in response to various NUC step and optionally start based on various standard or condition and perform NUC step.Also as discussed, according to based drive method (block 525,535 and 540) or NUC step can be performed based on the method defocused (block 530), to provide fuzzy picture frame (block 545).Fig. 8 further illustrates the various extra blocks 550,552,555,560,565,570,571,572,573 and 575 about Fig. 5 previously discussed.

As shown in Figure 8, row and column FPN item 824 and 820 and NUC item 817 can be determined, and apply above-mentioned item in an iterative manner, determine the item upgraded to make to use the picture frame 802 having applied first preceding paragraph.As a result, the institute of Fig. 8 can repeatedly upgrade in steps, and apply these with the noise in the picture frame 830 reducing host apparatus 102 continuously and will use.

Refer again to Figure 10, it illustrates the detailed implementation detail of various pieces relevant with streamline 800 in Fig. 5 and Fig. 8.Such as, block 525,535 and 540 is shown as the regular frame rate operation of the picture frame 802 to be received by streamline 800.In the embodiment shown in Figure 10, the decision made at block 525 is expressed as and determines rhombus (decisiondiamond), it is for determining whether Given Graph picture frame 802 changes fully, thus can think if picture frame is joined in other picture frames, this picture frame will strengthen fuzzy, therefore this picture frame is carried out adding up (in this embodiment, representing block 535 by arrow) and average (block 540).

Equally in Fig. 10, to be shown as with renewal rate operation, in this example embodiment, due to the average treatment performed at block 540 to the determination (block 550) of row FPN item 820, this renewal rate is 1/32 of sensor frame rate (such as, regular frame rate).Other renewal rates can be used in other embodiments.Although Figure 10 only identifies row FPN item 820, can in an identical manner, with the frame rate reduced to realize row FPN item 824.

Figure 10 also show the detailed implementation detail relevant with the NUC determining step of block 570.With regard to this respect, fuzzy graph picture frame can be read into line buffer 1030 (block RAM such as, provided by any suitable parts of infrared imaging module 100 and/or host apparatus 102 realizes).The flat field correction technology 700 of Fig. 7 can be performed to fuzzy graph picture frame.

In view of content of the present disclosure, should be understood that, technology described herein can be used for eliminating various types of FPN (such as, comprising the FPN of very high-amplitude), such as, and the row and column FPN of space correlation and space-independent FPN.

Other embodiments are also admissible.Such as, in one embodiment, the renewal rate of row and column FPN item and/or NUC item can be inversely proportional to the fuzzy estimate amount in fuzzy graph picture frame, and/or is inversely proportional to the size of local contrast value (such as, in the local contrast value that block 560 is determined).

In various embodiments, the technology of description is better than traditional noise compensation technology based on shutter.Such as, by using the step without shutter, do not need to arrange shutter (such as, as shutter 105), thus can reduced in size, weight, cost and mechanical complexity.If do not need the operation shutter of machinery, the power supply and maximum voltage that are supplied to infrared imaging module 100 or produced by infrared imaging module 100 also can be reduced.By being removed by the shutter as potential trouble spot, reliability will be improved.The potential image that step without shutter also eliminates caused by the temporary jam of the scene by shutter imaging interrupts.

Meanwhile, by using the fuzzy graph picture frame of catching from real scene (not being the even scene that shutter provides) to correct noise wittingly, noise compensation can be carried out by the picture frame similar with expecting those real scenes of imaging to radiation level.This can improve precision according to the determined noise compensation item of the technology of various description and efficiency.

As discussed, in various embodiments, infrared imaging module 100 can be configured to work at lower voltages.Especially, by being configured to work under low-power consumption and/or realize infrared imaging module 100 according to the circuit of other parameter work, other parameters described allow infrared imaging module 100 easily and effectively to realize in various types of host apparatus 102 (such as, mobile device and other devices).

Such as, Figure 12 shows the block diagram comprising another implementation of the infrared sensor package 128 of infrared sensor 132 and low-dropout regulator (LDO) 1220 according to disclosure embodiment.As shown in the figure, Figure 12 also show various parts 1202,1204,1205,1206,1208 and 1210, can to realize these parts with the previously described mode identical or similar about the corresponding parts of Fig. 4.Figure 12 also show bias voltage correction circuit 1212, and it can be used for adjusting (such as, with compensation temperature change, self-heating and/or other factors) one or more bias voltage being supplied to infrared sensor 132.

In some embodiments, LDO1220 can be set to a part (such as, be positioned on identical chip and/or wafer-class encapsulation is ROIC) for infrared sensor package 128.Such as, LDO1220 can be set to a part of the FPA with infrared sensor package 128.As discussed, this realization can reduce the power supply noise be incorporated in infrared sensor package 128, thus provides the PSRR of improvement.In addition, by utilizing ROIC to realize LDO, less die area can be consumed, and need less separation matrix (or chip).

LDO1220 receives by feed line 1232 input voltage that power supply 1230 provides.LDO1220 provides output voltage by feed line 1222 to the various parts of infrared sensor package 128.With regard to this respect, according to such as it can be used as that entirety is incorporated to this paper, that on Dec 9th, 2013 proposes application number to be 14/101 by way of reference, the various technology that the U.S. Patent application of 245 describes, LDO1220 can in response to the single input voltage received from power supply 1230, and all parts to infrared sensor package 128 provides substantially the same regulation output voltage.

Such as, in some embodiments, power supply 1230 can provide from about 2.8v to the input voltage of about 11v scope (such as, be about 2.8v in one embodiment), and LDO1220 can provide the output voltage from about 1.5v to about 2.8v scope (such as, be approximately in various embodiments 2.8,2.5,2.4 and/or more low-voltage).With regard to this respect, no matter power supply 1230 is to provide the conventional voltage scope of about 9v to about 11v, and be also to provide low-voltage (such as, about 2.8v), LDO1220 can be used for providing constant regulation output voltage.Therefore, although provide multiple voltage scope for input and output voltage, can be expected that, no matter how input voltage changes, and the output voltage of LDO1220 will remain unchanged.

Compared with the conventional power source for FPA, part LDO1220 being embodied as infrared sensor package 128 has lot of advantages.Such as, traditional FPA depends on multiple power supply usually, and each in described multiple power supply is discerptible powers to FPA, and all parts being distributed in FPA separated.By being regulated single supply 1230 by LDO1220, the discriminable all parts being supplied to the infrared sensor package 128 of (such as, to reduce possible noise) low-complexity of suitable voltage.Even if the input voltage from power supply 1230 changes (such as, if due to battery or make input voltage increase or reduce for the charging of the device of the other types of power supply 1230 or electric discharge), the use of LDO1220 also makes infrared sensor package 128 still can work in a constant manner.

The various parts of the infrared sensor package 128 shown in Figure 12 also can be embodied as at the lower operating at voltages of the voltage used than conventional apparatus.Such as, as discussed, LDO1220 can be embodied as provides low-voltage (such as, about 2.5v).This and the multiple high voltages be generally used for as traditional FPA powers define striking contrast, and described multiple high voltage is such as: for the voltage of the about 3.3v to about 5v for supplying digital circuits; For the voltage of about 3.3v of powering for mimic channel; And for the voltage of the about 9v to about 11v for load supplying.Of the same race, in some embodiments, the use of LDO1220 can reduce or eliminate the needs to the independent negative reference voltage being supplied to infrared sensor package 128.

With reference to Figure 13, other aspects of the low voltage operating of infrared sensor package 128 can be understood further.Figure 13 shows the circuit diagram of a part for the infrared sensor package 128 according to Figure 12 of disclosure embodiment.Especially, Figure 13 shows the miscellaneous part (such as, parts 1326,1330,1332,1334,1336,1338 and 1341) of the bias voltage correction circuit 1212 being connected to LDO1220 and infrared sensor 132.Such as, according to embodiment of the present disclosure, bias voltage correction circuit 1212 can be used for compensating the change depending on temperature in bias voltage.Be 7,679 by reference to the patent No., 048, date of publication is the similar parts indicated in the United States Patent (USP) in March 16 in 2010, the operation of these other annexes can be understood further, it can be used as entirety to be herein incorporated by way of reference.Can be also 6,812 according to the patent No., 465, date of publication be the various parts that indicate in the United States Patent (USP) on November 2nd, 2004 to realize infrared sensor package 128, it can be used as entirety to be herein incorporated by way of reference.

In various embodiments, all or part of bias voltage correction circuit 1212 can realize on integral array basis as shown in fig. 13 that (such as, for concentrating all infrared sensors 132 in an array).In other embodiments, all or part of bias voltage correction circuit 1212 (such as, each sensor 132 being copied in whole or in part) can be realized on single-sensor basis.In some embodiments, the bias voltage correction circuit 1212 of Figure 13 and miscellaneous part can be embodied as a part of ROIC1202.

As shown in figure 13, LDO1220 provides load voltage Vload to the bias voltage correction circuit 1212 along in feed line 1222.As discussed, in some embodiments, Vload can be approximately 2.5v, and in contrast, the size that can be used as the load voltage in traditional infrared imaging device is approximately the higher voltage of 9v to about 11v.

Based on Vload, bias voltage correction circuit 1212 provides sensor bias voltage Vbolo at node 1360.Vbolo is distributed to one or more infrared sensor 132 by the on-off circuit 1370 (such as, being represented by the dotted line in Figure 13) be applicable to.In some instances, can be 6,812,465 and 7,679 according to the patent No. quoted before this paper, the suitable parts indicated in the patent of 048 be to realize on-off circuit 1370.

Each infrared sensor 132 include by on-off circuit 1370 receive Vbolo node 1350 and can another node 1352 of ground connection, substrate and/or negative reference voltage.In some embodiments, the voltage at node 1360 place is substantially identical with the Vbolo at node 1350 place.In other embodiments, adjustable at the voltage at node 1360 place, to compensate the possible pressure drop relevant with on-off circuit 1370 and/or other factors.

The voltage that the voltage that usually uses than traditional infrared sensor bias voltage can be utilized lower is to realize Vbolo.In one embodiment, Vbolo can from about 0.2v to the scope of about 0.7v.In another embodiment, Vbolo can in the scope of about 0.4v to about 0.6v.In another embodiment, Vbolo is approximately 0.5v.By contrast, the normally used bias voltage of traditional infrared sensor is approximately 1v.

Compared with traditional infreared imaging device, make infrared sensor package 128 can have significantly reduced power consumption according to infrared sensor 132 of the present disclosure compared with the use of low bias voltage.Especially, the power consumption of each infrared sensor 132 is with square minimizing of bias voltage.Therefore, the reduction (such as, dropping to 0.5v from 1.0v) of voltage provides the reduction of significant power consumption, particularly when the reduction of described voltage is applied to the multiple infrared sensor 132 in infrared array sensor.The reduction of this power also can cause the minimizing of the self-heating of infrared array sensor 128.

Embodiment there is provided various technology for reducing the noise effect in the picture frame provided by the infreared imaging device being operated in low-voltage according to of the present disclosure other.With regard to this respect, when infrared sensor package 128 is with described low voltage operating, if do not corrected noise, self-heating and/or other phenomenons, in the picture frame that described noise, self-heating and/or other phenomenons can provide at infrared sensor package 128, become more obvious.

Such as, with reference to Figure 13, when LDO1220 remains on low-voltage Vload in a manner described herein, Vbolo also will remain on its corresponding low-voltage, and can reduce the relative size of its output signal.Therefore, noise, self-heating and/or other phenomenons can produce larger impact to the less output signal read from infrared sensor 132, thus cause the change (such as, mistake) of output signal.If do not corrected, these changes may show as the noise in picture frame.In addition, although low voltage operating can reduce some phenomenon (such as, self-heating) total number, but the error source that less output signal can make to remain (such as, residual self-heating) produces out-of-proportion impact to output signal during low voltage operating.

In order to compensate this phenomenon, various array sizes, frame per second and/or frame averaging can be utilized to realize infrared sensor package 128, infrared imaging module 100 and/or host apparatus 102.Such as, as discussed, various different array sizes can be considered for infrared sensor 132.In some embodiments, the infrared sensor 132 of the array sizes of scope from 32 × 32 to 160 × 120 can be utilized to realize infrared sensor 132.The array sizes of other examples comprises 80 × 64,80 × 60,64 × 64 and 64 × 32.Any desired size can be used.

Advantageously, when utilizing this relatively little array sizes to realize infrared sensor package 128, described infrared sensor package 128 without the need under carrying out more cataclysmal situation to ROIC and interlock circuit, can provide picture frame with relatively high frame per second.Such as, in some embodiments, the scope of frame per second can from about 120Hz to about 480Hz.

In some embodiments, array sizes and frame per second can relative to each other between increase and decrease (such as, with inversely proportional mode or other modes), to make larger array be embodied as, there is lower frame per second, and less array is embodied as and has higher frame per second.Such as, in one example in which, the array of 160 × 120 can provide the frame per second being approximately 120Hz.In another embodiment, the array of 80 × 60 can provide the higher frame per second being approximately 240Hz accordingly.Other frame per second are also admissible.

By array sizes and frame per second relative to each other between increase and decrease, no matter actual FPA size or frame per second be how many, and the row of FPA and/or the specific reading of row regularly can remain unchanged.In one embodiment, reading timing can be approximately every row or column 63 microsecond.

As the discussion before about Fig. 8, the picture frame that infrared sensor 132 is caught can be supplied to frame averager 804, described frame averager 804 asks the integration of multiple picture frame to have low frame per second (such as to provide, about 30Hz, approximately 60Hz or other frame per second) and the picture frame 802 (picture frame such as, after process) of signal to noise ratio (S/N ratio) of improvement.Especially, by being averaged to the high frame rate image frame provided by relatively little FPA, the picture noise produced effectively on average can being fallen and/or reduce significantly in picture frame 802 due to low voltage operating.Therefore, infrared sensor package 128 can be operated in the relatively low voltage provided by LDO1220 as discussed, and after frame averager 804 processes the picture frame 802 produced, infrared sensor package 128 can not be subject to the impact of extra noise in the picture frame 802 of described generation and relevant spinoff.

Other embodiments are also admissible.Such as, although show the single array of infrared sensor 132, can be expected that, multiple this array can be used together to provide the picture frame of high-resolution (such as, a scene can imaging on multiple this array).This array can be arranged on multiple infrared sensor package 128 and/or be arranged in infrared sensor package 128 of the same race.As described, each this array all can be operated in low-voltage, and also can be the relevant ROIC circuit of each this array configurations, to make the frame per second work that each array still can be relatively high.Shared or dedicated frame averager 804 can be averaged to the high frame rate image frame provided by this array, to reduce and/or to eliminate the noise relevant to low voltage operating.Therefore, still high-resolution Thermo-imaging system can be obtained when being operated in low-voltage.

In various embodiments, infrared sensor package 128 can be embodied as suitable size, can use together with the socket 104 (such as, for the socket of mobile device) of little form factor to make infrared imaging module 100.Such as, in some embodiments, infrared sensor package 128 can be embodied as the chip size that scope is about 4.0mm × approximately 4.0mm to about 5.5mm × about 5.5mm (such as, in one embodiment, about 4.0mm × approximately 5.5mm).Infrared sensor package 128 can be embodied as this size or other suitable sizes, to make it possible to use together with the socket 104 being embodied as various sizes, the size of described socket 104 is such as: 8.5mm × 8.5mm, 8.5mm × 5.9mm, 6.0mm × 6.0mm, 5.5mm × 5.5mm, 4.5mm × 4.5mm and/or other jack sizes, such as, as it can be used as by way of reference entirety be incorporated to herein, application number that June 10 in 2011 proposes is those sizes shown in the U.S. Provisional Patent Application table 1 of 61/495,873.

Power system can be used for such as producing electricity and/or mechanical force usually.In some embodiments, power system can comprise the turbogenerator that burning drives generator, photovoltaic solar system and underground heat, sun power or current to drive.In other embodiment, power system can comprise the mechanical force generator that burning, electricity or current drive.Such as, power system can be used for the vehicles driving any type, such as aircraft, ship, train or general-utility car, or can be used for the machine tool driving hand mower or any other type.Produce the power system of mechanical force by driving shaft, electromagnetic actuators or other interface agencies and the machine interface that pass through some.

For monitoring the system of power system, all those systems as described herein, contribute to utilizing power system to carry out various application.Such as, surveillance can be used for for efficiency, performance, reliability, convenience (such as, scheduling) or these the operation of combination in any monitoring and controlling power system exactly, and As time goes on, according to instant needs, the emphasis in power system operation can change.

Especially, the supervisory system comprising various infrared imaging module 100 described herein has many advantages compared with general surveillance system.Such as, infrared imaging module 100 can be configured to when being physically attached to the active parts of power system, with when very high details and high-precision real or close to the temperature and the state that monitor power system in real time.This makes power system owing to not detecting (such as; supervision due to inaccurate or free delay) potential motive system maintenance issues and/or tail off stop time of safeguarding due to surveillance, safeguarding can when needing occur when interrupting the work of power system.Each infrared imaging module 100 can be configured to make multiple parts in its visual field (FOV) by aiming at multiple parts, monitors multiple parts of power system simultaneously.

In some embodiments, infrared imaging module 100 can be configured to such as detect polytype gas (such as, carbon monoxide, fuel fumes) thermal deviation (such as, exceptional part temperature), the density/local density of gas, leak of liquid, parts thermal capacity and pieces conduct heat rate, and when not suffering the eurypalynous heat that exists in conventional sensors or other sensor delay, can do like this.And infrared imaging module 100 can be configured to As time goes on record above-mentioned test item and the subtle change detected in the part temperatures detected or state.Therefore, surveillance described herein is easier than general surveillance system to be realized, and by providing the more detailed data that obtain sooner than conventional sensors and for drawing more best performance (such as, passing through FEEDBACK CONTROL) from power system.

With reference now to Figure 14, Figure 14, show the block diagram of the surveillance 1400 of the parts for monitoring power system 1430 according to disclosure embodiment.System 1400 can comprise one or more infrared imaging module 1402a, 1402b, 1402c, processor 1410, storer 1412, communication module 1414, display 1416, control module 1432, load module 1434 and other surveillance parts 1440.In a suitable case, the element of system 1400 can to realize with the same or similar mode of the counter element of Fig. 1 host apparatus 102, and can be configured to perform various NUC process as described herein and other processes.

In some embodiments, each infrared imaging module 1402a, 1402b, 1402c can be the little form factor thermal camera or imaging device that realize according to various embodiment disclosed herein.Such as, each infrared imaging module 1402a, 1402b, 1402c can comprise FPA that is that realize according to various embodiment disclosed herein or that otherwise realize in a suitable case.In addition, in some embodiments, each infrared imaging module 1402a, 1402b, 1402c can realize with one or more non-thermal video camera of substantially staying altogether, and are oriented so that the FOV of at least part of overlapping corresponding infrared imaging module of the FOV of non-thermal video camera.In such embodiment, the image that infrared imaging module 1402a, 1402b, 1402c catch and/or the image that view data can be caught with one or more non-thermal video camera as described herein and/or view data superpose, merge, mix or otherwise combine (such as, by processor 1410).Each infrared imaging module 1402a, 1402b, 1402c can be configured to catch, process and/or manage infrared image, and infrared image comprises the heat picture of a part for power system 1430.

Infrared imaging module 1402a, 1402b, 1402c can be installed in power system 1430 everywhere, with make the parts of power system 1430 at least partially in the visual field of at least one infrared imaging module 1402a, 1402b, 1402c.Such as, as by what further describe about Figure 16, infrared imaging module 1402a, 1402b, 1402c can be used in vehicle environmental or other environment.More generally, infrared imaging module 1402a, 1402b, 1402c can be placed/be mounted to any parts observing power system 1430.Such as, hinder and expect that the power system components observed can be moved, eliminates or be equipped with infrared transmission material, arrive specific infrared imaging module to make the infrared radiation from power system 1430 expectation part.

In some embodiments, each infrared imaging module 1402a, 1402b, 1402c can comprise respective optical element 1404a, 1404b, 1404c (such as, infrared transmission lens, infrared transmitting prisms, ir reflector, infrared optical fiber), they are the infrared radiation guiding infrared imaging module 1402a of ultromotivity system 1430 part, the FPA of 1402b, 1402c in the future.When the position installation infrared image-forming module difficulty limiting FOV specific or can not time, such optical element can be used.Such as, flexible optical fibre cable can be used for infrared radiation from the infrared imaging module be sent to by circuit in seal member (such as, water cooling system circuit) on the shell being arranged on power system 1430.Such optical element also can be used for the FOV suitably limiting or change infrared imaging module.Can selectively provide switchable FOV (such as, by the infrared imaging module of correspondence and or processor 1410 can select), to provide the distant place replaced and the close-up illustration of an element portion of power system 1430.

By infrared imaging module 1402a, 1402b, 1402c catch, the infrared image that processes and otherwise manage can be standardized infrared image (such as, heat picture) on radiation detection.Form the dsc data (such as, absolute temperature) that the pixel of catching image can comprise calibration.As above about Fig. 1 infrared imaging module 100 as described in, each infrared imaging module 1402a, 1402b, 1402c and/or associated components can use suitable technology to calibrate, and are the heat picture of correct calibration with the image making infrared imaging module catch.In some embodiments, suitable calibration process periodically performs by each infrared imaging module and/or processor 1410, keeps calibrating accurately to make infrared imaging module and the heat picture of catching thereof.

Processor 1410 can be implemented as any suitable treating apparatus as described in the processor 195 about Fig. 1.In some embodiments, processor 1410 can be other conventional processors and the part controlling electronic installation, maybe can realize with other conventional processors and control electronic installation, this control electronic installation can be installed together with power system 1430 or with the machine driven by power system 1430.Such as, the vehicles driven by power system 1430 can comprise for control & monitor power system or driven the various mechanically operated processor of the vehicles or control electronic installation, the processor for amusement and vehicle information system, the processor for navigational system and/or the processor for remote diagnosis system, and wherein any one all can be used for the part realizing whole processor 1410 or processor 1410.In other embodiments, processor 1410 can control electronic installation with processor and any power system components interface relevant with such processor to such other and communicates.In some embodiments, processor 1410 can be configured to such as control, monitor power system 1430 and/or communicate with power system 1430, and in some embodiments, arrange according to the plan of user, technician or arranged by factory default and do like this.This plan can be determined such as whether to provide maintenance instruction to user, or determine when to enable one or more power system components.

Processor 1410 can be configured to the miscellaneous part interface of system 1400 and communicate, and to perform Method and Process described herein, the one or more parts comprised to power system 1430 provide control signal.Processor 1410 can be configured to the heat picture at least partially of the parts receiving the power system 1430 of being caught by infrared imaging module, perform thermal image processing operation as further described herein, and from heat picture, extract data, with a part for such as determining means, parts or the state of miscellaneous part of power system 1430.The state that processor 1410 also can be configured to editor, the data analyzing or otherwise process extraction, heat picture and determine, to produce the monitor message about the parts of power system 1430, such as about the monitor message of the state of the detection of the parts of power system 1430.

Such as, according to the heat picture of the calibration that one or more infrared imaging module 1402a, 1402b, 1402c provide, processor 1410 can the total temperature of determining means or the temperature of parts specific part.Processor 1410 can produce monitor message based on the temperature determined, this monitor message comprises such as temperature reading.Processor 1410 can the temperature of determining means further whether within the scope of normal working temperature, and produce and comprise the notice of indicated temperature outside safe range or the monitor message of alarm.

In another example, processor 1410 can perform the operation of various thermal image processing and heat picture analysis, to obtain Temperature Distribution and the variance distribution plan of power system components to the heat picture of a power system components part.The distribution plan obtained can associate with the distribution plan of abnormality and/or mate with test example as the water blocked or leak or oil cooling system by processor 1410, out of order water or oil pump, the combustion chamber charge leaked or vent valve, the air/fuel mixture of non-optimal, out of order bearing, overheated combustion engine parts or motor component, exhaust system is leaked, accumulation of debris is on component inner surface, out of order high pressure (HV) battery or other states further described herein.

In another embodiment again, processor 1410 can to combustion engine (such as, comprise inlet manifold, cylinder head, exhaust manifold, crankcase, turbine and aftercooler), motor (such as, comprise power switch circuit, rotor, motor winding, bearing) and/or the heat picture of other power system components perform the operation of various thermal image processing and heat picture analysis, with fracture detection, leakage, foreign matter, distortion and other abnormality.Based on detection, processor 1410 can produce the alarm of description or the monitor message of other visions or sound notification that comprise instruction abnormality and/or abnormality.

In some embodiments, processor 1410 can be configured to use suitable method and algorithm to convert the heat picture of the part of power system 1430 to user's visual image (such as, Thermogram).Such as, pixel that is that the heat record data be included in heat picture can be converted into gray scale or color range is to construct the image that can observe over the display.In some embodiments, heat record data and/or Thermogram can combine (such as with the image of being caught by one or more non-thermal video camera and/or view data, by superposition, fusion, mixing and/or other combined methods), to build user's visual image.User's visual image selectively can comprise legend or the scale of the approximate temperature of instruction respective pixel color and/or intensity.Such user's visual image, if presented (such as, on display 1416) over the display, then can be used for confirming or the abnormality that detects of understanding system 1400 better.The monitor message that processor 1410 produces can comprise such user's visual image.

Storer 1412 can comprise the one or more memory storages storing data and information (comprising heat picture and monitor message).Store its device and can comprise the various types of storeies stored for heat picture and other information, it comprises volatibility and non-volatile memory device, such as RAM (random access memory), ROM (ROM (read-only memory)), EEPROM (electricallyerasable ROM (EEROM)), flash memory or disc driver.In one embodiment, in order to consult and diagnose further the state of the parts monitored by system 1400, be stored in heat picture in storage arrangement and monitor message can be retrieved (such as, using suitable reader and/or diagnostic tool by technician).In some embodiments, processor 1410 can be configured to perform the software instruction be stored on storer 1412, to perform various method, process or operation by mode described herein.

Display 1416 can be configured to the monitor message presenting, indicate or otherwise pass on processor 1410 to produce.In one embodiment, display 1416 can implement the various of conventional annunciators, measurement instrument and the warning lamp be similar on power system instrument panel and light icon, symbol, indicator and/or analog measuring instrument.The icon lighted, symbol and/or indicator can indicate the one or more notice relevant to monitor message or alarm.The icon lighted, symbol or indicator also number-letter relation table digital display panel (such as, segmentation LED panel) supplement, to show the letter and number representing other monitor messages, and the description of the abnormality of such as temperature reading, detection or classification etc.

In other embodiments, display 1416 available electron display screen realizes, the video display that such as liquid crystal display (LCD), cathode-ray tube (CRT) (CRT) or various other types are usually known and monitor, comprise touch-sensitive display.Display 1416 can be suitable for presenting the visual heat picture of user of being come from the heat picture conversion that infrared imaging module is caught by processor 1410.Can consider, can by routine information display screen, those information display screens of finding in the such as usual vehicles driven in power system 1430 (such as, for airborne entertainment system interface, show navigator information, show rear view camera image and show the vehicle information of various other types) are as display 1416.

Communication module 1414 can be configured to communication between all parts of promotion system 1400 and interface.Such as, the element of such as infrared imaging module 1402a, 1402b, 1402c, display 1416, control module 1432, load module 1434 and/or other surveillance parts 1440 can send data by communication module 1414 to processor 1410 and receive data from processor 1410, wired and/or wireless connections between the various parts of communication module 1414 ALARA Principle (such as, by the wireless communication protocol of proprietary RF link, proprietary infrared link and/or standard, such as IEEE802.11WiFi standard and bluetooth).Such wireless connections enable infrared imaging module be arranged on the place being inconvenient to provide wired connection, for example, are such as arranged on the parts of rotation or otherwise movement.

Communication module 1414 can be configured to enable the parts of system 1400 and power system 1430 or other existing electronic units of machine of being driven by power system 1430 to communicate and interface further.Such as, processor 1410 communicates with electronic unit with other existing sensors with entertainment systems, vehicle navigation system by communication module 1414 and power system electronic control unit (ECU), vehicle information.With regard to this respect, communication module 1414 can support various interfaces, protocol and standard for networking, such as controller zone network (CAN) bus, vehicle area network (VAN) standard, Local Interconnect Network (LIN) bus, towards system transfers (MOST) network of media or ISO11738 (or ISO bus) standard.In addition, communication module can be configured to the control signal using these interfaces and agreement sending processor 1410 generation.

In some embodiments, system 1400 can comprise many communication modules 1414 of the various application of the system of being suitable for 1400 in various types of power system.In other embodiments, communication module 1414 can be incorporated into the miscellaneous part of various system 1400 or be implemented as the part of various miscellaneous parts for system 1400.Such as, infrared imaging module 1402a, 1402b, 1402c, processor 1410 and display 1416 is each comprises subassembly, this subassembly can be configured to the operation of executive communication module 1414, and when there is no independent communication module 1414, can be communicated with one another by wired and/or wireless connections.

Control module 1432 can comprise the one or more modules being configured to provide direct control signal to power system 1430, such as, without the need to rely on communication module 1414 and relevant agreement and with the control electronic device communications of power system 1430.Like this, control module 1432 can be can with one or more parts of power system 1430 (such as inlet manifold or turbine) direct interface such as directly to control these operation of components (such as, air/fuel mix, or supercharging) any module.In some embodiments, control module 1432 can be configured to make system 1400 that the control electronic installation of power system 1430 can be made invalid, or the additional control providing the control electronic installation of power system 1430 not provide.

In a similar fashion, load module 1434 can comprise the one or more modules being configured to receive direct supervisory signal from power system 1430, such as, without the need to rely on communication module 1414 and relevant agreement and with the control electronic device communications of power system 1430.Like this, load module 1434 can be can with one or more parts of power system 1430 (such as inlet manifold or turbine) direct interface with the such as direct any module receiving supervisory signal (such as, indicating the signal of air/fuel mixture or boost pressure) from these parts.In some embodiments, load module 1434 can be configured to make system 1400 can ignore the supervisory signal provided by the control electronic installation of power system 1430, or the additional supervisory signal providing the control electronic installation of power system 1430 not provide.In some embodiments, control module 1432 and load module 1434 can be configured to make system 1400 can provide closed loop option, for monitoring and controlling power system 1430, one or more parts of separating with the control electronic installation of power system 1430.

In some embodiments, other system parts 1440 can comprise other sensors, such as temperature sensor (such as, thermopair, infrared thermometer), humidity sensor, conventional digital video camera, electric transducer (such as, voltage/current/ohmmeter) and/or pressure transducer (such as, rain glass).Sensor is temperature, humidity or pressure transducer such as, by processor 1410 for compensate for environmental conditions, thus can obtain the heat picture of various parts of power system 1430 and the analysis more accurately of the state of derivation.Sensor such as conventional digital video camera and electric transducer can provide and can be utilized to obtain the heat picture of various parts of power system 1430 and the reference data points analyzed more accurately of the state of derivation and/or background information by processor 1410.The further example of background information can be provided with reference to Figure 18 below.

Other system parts 1440 also can comprise any other device useful to the various application of system 1400.In some embodiments, other system parts 1440 can comprise the monitor message that can be used for producing based on processor 1410 send audible alarm or notice clock, there is loudspeaker for generation of the interlock circuit of tone or other devices.In further embodiment, other system parts 1440 can comprise accept such as power system 1430 expected performance, system 1400 notice arrange, the user interface of user's input of external sensor data or background information.

In various embodiments, may be combined with and/or realize or do not combine and/or realize one or more parts of system 1400 according to application demand.Such as, processor 1410 can combine with any one in infrared imaging module 1402a, 1402b, 1402c, storer 1412, display 1416 and/or communication module 1414.In another example, processor 1410 can combine with any one in outer imaging sensor 1402a, 1402b, 1402c, wherein only some operation of processor 1410 is performed by the circuit (such as, processor, logical unit, microprocessor, microcontroller etc.) in arbitrary infrared imaging module.

Therefore, can system 1400 be arranged in power system 1430, be arranged in power system 1430 or be otherwise attached to power system 1430, to provide on the machine of the state of various power system components with real-time supervision, various power system components is combustion engine, motor, HV battery, HV generator, shell, water/oil cooling system, gearing etc. such as.Such as, when for provide power to the vehicles and detect deformity that crackle formed, carbon monoxide or fuel leakage and higher than normal temperature time, system 1400 can be used for carrying out the state of power system 1430 on machine with real-time supervision.It is also conceivable to, system 1400 can be adjusted or be revised, to monitor the various miscellaneous parts of the machine driven by power system 1430.

Figure 15 shows the nonexcludability group of the parts of the power system 1430 that can be monitored by system 1400.Such as, what power system 1430 can comprise combustion engine 1510, motor 1540 and be configured to support the operation of internal combustion engine 1510 or motor 1540 and make the mechanical dominance of power system 1430 to meet in other power system components of application-specific is one or more.Although Figure 15 does not clearly illustrate, but power system 1430 can add or optionally comprise the Power Generation Unit of other types attachedly, such as, the thermo-motor such as driven by thermal gradient and bank device, with the generator driven by chemistry, photon and/or external mechanical force (such as, wind, tide, dynamical motion).Figure 15 should not be understood to exclusiveness set or the quantity that can be similar to the system monitoring of system 1400 and/or the power system components of control, and is only any grouping that the object described and work out provides the power system components of Figure 15.In addition, in some embodiments, except other power system components that Figure 15 does not clearly illustrate, power system can comprise the one or more of each parts in Figure 15.

Combustion engine 1510 can be such as the combustion engine of gasoline, diesel oil, rock gas, propane or any other type, and can comprise and be configured to make the operation of combustion engine 1510 and power system 1430 meet multiple parts of application-specific.Such as, inlet manifold 1512, it, for combining air and fuel and potpourri being transported to the firing chamber of combustion engine 1510, can be configured to adjust the fuel of each firing chamber being transported to combustion engine 1510 or the amount of air or potpourri according to the power stage expected and/or efficiency.Optimum apjustment can be depending on the temperature of fuel, air or air/fuel mixture and the real time temperature of firing chamber that are transported to inlet manifold.

Turbosupercharger (" turbine ") 1514, its can with inlet manifold 1512 compact siro spinning technology, can be configured to use Exhaust Gas to drive increase (such as, promote) be transported to the air of inlet manifold 1512 and/or the turbine of air/fuel mixture pressure, and its usefulness and state are by detecting its absolute temperature and monitoring for the thermal gradient between Exhaust Gas and the entrance and exit of conveying gas.In some embodiments, turbine 1514 can comprise waste gate, variable internal geometry and/or actuator reduction valve are transported to inlet manifold 1512 boosting with adjustment.In further embodiment, power system 1430 can additionally or optionally comprise supercharger (supercharger), it also can promote the pressure of air and/or the air/fuel mixture being transported to inlet manifold 1512 (such as, adjusted similarly), but supercharger obtains boost power from the mechanical connection of the crank pulley 1528 with such as combustion engine 1510.

Aftercooler 1516, its also with inlet manifold compact siro spinning technology, can be the heat being configured to draw the air of the adiabatic compression of carrying from turbine 1514, to increase the usefulness of turbine 1514 and to increase the sealed heat-dissipating device formula device being transported to the compressed-air actuated density of inlet manifold 1512.Such as, usefulness and state that the chilled water of aftercooler 1516 and compressed-air actuated import and outlet temperature can indicate aftercooler 1516 is provided to.In some embodiments, the gas of heating that one or more additional aftercooler to be used by power system 1430 with cooling or produced can be placed on other ground of power system 1430.

Timing system 1518 can be for the machinery of the combustion period of combustion engine 1510 firing chamber timing and/or electric device, and the improper meeting of timing causes the parts abnormal temperature being everywhere couple to inlet manifold 1512, cylinder head 1520 and exhaust manifold 1522.In some embodiments, cylinder head 1520 can be used for the oil of seal lubrication cam and the valve for firing chamber.Cylinder head oil moving component that is usual and firing chamber is in thermal equilibrium, and cylinder head is usual and cylinder head oil heat is in balance, and therefore abnormal cylinder head temperature often indicates the abnormal operation of combustion engine 1510.

Exhaust manifold 1522 can be used for collect from one or more firing chamber Exhaust Gas and they are flowed to turbine 1514 and exhaust system 1568.Effluent air temp can show normal or malcombustion, gas admittance valve and the operation of vent valve, the operation of turbine 1514; and because exhaust manifold 1522 is usual and partial exhaust gas gas is in thermal equilibrium, so abnormal exhaust manifold temperature often indicates the misoperation of combustion engine 1510.

In the same way, the temperature of crankcase 1524, oil groove 1526, crank pulley 1528, alternator 1530, flywheel 1532 and other engine components 1534 can indicate the operation problem manifested of combustion engine 1510 and/or all parts itself.Other engine components 1534 can such as comprise such as following parts: oil filter, petrolift, be embedded in oil in combustion engine 1510 and/or water cooling pipeline, allow in combustion engine 1510 or by the bearing of the outstanding component movement of combustion engine 1510, hot melt between the parts of combustion engine 1510, compressor of air conditioner, and for making the various parts of the miscellaneous part of combustion engine 1510 and power system 1430, machine that power system 1430 drives or surrounding environment interface.

Motor 1540 can comprise and can be monitored and/or adjust to make the operation of motor 1540 and power system 1430 meet multiple parts of application-specific.Such as, power switch circuit 1542, it is used in special time and correct polarity electric power is transported to particular motor winding 1546, can be configured to the opportunity or the amplitude that adjust the power of conveying according to the power stage expected and/or efficiency.Optimum apjustment can be depending on the temperature of HV battery 1584.In addition, the temperature of power switch circuit 1542 can indicate the fault manifested before power switch circuit 1542 lost efficacy.

In a similar fashion, the temperature of rotor 1544, motor winding 1546, bearing 1548 and other motor components 1550 can indicate the operation problem that motor 1540 and/or all parts itself are manifesting.In addition, such temperature can indicate magnitude that is instantaneous on motor 1540 or time averaging mechanical load.Other motor components 1550 can comprise, such as, and such as following parts: ground terminal, motor housing, and for making the various parts of the miscellaneous part of motor 1540 and power system 1430, machine that power system 1430 drives or surrounding environment interface.

As mentioned above, Figure 15 also comprises many power system components, this power system components is configured to the operation of the Power Generation Unit supporting combustion engine, motor or any other type, and is configured to make power system 1430 can provide power to particular machine and/or application.Such as; shell 1560 can be sealing, ventilate, rigidity, shell that is flexible or any other type or mixed type; it is configured the part protecting, support, move or hold whole power system 1430 or power system 1430, is system 1400 in some embodiments.Shell 1560 can be such as metal, glass fibre, pottery, carbon fiber or any other material.In some embodiments, shell 1560 can be light with outside through aerodynamic design, or the structural member of some relative thin that bolt connects or weld together can be comprised.In further embodiment, shell 1560 can be formed to provide installation to the one or more infrared imaging modules at least partially for imaging power system 1430 parts.The part of shell 1560 can be monitored with test example as the weakness in shell 1560 or crackle, or detect the temperature being thermally connected to the miscellaneous part of shell 1560.

Fan 1562 can be used for any other parts of cooling combustion engine 1510, motor 1540, HV battery 1584, water-cooling system 1564, oil cooling system 1566 or power system 1430.The speed of fan 1562 can regulate according to the malfunction of the power stage of the temperature of such as associated power system unit, environment temperature, power system 1430 or fan 1562.

In some embodiments, the position of the part of water cooling system 1564 can near fan 1562.Water cooling system 1564 can comprising such as one or more heating radiator, water pipeline, water pump, for accessing flexible pipe or pipe, conventional thermostat and the nozzle of outside body of water, and can be used for any other parts of cooling combustion engine 1510, motor 1540 or power system 1430.In one embodiment, water cooling system 1564 can use with the aftercooler of sealing the gas using with cooling power system 1430 or produce jointly.In another embodiment, water cooling system 1564 can comprise the nozzle being configured to eject water to and carrying out direct refrigerating gas in relatively hot air-flow (such as, Exhaust Gas).In further embodiment, absolute and/or the differential temperature of the pairing entrance and exit of water cooling system 1564 can be used for efficiency and the running status of the water cooling system 1564 determined, and infers that heat is couple to the running status of the power system components of water cooling system 1564.

Similarly, oil cooling system 1566 can comprise such as one or more heating radiator, oil pipeline, oil pump and nozzle, and can be used for any other parts of cooling combustion engine 1510, motor 1540 or power system 1430.In some embodiments, absolute and/or the differential temperature of the pairing entrance and exit of oil cooling system 1566 can be used for efficiency and the running status of the oil cooling system 1566 determined, and infers that heat is couple to the running status of the power system components of oil cooling system 1566.

Exhaust system 1568 is attachable to exhaust manifold 1522, and comprises sound suppressor, catalytic converter, particulate scrubber and the piping system from exhaust manifold to exhausr port.The temperature of exhaust system 1568 can indicate normal in such as engine 1510 or abnormal burning, or the leakage of any part along exhaust system 1568.In some embodiments, in order to provide suitable back pressure and the performance of motivation of adjustment system 1430 to the miscellaneous part of such as exhaust manifold 1562, turbine 1514 or power system 1430, exhaust system 1568 can comprise the selectable unit of adjustment air-flow compression.In further embodiment, exhaust system 1568 can comprise the selectable unit of the air-flow providing heating to heating element, for one or more parts of heated power system 1430 or the machine that driven by power system 1430.

Exhaust system 1568 also can comprise such as aftercooler, to cool Exhaust Gas before other element of being discharged by Exhaust Gas or be transported to exhaust system 1568.In some embodiments, exhaust system 1568 can comprise the interface with water cooling system 1564, and this water cooling system 1564 comprises the one or more nozzles be directly sprayed onto by chilled water in Exhaust Gas that exhaust manifold 1522 carries.Such nozzle interface can be positioned at exhaust manifold 1522, exhaust system 1568 is inner, or forms a part for the interface between exhaust manifold 1522 and exhaust system 1568.

The operation problem that fuel system 1570, accessory battery 1572, the regularly temperature of electronic installation 1574 and conventional sensors electronic installation 1576 can indicate power system 1430 and/or all parts itself manifesting.Such as, the temperature of fuel system 1570 can indicate temperature and/or the density of the fuel being transported to combustion engine 1510, and can indicate the fuel ignition in imminent lack of fuel or fuel system.Timing electronic installation 1574 can be used for the opportunity of burning in such as electrical control combustion engine 1510 or signals the opportunity of burning in combustion engine 1510, and it can be connected with timing system 1518.Other aspects of the operation of the temperature of routine measurement and power system 1430 can be supplied to and control electronic installation 1594 and/or system 1400 by conventional sensors electronic installation 1576.

The temperature of gearing 1578, driving stem/axle 1580 and differentiator 1582 also can indicate the present operation problem of the machine of power system 1430, all parts itself and/or power system 1430 driving.Such as, the high temperature of gearing 1578 and/or differentiator 1582 can indicate be couple to power system 1430 via gearing 1578 and/or differentiator 1582 machine on the increase of load.

In some embodiments, high pressure (HV) battery 1584 can be used for the electric power such as providing storage via HV cable 1586 to motor 1540.HV generator 1590 can be used for such as producing for motor 1540 or for being stored in the electric power in HV battery 1584.In some embodiments, the electricity of storage or generation can be supplied to external system via high-tension cable 1586.Absolute temperature on high-tension battery 1584 or thermograde can indicate the life-span of the too high or high-tension battery 1584 of the fault, the charge or discharge speed that are such as manifesting.The operation problem that the temperature of HV cable 1586 and HV generator 1590 can indicate motor 1540 and/or all parts itself manifesting or fault.Ohmic load 1588 can be used for such as making HV battery 1584, HV generator 1590, and the Feedback of Power electric discharge provided by the external force on motor 1540.In some embodiments, ohmic load 1588 can adjust the resistance increasing or reduce it according to the discharge rate expected.The temperature of ohmic load 1588 can indicate the quantity of power of the ohmic load 1588 arrived by each component feed coupled that is single or combination.

Interface module 1592 can be used for making any parts of power system 1430 or parts group and external system (such as system 1400) interface.Therefore, do not need to communicate with control electronic installation 1594, via interface module 1592, system 1400 directly can control the operation of power system 1430.Interface module can comprise and is positioned at whole power system 1430 one or more modules everywhere, and can comprise servomechanism installation, actuator, sensor, electricity input, pneumatic input and other interfacings.Control electronic installation 1594 can be used for the operation of all or any subgroup controlling power system 1430 or control the machine of power system 1430 driving, and in some embodiments, control electronic installation 1594 and can be couple to interface module 1592.Control electronic installation 1594 can comprise one or more processor, storer and be configured to the communication module of the routine operation being convenient to power system 1430.Interface module 1592 and the temperature controlling electronic installation 1594 can indicate the electric fault, the environment that are manifesting overheated (such as, fire) or interface module 1592 and the exception that controls electronic installation 1594 use, such as control loop fluctuation excessively uses one or two module.

Other power system components 1596 can comprise such as following parts: the electric heater of the temperature of gentle power system 1430 in cold climate, the leveling system of at least some part of response external acceleration motivation of adjustment system 1430, environmentally state opens or closes the ventilating opening of shell 1560 and couples/remove the ventilation for enclosures mouth actuator coupling power system 1430, and for making the various parts of the miscellaneous part of the parts of power system 1430 and power system 1430, the machine driven by power system 1430 or surrounding environment interface.The temperature of other power system components 1596 can indicate and/or adjust the state of one or more parts of such as power system 1430.

Figure 16 shows the possible configuration of of system 1400 of the parts monitoring power system 1430.As shown in figure 16, system 1400 can be distributed on power system 1430, its shell 1560 and/or the equipment (such as the vehicles) that driven by power system 1430 whole on, and can comprise multiple infrared imaging module 1402a, 1402b, 1402c, wherein each infrared imaging module is arranged to the parts of observation power system 1430 at least partially.Infrared imaging module 1402a can be placed in the nearby components of power system 1430, and infrared imaging module 1402b can be arranged on the parts of power system 1430, and infrared imaging module 1402c can be arranged in the parts of power system 1430.

Such as, one in Figure 16 in infrared imaging module 1402a can be arranged to make a part for fuel tank 1570 and/or exhaust system 1568 in FOV1406a, as shown in figure 14.In another example, one in infrared imaging module 1402b can be installed to the parts of the power system 1430 in Figure 16 (such as, motor 1540) on, to make a part for engine 1510, water cooling system 1564, water cooling pipeline 1565, axle 1575 and/or exhaust system 1568 in FOV1406b, also as shown in figure 14.In further example, it is interior (such as that infrared imaging module 1402c can be installed in Figure 16 medium power system 1430 parts, in exhaust manifold, or in engine 1510), to make the interior section of engine 1510, turbine 1514, aftercooler 1516 and/or inlet manifold in inner FOV1406c, as shown in figure 14.

Water cooling system 1564 (such as, heating radiator) be couple to engine 1510 by water pipeline 1565, fuel tank 1570 is couple to engine 1510 by burning line 1571, accessory battery 1572 is couple to engine 1510 by battery terminal 1573, driving stem 1580 is couple to axle 1581 by least one differentiator 1582, and controls electronic installation 1594 and be couple to each power system components by controlling electronic installation circuit 1595.

In one embodiment, comprise infrared imaging module 1402a, the system 1400 of 1402b and 1402c can be configured to monitor water cooling system 1564, water pipeline 1565 and engine 1510, to determine whether have blocking, leakage or out of order water pump in water cooling system 1564.In addition, system 1400 can be configured to the temperature of the part monitoring water cooling system 1564, water pipeline 1565 and engine 1510, and such as use the control signal produced by the processor 1410 of system 1400 to adjust based on the adjustment of these temperature the handling capacity of water cooling system 1564, or when temperature indicates any other parts of engine 1510 or power system 1430 overheated, even turn off engine 1510.The control signal that processor 1410 produces such as can be transferred to power system 1430 via communication module 1414 and/or control module 1432, as shown in figure 14.

System 1400 can be configured in the mode similar with monitoring water cooling system 1564 to monitor the oil cooling system of power system 1430.The oil monitored and/or the temperature of water cooling system can indicate the failing bearings in such as engine 1510, or in the embodiment being therefore actively cooled and/or monitoring at motor 1540, the failing bearings in instruction motor 1540.In one embodiment, such as, when the temperature of at least one parts relevant to engine 1510 reach low temperature limit time, or according to plan, system 1400 can to crankcase heater energising with the temperature increasing engine 1510.Such as, when engine 1510 does not work, in order to make the miscellaneous part of engine 1510 and power system 1430 remain near the nominal operating temperature of engine 1510 before the operation of the plan of engine 1510, can be energized to crankcase heater.

In another embodiment, system 1400 can be configured to monitor the cylinder head of engine 1510, exhaust manifold or miscellaneous part, to determine whether firing chamber has gas admittance valve or the vent valve of leakage.Whether system 1400 can be configured to such as to use the control signal of at least one parts being transported to power system 1430 leaking and the valve of what type adjusts igniting opportunity or state (such as, opening or closing) in leakage based on being valve.

In further embodiment, system 1400 can be configured to such as directly by distinguishing fuel/air mixture content to the direct imaging of potpourri, or indirectly by monitoring that at least part of temperature of parts of power system 1430 monitors the air/fuel mixture of engine 1510, and adjust potpourri based on other temperature of the load in the temperature of surrounding air or pressure, power system 1430, the load sharing of motor 1540 and the parts of power system 1430 and/or state.In some embodiments, by communication module 1414 and/or the load module 1434 from power system 1430, user interface or other system acceptance signal, a part for monitor message (such as, ambient condition, load) can be supplied to the processor 1410 of system 1400.In addition, the air/fuel mixture that system 1400 can be configured to basis adjusts turbine wastegate and/or turbo boost pressure.

In further embodiment, system 1400 can be configured to the temperature monitoring such as HV battery 1584 excessive heat or local pyrexia, and adjusts charge or discharge or the charge or discharge rate of HV battery 1584, to alleviate the fault that may occur.

In one embodiment, system 1400 can be configured to control the ventilation activated of shell 1560 and/or the speed of fan 1562, to regulate the temperature of one or more parts of power system 1430.In addition, if power system 1430 comprises gearing and/or motor 1540, then system 1400 can be configured to such as adjust the output of ratio of gear, combustion engine power stage and/or motor power (output) according to the load in power system 1430.By thermal imaging and/or background information, power system 1400 can determine the load on any parts of power system 1430 or power system 1430, and can make adjustment with emphasized reliability (such as, between plan or the maintenance detected need, the time is long), efficiency (such as, for power demand export use fuel minimum), performance (such as, maximum available power) and/or convenience is (such as, automatic monitoring and control, a very long time seldom or does not at all have and user interactions).

Forward Figure 17 to now, Figure 17 shows the vehicle dashboard 1700 of the analog scale 1702 with system 1400 according to disclosure embodiment and display 1704.In some embodiments, display 1704 may correspond to the display 1416 in such as Figure 14, or the display of the equipment that can comprise power system 1430 and/or driven by power system 1430, this display is configured to show the monitor message transmitted by communication module 1414.

As shown in figure 17, conveniently user observes the vehicles, display 1704 can with the electronic display be placed in vehicle dashboard 1700 (such as, LCD screen, CRT screen or other suitable display, such as touch-sensitive screens) realize presenting monitor message that processor 1410 produces for being watched by the user of the vehicles easily.The shielding sectional drawing display of display 1704, display 1704 can present the monitor message comprising one or more notice, and such as power system indicates chart 1706, alarm 1708, the description 1720 of state of various parts, temperature reading/legend 1712 and/or about the visual heat picture 1740 of the user of power system components.In various embodiments, the monitor message that display 1704 presents can present with text and/or graphic form.The notice of monitor message additionally or alternatively can provide can listen form.Therefore, by display 1704,1400 systems (such as, when the vehicles that power system 1430 drives are used) drives to the user of power system 1430 or power system 1430 in real time the user of machine can present monitor message.

In one embodiment, display 1704 can be touch-sensitive display, and can be configured to present power system sign chart 1706, this power system indicates differentiable (such as, shade) warning zone 1710 that chart 1706 comprises the possible problem of engine 1510 of instruction power system 1430.In one embodiment, display 1704 also can be configured to provide user interface, and user can by selecting differentiable warning zone 1710 by the respective regions being pressed in touch-sensitive display 1704, and can displays temperature legend 1712,1720 and the visual heat picture 1740 of user are described.Temperature legend 1712 can indicate the relative temperature deviation of the temperature data such as provided visually by heat picture 1740, or can indicate the scope of the absolute temperature providing 1740 to provide by heat picture.Describe 1720 and can identify such as one or more relevant power system components, the temperature of power system components and the state of power system components, and the user's selectable option correcting or alleviate the state may damaging power system can be comprised.

Such as, temperature legend 1712, describe 1720 and the visual heat picture 1740 of user can indicate, the delivery temperature of cylinder 3 exceedes more than normal working temperature 20 degrees Fahrenheit measured at part 1742 place of exhaust manifold, and exceedes normal working temperature about 10 degrees Fahrenheit at part 1744 place of exhaust manifold.In one embodiment, such as, user can by selecting description 1720 by the respective regions being pressed in touch-sensitive display 1704, thus apprizing system 1400 user knows determined state and by the work of hand relieving power system.In another embodiment, user selects description 1720 can enable system 1400 such as by least shifting fuel from cylinder 3 and igniting subtracts relieved state until technician revises this state by suitable maintenance.

With reference now to Figure 18, Figure 18, show the process flow diagram of the process of the parts of the supervision power system according to disclosure embodiment.Such as, each infrared imaging module 1402a of processor 1410 and/or system 1400 can be passed through, 1402b, 1402c also utilizes optical element 1404a, 1404b, 1404c, storer 1412, communication module 1414, display 1416, control module 1432, any one in load module 1434 or other surveillance parts 1440 carrys out one or more parts of implementation 1800, wherein each infrared imaging module 1402a, 1402b, on one or more parts that 1402c can be installed in power system 1430 or be arranged in one or more parts of power system 1430.It should be appreciated that the object certainty annuity 1400 and power system 1430 that are only citing, and any other suitable system can be able to be arranged in any other suitable power system, to perform the part of whole process 1800 or implementation 1800.

At block 1802, the one or more infrared imaging modules by system 1400 catch one or more heat pictures of the part of the parts of power system (such as, power system 1430).Such as, as shown in figure 16, can by being arranged on the infrared imaging module 1402a on the shell 1560 of power system 1430, 1402b, 1402c, catch and comprise from combustion engine 1510 (such as, comprise inlet manifold 1512, cylinder head 1520, exhaust manifold 1522, crankcase 1524, turbine 1514 and aftercooler 1516), motor 1540 (such as, comprise power switch circuit 1542, rotor 1544, motor winding 1546, bearing 1548), HV battery 1584, HV generator 1590, water/oil cooling system 1564/1566, the heat picture of the thermal-radiating image of gearing 1578 and/or other power system components.Such as, one or more heat picture can be received at the processor 1410 being couple to one or more infrared imaging module via wired or wireless link communication.

At block 1804, such as, processor 1410 can be passed through, by infrared imaging module 1402a, 1402b, 1402c and/or by various sensor (such as, comprise other surveillance parts 1440), in storer 1412, store one or more heat picture and relevant background information.Background information can comprise the various characteristic relevant to heat picture and ambient condition, and such as timestamp, environment temperature, ambient pressure, general power export, the translation motion of power system 1430, monitored the identification of power system components in elapsed time since starting and/or heat picture and coordinate thereof.

Background information can instruct and how process, analyze and/or use heat picture.Such as, background information can disclose that heat picture is inlet manifold 1512 or exhaust manifold 1522, heat picture when experiencing infrabar or high ambient temperature.This heat picture can be used for detecting abnormal high total temperature, the air/fuel mixture of non-optimal and other abnormality.

In other modes this and various, (such as, by processor 1410) background information can be utilized to determine the suitable application of associated hot image.As described in further detail hereinafter, background information can also provide input parameter for execution heat picture analysis and distribution plan cut open to paint.In various embodiments, background information can directly be collected at processor (such as, processor 1410), process or otherwise manage, and is not stored in independent storer.

Such as, at block 1806, by using various NUC technology disclosed herein, NUC process can be performed to remove noise wherein to what catch with the heat picture stored.In one embodiment, the background information relevant to heat picture can be analyzed, to select the fuzzy heat picture (such as, based on the heat picture of motion blur) treating to be used by NUC process described herein.

At block 1808, can determination operation pattern alternatively.Operator scheme can comprise training mode and monitoring mode.Such as, use switch, power system diagnostic device and/or other suitable input medias, be operated in user in power system 1430 or technician can make system 1400 enter training mode.Alternatively, when system 1400 detects some trigger condition, such as, when first time installation system 1400 maybe when installing new power system components, system 1400 can be made to enter training mode.

If be in training mode at block 1808 certainty annuity (such as, system 1400), then baseline parameter and distribution plan can according to the heat picture structures of catching at block 1810.At block 1812, baseline parameter and distribution plan that (such as, in storer 1412) construct can be stored.Baseline parameter and distribution plan can represent the normal operating state of the various power system components in heat picture, and comprise image coordinate and border, temperature range, heating and cooling characteristic (such as, thermal capacity, temperature conductivity), other characteristics of the power system components in Temperature Distribution and variance pattern and heat picture.

Baseline parameter and distribution plan can be built by the various statistics of Collection and analysis.Such as, the background of statistics and prospect modeling technique (such as, using the time series mean value of pixel value to distinguish static background from dynamically " interest area ") can be used for identifying coordinate and the border of various parts in heat picture.Perform thermal imaging analysis and distribution plan during the baseline parameter constructed in training mode and distribution plan are used in monitoring mode to cut open and paint, to determine the state of the various power system components in heat picture.

When the various characteristics of power system components depart from predetermined factory-said value, training mode is useful.Such as, after-market power system components may be different from factory power system unit dimensionally, this can find at block 1810-1812 (such as, due to the different images coordinate that has in heat picture and border) and record.In another example, for severe service components market parts, normal working temperature scope and Temperature Distribution pattern can be different, and severe service components market parts can be tolerated higher temperature or even perform better in higher temperature.

In some embodiments, baseline parameter and distribution plan can input (such as, by technician or the user of power system) and not execution block 1810-1812 by hand.In some embodiments, the manufacturer of machine that can only be driven by power system, power system in factory and/or the setter of surveillance (such as, system 1400) are programmed baseline parameter and distribution plan in advance, and not execution block 1808-1812.

If at block 1808 certainty annuity (such as, 1400 systems) be in monitoring mode, then can perform heat picture analysis and distribution plan to heat picture to cut open and paint operation (such as, by processor 1410), to determine the state of various power system components and to produce corresponding monitor message.

At block 1820, for each power system components in heat picture, identifiable design border and pixel coordinate.Such as, from the heat radiation of turbine 1514, aftercooler 1516, cylinder head 1520, exhaust manifold 1522 and exhaust system 1568, by identifying the border that they are each and pixel coordinate and they are distinguished from each other and make a distinction with shell 1560.In one embodiment, basic parameter and/or the background information relevant to heat picture can provide border and pixel coordinate for the power system components in heat picture.Such as, can according to background information for the position of the rotor 1544 of motor 1540 adjust predetermined (such as, during training mode or in factory) baseline boundary and coordinate, with when do not perform block 1820 further image procossing realize determining border and pixel coordinate.

In another embodiment, the pixel coordinate of each power system components and border identify in real time by performing edge detection algorithm, spot detection algorithm and/or other suitable image processing algorithms to heat picture.In various embodiments, any combination of scan picture operation, background information and baseline parameter can be used for identifying the border in the border of power system components and coordinate and heat picture.

At 1822 pieces, according to the heat picture of the thermal-radiating image comprised from various power system components, the temperature at least partially of the parts of power system 1430 can be determined.As about the infrared imaging module 1402a of Figure 14,1402b, 1402c discuss, radiation-curable mensuration ground calibration heat picture to be to comprise the temperature data of the calibration of each pixel in heat picture.Corresponding to the thermal-radiating pixel of certain power system components by analyzing, this power system components temperature reading all or in part can be obtained.By utilizing the emissivity of the material of these parts of composition, temperature reading can be improved further.

At block 1824, the temperature reading obtained can be compared with baseline parameter and distribution plan at block 1822, to determine that the temperature of various power system components is whether in normal range of operation.The abnormal work temperature of power system components can indicate imminent fault or break down.In addition, even if power system components is not out of order completely, abnormal working temperature also can cause hydraulic performance decline usually.Such as, the abnormal high temperature of a part for exhaust manifold 1522 may cause the sealing between exhaust manifold 1522 and combustion engine 1510 degenerate and cause ultimate failure.Similarly, the abnormal high temperature of turbine 1514 may cause final fault.In another example, the abnormal high temperature of the rotor 1544 of motor 1540 can indicate the final fault (such as, locking) of friction increase and rotor bearing or other bearings 1548.

If one or more abnormal temperature state detected, notice or alarm flag correspondingly can be set, to comprise suitable alarm at monitor message.Such as, flag is warned can to indicate the abnormal high temperature state of a part for certain parts, these parts or other associated components of power system.

In one embodiment, heat picture can be analyzed to detect the heat of power system components or cold part.Hot or cold part is the regional area of the bulk temperature departing from power system components.Hot or cold part usually indicates the formation of structure failure point and manifests, and it finally can cause the fault of power system components.It should be appreciated that because heat or cold part are local, so the total temperature of monitored parts may still in normal range.Therefore, detection that is hot or cold part can detect and warn and do not detected by independent abnormal temperature the precarious position disclosed.

Such as, the relatively cold point in cylinder head 1520 can indicate the talus accumulation on inside cylinder head 1520, if chip is separated with cylinder tegmental wall, then may finally cause oil system to block.Similarly, the focus of aftercooler 1516 cold side may indicate the blocking of cooling pipeline, and this finally may cause the fault of the miscellaneous part of aftercooler 1516, turbine 1514 or power system 1430.In addition, little focus or cold spot may indicate water or oil to leak.The heat picture 1740 that the user of Figure 17 is visual shows the very hot point 1742 and hot point 1744 that can separate with the remainder circle of good definition of engine 1510/ exhaust manifold 1522.

In one embodiment, by the parts to power system 1430 at least part of clear (such as, not fuzzy) heat picture performs spot detection operation or other suitable heat picture analyses, can detect these and other heat or cold parts of various power system components.As described in about block 1804, the background information relevant to heat picture can be analyzed, to determine that heat picture is clearly or fuzzy.If basic background information determination heat picture is enough clearly, then can perform spot detection operation or other suitable heat picture analyses.If one or more heat or cold point detected, then notice or alarm flag correspondingly can be set, to comprise suitable alarm at monitor message.

In another embodiment, (such as, by processor 1410) heat picture can be analyzed to detect the crack of power system components.Various power system components, such as crankcase 1524, exhaust manifold 1522, oil groove 1526 and heating radiator/water cooling system 1564 may produce crack.Interrupt because himself usually can be revealed as thermal gradient by such crack in heat picture; so they can detect; in one embodiment, them are detected by detecting operation, edge detecting operation or other suitable operations to the heat picture of these parts line performed for detecting thermal gradient interruption.With the detection type of heat or cold spot seemingly, if based on background information determination heat picture be enough clearly, then can perform crack detection.If one or more crack detected, then notice or alarm flag correspondingly can be set, to comprise suitable alarm at monitor message.

At block 1826, can analyze (such as, by processor 1410) heat picture to be to obtain Temperature Distribution and the variance distribution plan of heat picture medium power system unit, and use obtains variance distribution plan to detect the abnormality of power system (such as, power system 1430) from heat picture.According to Temperature Distribution uneven in power system components and variance, various abnormality can be indicated.Such as, Figure 17 shows and may be presented in uneven Temperature Distribution on exhaust manifold 1522 and pattern.As shown in figure 17, the concentrated vent valve that can indicate leakage of the highest abnormal temperature of closer engine 1510 firing chamber.

In one embodiment, can associate with variance distribution plan relative to the predetermined temperature profile of abnormality, mate, distribution plan cuts open the Temperature Distribution painting or otherwise compare and obtained by heat picture and variance distribution, with the various abnormality of detection and Identification.Such as, if the distribution plan obtained mates with this kind of state, then processor 1410 detection and Identification power system 1430 can have the vent valve of leakage.

In another embodiment, can by comparing the distribution plan that obtained by heat picture and detecting abnormality about block 1810-1812 at above-described baseline profile figure.Because baseline profile figure can represent the normal work distribution plan of power system components, so can abnormality be indicated with the deviation (such as, uneven Temperature Distribution) of baseline profile figure.Such as, there is the waste gate of leakage, turbine fires the oil that falls pile up or the too Temperature Distribution of the turbine 1514 of high pressure-charging pressure and variance distribution, the baseline profile figure representing smooth and uniform Temperature Distribution and variance may be departed from.

In another embodiment again, any uneven Temperature Distribution and variance all can be detected as exception, and itself and abnormality distribution plan or baseline profile figure are not compared.In various embodiments, above-mentioned distribution plan can be used to cut open and paint any combination of operation to detect abnormality.In the embodiment obtained distribution plan and abnormality distribution plan and/or baseline profile figure compared, the background information relevant to heat picture can be analyzed to select suitable distribution plan.Such as, some abnormality distribution plans and/or baseline profile figure can be configured to make it compared with the distribution plan obtained according to not fuzzy heat picture.If background information instruction heat picture is not fuzzy, then such abnormality distribution plan and/or baseline profile figure can be selected for comparing with it.

In the embodiment that obtained distribution plan and abnormality distribution plan compare, various distribution plan can be adjusted according to baseline profile figure and cut open and paint operation.Such as, can offset, shift or otherwise change the data point in abnormality distribution plan, to make up the baseline profile figure being different from the predetermined distribution plan that dispatches from the factory.

If paint operation detection to one or more abnormality by cuing open the above-mentioned various distribution plan of block 1826, then can notice flag be correspondingly set, to comprise suitable notice and/or alarm at monitor message.

It should be appreciated that the various distribution plans comprised in block 1826 cut open the process 1800 of painting operation, can some abnormality of early detection, these abnormality can keep not detecting if it were not for getting up early detects until affected power system components permanent damages.Such as, owing to occurring that in the region that blocks, the temperature of crankcase 1524 can increase slightly in the past, the distribution plan in block 1826 cuts open and paints operation can be increase slightly in the region of blocking by manifesting in crankcase 1524 temperature by the jam detection manifested in inspection oil/water cooling system 1566/1564.This state of words if not so may keep not detecting---or even when having installed conventional cooling system temperature sensor, until become obvious due to crackle or the damage of inner bearing to engine of breaking down.Therefore, process 1800 can early detection, and early detection makes the operating personnel/owner of power system reduce maintenance cost by the wearing and tearing avoiding power system components not formally to be shaped.

Such as, in one embodiment, by the thermal capacity of tracking power system components (such as, turbine 1514, aftercooler 1516, exhaust manifold 1522 and exhaust system 1568) and the degeneration of temperature conductivity, the wearing and tearing of power system components can be determined.As everyone knows, these systems can be looked at as dispel the heat in the gas comprised at them heat sink.So, the degeneration of thermal capacity or the change of temperature conductivity can indicate the wearing and tearing (such as, the loss of part quality) of these parts.

Associating by for specific time interval the temperature variation of power system components being exported with power system, thermal capacity and/or temperature conductivity can be obtained.Such as, utilize two or more heat picture, temperature contrast can be determined (such as, by comparing the temperature reading obtained at block 1822), and can by the background information relevant to this two or more heat picture (comprise driven by power system translation motion, timestamp and other related datas) obtain output power.If thermal capacity or temperature conductivity are reduced to certain level relative to baseline, then notice or alarm flag correspondingly can be set, to comprise suitable alarm at monitor message.

In another embodiment, by comparing the temperature difference between the convex surfaces of power system components and groove, can determine the wearing and tearing of power system components, its further groove represents the thin part of the ratio convex surfaces of power system components.In this embodiment, wearing and tearing can be determined by the heat picture of power system components, and it analyzed (such as, by performing edge and/or line detection operation) can also obtain the temperature difference between detected groove and convex surfaces with detecting groove.Such as, if background information instruction heat picture is (such as, 30 is not fuzzy) the thermal-radiating image of the power system components reaching normal working temperature that comprises to control oneself clearly, then processor 1410 can perform temperature difference analysis.If the temperature difference for outside the threshold value of given state, then correspondingly can arrange notice or alarm flag, to comprise suitable alarm at monitor message.

In further embodiment, (such as, by processor 1410) heat picture can be analyzed to detect via the gas of the slit leakage in power system components and/or liquid.In one embodiment, heat picture such as can distinguish carbon monoxide and/or fuel fumes from air, and the leakage detected in exhaust system 1568 (such as, carbon monoxide), inlet manifold 1512 or fuel system 1570 (such as, fuel fumes).In another further embodiment, the heat picture from inlet manifold 1512 interior section can being analyzed, with the relative density by detecting air and fuel in potpourri, detecting the air/fuel mixture of the firing chamber being transported to combustion engine.

At block 1828, use suitable method and algorithm heat picture can be converted to the visual heat picture of user (such as, Thermogram).Such as, as above about Figure 14 processor 1410 as described in, the image that the heat record data be included in heat picture can be converted to gray scale or color range pixel can be seen to build people.The visual heat picture of user selectively can comprise legend or the scale of the approximate temperature of instruction respective pixel color and/or intensity.The heat picture that such user is visual, if present over the display (such as, on display 1416/1704), then can contribute to user or technician and confirm or understand the state detected by process 1800 better, or identify visually not by state that process 1800 detects separately.

At block 1830, can by collecting, editor, to analyze or other modes manage and cut open various notices, alarm and the data of painting operation from above-mentioned various heat picture analysis and distribution plan and produce monitor message.In one embodiment, monitor message can comprise one or more notices of the state based on various detection, one or more descriptions of the state detected (such as, the position of the abnormality detected and classification), one or more temperature readings of one or more power system components, the heat picture that one or more users of associated power system unit are visual, and/or other data and alarm.Therefore, monitor message can comprise integrated data about the state of various power system components and warning, so, can advantageously enable user or the owner avoid infringement of a high price.

At block 1832, (such as, in storer 1412) background information, the monitor message of generation can be stored and/or other obtain or the data that produce.In order to consult and diagnose further the state of monitored various power system components, user can fetch or recover stored information and data subsequently.

In one embodiment, can to monitor message and obtain within a certain period of time and/or produce other related datas and carry out trend analysis.This analysis can produce the summary view of the various states of power system components.Can upgrade and/or store this trend summary at block 1832, and be fetched by user subsequently, such as, to be used as the guidance of one or more parts of correct maintenance power system (such as, power system 1430).In an example, trend summary can comprise the average image of the visual heat picture of the user of the parts of power system.In another example, the trend summary of storage can comprise related data between monitor message and some or all of background information (such as, translation motion reading, crankshaft rotating velocity readings, the load reading of machine that driven by power system).This related data can be used for disclosing various factors to the impact of power system components.Such as, be under relatively large load or under relatively little load based on power system, user optionally consults monitor message summary.

In some embodiments, monitor message, trend summary and/or other related datas can be supplied to the conventional airborne data recording equipment for storing.Such as, many power systems are all equipped with data acquisition and pen recorder.Can make monitor message and other operate relevant data (such as, operating duration, power system load) synchronous and to other operate relevant data be stored in together in this device for real-time with operate after analysis.

In example more specifically, service data pen recorder can have multiple video port, and it is for storing the multiple video flowings synchronous with other service datas various.The visual heat picture stream of user (heat picture that the user such as, produced at block 1828 is visual) can be admitted to one of these video ports for synchronously storing.The visual heat picture stream of user even can be tiled, spliced or otherwise combined, to illustrate the different piece of power system simultaneously.

At block 1834, one or more power system components can be adjusted based on monitor message, to emphasize the reliability of power system, performance, efficiency and/or the convenience to power system user.In one embodiment, if the efficiency of monitor message instruction non-optimal or power stage, then can produce the various parts that signal adjusts combustion engine 1510 automatically to the control actuator and electronic installation that are attached to various power system components by processor (such as, processor 1410).In one embodiment, if the potpourri of monitor message instruction short bunker, then can be activated the waste gate of turbine 1514 by processor, automatically adjust air/fuel mixture.In other embodiments, user (such as, technician) can adjust potpourri and/or turbo boost pressure based on such as the monitor message of the storage as described in for block 1832 and/or trend summary.Based on comprehensive with the automatically this of real-time monitor message and/or manually adjust, enable the working temperature that various power system components keeps suitable, thus realize best combustion and/or power efficiency.In addition, in order to provide the facility of power system and operate reliably, various power system components can be adjusted according to user, technician or the plan in factory set by processor (such as, processor 1410).

User can also be given, the vehicles everyone, technician or other suitable users present monitor message display on such as display 1704/1416.In one embodiment, monitor message can be presented on the display of contiguous power system installation (such as, display 1416) on, to make when power system is by use, user can understand any danger of various power system components and/or state of a high price in real time.

In various embodiments, such as, based on the image of being caught by one or more non-thermal video camera and/or view data, and/or based on as described herein by superpose, merge, mixing and/or otherwise combine infrared, heat and/or non-thermographic build combination image, process 1800 can be suitable for determining and presenting monitor message.Such as, use method described herein, to be applicable to the non-thermal and/or combined imaging of process, can help to locate and/or otherwise indicate the specified point being about to break down with the reflection/radiation of the engine components of non-thermal wavelength imaging, it compares the spatial accuracy and/or reliability with increase than independent thermal imaging.

Therefore, it is to be appreciated that, process 1800 allow to the correlation behavior using conventional sensors (such as, temperature sensor, chemical sensor) can not detect and/or not have when power system is in dormancy or stationary state the various power systems that the inspection of expert just can not identify carry out on machine with real-time detection and warning.Also should consider, except power system components, process 1800 can adjust or revise for monitoring other mechanical parts various that power system drives.In addition, process 1800 allows to control power system based on this supervision, one or more with what emphasize in the reliability of power system, performance, efficiency and user friendliness.

In a suitable case, realize by the combination of hardware, software or hardware and software the various embodiments that the disclosure provides.Meanwhile, in a suitable case, when not departing from spirit of the present disclosure, proposed various hardware component and/or software part can be merged into and comprising software, hardware and/or the composite component of the two.In a suitable case, when not departing from spirit of the present disclosure, proposed various hardware component and/or software part can be separated into and comprise software, hardware or the subassembly of the two.In addition, in a suitable case, can be expected that, software part can be embodied as hardware component, and vice versa.

According to software of the present disclosure, such as, non-transitory instruction, program code and/or data can be stored in one or more non-transitory machine readable media.Can also be expected that, can use networking or one or more general or special purpose computer of not networking and/or computer system implement herein mentioned by software.In a suitable case, the order of various step described herein can change, merges into composite steps and/or be separated into sub-step, to provide function described herein.

Embodiment described above only illustratively, instead of limits the utility model.It is to be further understood that, according to principle of the present utility model, many amendments and change are possible.Therefore, scope of the present utility model is only limited by claims below.

Claims

1., for monitoring a system for power system, it is characterized in that comprising:

Infrared imaging module, it is arranged as the parts of observation power system at least partially, and described infrared imaging module comprises the focal plane arrays (FPA) of the heat picture of the described part being configured to catch described parts; With

Processor, it is to described infrared imaging module communication and be configured to process described heat picture to produce the monitor message relevant with described power system.

2. system according to claim 1, is characterized in that wherein:

Described processor is configured to analyze described heat picture to determine the temperature of the described part of described parts; With

Described monitor message comprises the notice based on the temperature determined.

3. system according to claim 1, is characterized in that wherein:

Described processor is configured to analyze described heat picture, to determine the state of the power system relevant to the described part of described parts; With

Described monitor message comprises the notice based on the state determined.

4. system according to claim 3, is characterized in that wherein:

The described state determined is the state of blocking or the water leaked or oil cooling system, the state of out of order water or oil pump, the combustion chamber charge leaked or the state of vent valve, the state of the air/fuel mixture of non-optimal, the state of out of order bearing, overheated combustion engine parts or the state of motor component, the state that exhaust system is leaked, the state of accumulation of debris on component inner surface, the state of out of order high-voltage battery, the state of parts or power system load, the state of component wear, the state of part distortion, the state in parts crack, the state of the state that carbon monoxide leaks or fuel leakage.

5. system according to claim 1, is characterized in that wherein:

Described system configuration is the control signal producing the adjustment one or more power system components relevant to described heat picture based on described monitor message.

6. system according to claim 5, is characterized in that wherein:

Described control signal is according to the handling capacity of the charge or discharge of described monitor message adjustment air/fuel mixture, fuel chambers fired state or opportunity, turbine exhaust door state, turbo boost pressure, high-voltage battery, high-voltage battery charge rate or discharge rate, crankcase heater voltage, ventilation for enclosures, fan speed, water or oil cooling system, ratio of gear or combustion engine or output power of motor.

7. system according to claim 1, is characterized in that wherein:

In the described part that described infrared imaging module is installed to described power system, the shell of described power system, the described parts of described power system or described parts or inner.

8. system according to claim 1, is characterized in that wherein:

Described power system comprises combustion engine or motor.

9. system according to claim 1, is characterized in that wherein:

Described power system is used for providing machine power to machine; With

Described monitor message is supplied to the user of described machine.

10. system according to claim 9, is characterized in that wherein:

Described machine is that land is used, water is used or the sky vehicles; With

When described user is using described machine, described monitor message be supplied to user at least partially.

11. systems according to claim 1, it is characterized in that wherein catching described heat picture in the very first time, described system comprises further:

Non-thermal video camera, it is configured to the non-thermographic of the described part of parts described in the second Time capture, and wherein said processor is configured to process described heat picture and described non-thermographic to provide the image of combination; With

Display, it is configured to the image presenting described monitor message and described combination to user.

12. systems according to claim 1, is characterized in that wherein:

Described focal plane arrays (FPA) comprises the array being suitable for the micro-metering bolometer receiving the bias voltage selected from the scope of 0.2 volt to 0.7 volt.

13. systems according to claim 12, is characterized in that wherein:

The described array sizes of micro-metering bolometer is 80 take advantage of 60; With

Described infrared imaging module is suitable for being inserted into size and is less than 8.5mm and takes advantage of in the socket of 8.5mm.

14. systems according to claim 1, is characterized in that wherein:

Described heat picture is the not fuzzy heat picture of the described part of described parts;

Described infrared imaging block configuration is the heat picture of the deliberate fuzziness of the described part of catching described parts; With

Described processor be configured to determine multiple nonuniformity correction item based on the heat picture of described deliberate fuzziness and heat picture not fuzzy described in described nonuniformity correction item is applied to remove noise from described not fuzzy heat picture.