WO2012032215A1

WO2012032215A1 - Thermography method and system

Info

Publication number: WO2012032215A1
Application number: PCT/FI2011/050078
Authority: WO
Inventors: Ari Nevalainen; Juha Frant; Jari Karhu
Original assignee: Thermidas Oy
Priority date: 2010-09-06
Filing date: 2011-02-01
Publication date: 2012-03-15
Also published as: FI20105928A0

Abstract

The present invention relates to a method and system for thermal imaging. In the method several thermal images successive in time(1) are formed from an object (2), forming a time varying thermal image signal (3). In accordance with the invention at least one other signal parameter (4) is measured essentially synchronously with the thermal images (1), forming a time varying other signal (5), and a combination signal (6) is formed as a combination of a time varying thermal image signal (3) and the time varying other signal (5).

Description

THERMOGRAPHY METHOD AND SYSTEM

Field of invention The invention relates to a method for thermal imaging according to the preamble of claim 1 and to a system according to claim 16

The present invention relates to non-invasive detection of the thermal radiation from the human body for the purpose of inferring information related to the distribution of temperature within the body. In particular, this invention relates to thermography that combines other relevant information from the environment where the recording is conducted. Additionally, the invention relates to the combined recording of biological signals from the same target body with the thermal imaging information in order to detect diagnostically relevant changes more reproducibly and reliably than would otherwise be possible.

Prior Art

Thermography devices are presently utilized for the measurement of the temperature of various targets. For example, thermal imaging devices have been conventionally used to detect heat leakage from buildings, but lately they have been also used for determining the temperature of biological tissue. A thermal imaging, or thermography, device detects the electromagnetic radiation that originates from the heat of materials. The wavelength of this radiation depends on the heat and other characteristics of the body. The radiation due to heat in the human body is at longer wavelengths than the human eye can see, and infrared camera is therefore used to show this radiation.

Thermography devices show the surface temperature of an object. The thermal radiation depends on the distribution of heat sources or sinks within the object and on the heat conduction characteristics of the body as well as on the convection/radiation from the body. The ambient temperature and conditions affect the imaging also. In brief, infrared camera generates images with thermal data in real time, and it is possible in a fraction of a second to image large areas of the human body.

Currently, one manufacturer, Flir Inc., promotes thermal imaging for numerous applications ranging from night vision systems to medical thermal imaging. In medical uses, thermal imaging has been used for example for screening people for elevated temperatures which may indicate a viral infection, such as swine flu, SARS or malaria. Flir Inc has disclosed various uses, suggesting also that the skin temperatures can be used as information that indicates normal or abnormal human physiology. Medical research has shown thermography to be a useful tool in research as well as being helpful in the diagnosis of breast cancer, nervous system disorders, metabolic disorders, neck and back problems, pain syndromes, arthritis, vascular disorders, and soft tissue injuries among others.

A major benefit of thermography in medical applications is that it is completely non-invasive. Its medical uses can possibly even replace use of other invasive techniques. A second major benefit is that thermography can be very quick.

Disadvantages of the Prior Art

The known uses of infrared imaging in medical diagnostic applications are relatively few because the methods of measurements and the devices currently known produce results that do not easily comply with the requirements of medical applications in what comes to reliability and reproducibility. Another known disadvantage is that the temperature of the body changes constantly and relatively quickly for instance due to variations in blood circulation and extracorporeal temperatures. The known methods provide thermal image quality that is insufficient for reliable and reproducible detection of clinically relevant features and changes in biological tissue.

Aim of the Invention It is the aim of this invention to reduce or remove at least some of the disadvantages of the prior art and to provide method and equipment that utilizes additional and essentially simultaneously acquired information to the analysis of the thermal images. Such information can be derived from the target itself or of the environmental conditions. In particular the invention describes a de- vice and method for the measurement of dynamic thermal radiation of biological tissue that can provide reliable information about the fast changes (milliseconds to seconds) in the temperature distribution of biological tissue. Further aims are reliability and reproducibility improvements. Summary of the Invention

The device includes an infrared (IR) sensitive camera. The camera consists of a detector array and electronics that process the detector signals. The electronics is typically in the form of programmable logic circuit(s) that process and convert the signals suitable for display on the camera's display. In this invention, however, the IR signals detected are processed combined with one or more additional signals that are derived from other types of signals that are not necessarily IR radiation of the body. Such signals can be either biological/physiological signals from the same biological body, or environmental signals such as temperature, humidity or distance to target. Signal processing (note that in what follows the terms signal processing and image processing are used interchangeably) is accomplished by combining the non-IR signals in the same programmable circuits where the IR detector signals are processed. Alternatively, it may be advantageous to add a second stage of processing, such as programmable logic or computer, where the different signals are combined in signal processing. It is necessary for these steps that the IR and non-IR signals are collected in time-wise synchronous fashion, i.e., essentially simultaneously. For this purpose, the device features one or more electronics circuitries. With 'essentially simultaneous', or 'essentially synchronously', it is intended here that in case of physiological signals, they reflect the same phases of the biological phenomena of interest in the thermal image signals. In case of other physical signals, 'essentially simultaneous' has a wider range depending on how fast varying the physical signal is; for instance, if environmental temperature is stable throughout the measurement session, it may be enough to measure it only once, or at least at low sampling rate, while other parameters may require higher sampling rate and synchronization with the thermal image signal. For physiological signals (such as ECG) this often means that the signals should be collected with time synchronization typically better than the time between two heart beats, i.e., better than some hundreds of milliseconds, and preferably the time synchrony should be better than ten milliseconds. In other foreseeable applications the resolutions may vary greatly, for instance, from milliseconds to seconds.

More specifically, the method for thermal imaging according to the invention is characterized by what is stated in the characterizing part of claim 1 .

Further, the system according to the invention is characterized by what is stated in the characterizing part of claim 23.

The invention provides significant benefits.

Advantages

The invention herein described has several advantages. The detection and signal/image processing methods allow for unprecedented quality of the thermal imaging data. The data quality enables measurements that lead to a level of reproducibility and repeatability necessary for detecting information that has diagnostic utility. Since the analysis phase can utilize additional data, such as the environmental parameters like temperature and humidity and distance to target, it becomes possible to exclude most of the error sources that add uncertainty to thermal imaging. Improved data quality enhances the capabilities for detecting diagnostically relevant information in the thermal images and reduces probability of human errors.

Description of the preferred embodiments

In the following, the invention will be examined in greater detail with the help of exemplifying embodiments illustrated in the appended drawings, in which

FIG. 1 shows as a block diagram of the prior art. FIG. 2 shows as block diagram one embodiment of the present invention.

FIG. 3 shows as block diagram another embodiment of the present invention.

In accordance with figures 2 and 1 the system includes an infrared camera 10 having detectors 20 for infrared light. The detector signals are fed to a processor 21 , which can be used to transfer the signals digitally to a computing unit 23. Alternatively, the computing unit 23 and the processor 21 can be combined into one unit. The processor or the computing unit may include one or more inputs 30 that can be used to add information content for the analysis of the infrared images 1 . The preferred embodiment includes at least the measurement of the ambient humidity and temperature. The information are used to calibrate the thermal imaging measurement so that uncertainties caused by varying environment are minimized. In practice the measurement of the humidity and the temperature need to be performed to a suitable accuracy to allow for efficient analysis of the thermal images. This is important also in cases where the same subject is studied on multiple visits for followup. In the imaging method several thermal images successive in time 1 are formed from an object 2 and from these individual images is formed a thermal image signal 3 comprising information of several successive images 1 . At least one other signal parameter 4 is measured essentially synchronously with the ther- mal images 1 , forming a time varying other signal 5, and a combination signal 6 is formed as a combination of a time varying thermal image signal 3 and the time varying other signal 5. The combination signal can also be time varying, or it can be processed into a static image that includes information from the time varying signals.

Thermal imaging speed depends on the application. Typical speed is to acquire at least 50 frames per seconds (fps). Thermal image temperature resolution can vary also widely. In practical foreseeable applications the resolution can vary between 0.001 to 0.5 degrees. Good temperature resolution is required for instance to detail the vein/arterial image temperature gradients where the difference can be only in the order of 0.01 to 0.05 degrees.

Although the spectral content of the thermal images can be wide, it is preferable to focus on 8 to 12 μιτι, to avoid interferences from room lighting (e.g., flashing of the lights due to breakage or flaws in the lighting). The device features possibility to measure with the device itself a reference temperature of the body. A practical location of the body is the throat through the open mouth. Alternatively this reference temperature may be measured from eye or ear. This temperature is used in the system software to refine the analysis. The invention includes an additional or optional possibility to input biological signals in addition to the environmental signals. In particular, it is preferable to record the subject's ECG (electrocardiograph) signal to synchronize the thermal imaging measurements. ECG is recorded using bioamplifiers and provides information about the pulsation of the heart. Since heart pulsation is irregular, direct measurement using ECG is needed in any application that examines the correlation of the thermal imaging signals with heart pulsation cycle. An example of such application is to differentiate between the venous and arterial phases. The device can thereby form images that are used to visualize the veins and the arteries of a subject. This can be for example aver- age (or similar or more complex calculation) of several instances. In this case, it is advantageous that the frame rate of thermal imaging is up to double the heartbeat rate (i.e., up to 4 fps), or in case of lower frame rate, it is beneficial to acquire more thermal imaging data and use postprocessing tools to select frames where the image is in the same phase of the ECG signal. Obviously, other physiological signals can be used as well. These include heartbeat variability rate, electromyography, skin conductance, pulse oximetry, breathing rate or phase, ultrasound etc.

Also other optional inputs can be used in the analysis. Such are distance from camera to object 2, which can be determined using a laser, for example. The method includes the possibility of acquiring one or more reference images, which is used to determine a calibration temperature for the analysis of the actual diagnostic images. The image content is calibrated, for instance, to the emissivity of a target in the body that has relatively stable temperature, such as the open mouth or throat. The device may additionally include a feedback loop that can be used to feed control signal to either the camera or the non-IR detection circuitry. An example of this is the possibility to move or change the size of the imaging area to fit different sizes of the same extremity and/or to account for changes in the distance of known and easily recognizable landmarks in biological body from the sensors. Also the focusing of the optics can be automated. It may also be beneficial to analyze environmental parameters measured and use them to control for example the sensitivity scales of the thermal imaging detection system. This leads to better signal-to- noise ratio or repeatability of the imaging results. Additionally, the system can have a means for calibrating the thermal images to a temperature of a location in the body that has relatively stable temperature independent of the environment. A good location in the human body is the throat, the temperature of which can be measured for calibration purposes using the same thermal imaging device pointed to the subject's open mouth. The calibration information is fed to the same signal analysis that is described above.

The required resolution of the image depends on the application. While good quality images are achievable already with tens of thousands of pixels, the high-definition regions in the body, and especially arterial/venal imaging can benefit from the use of as many pixels as available. The presently available cameras typically allow for at leasts 70.000 pixels.

In accordance with the invention the combination signal 6 is formed as a combination of a time varying thermal image signal 3 and the time varying other signal 5 such that for example algorithm 1 (correlation) is used.

Algorithm for forming combination signal 6 may be for example such an algorithm that forms any weighted sum of the thermal imaging signals or a subset of the images, with the subset chosen based on mathematically presented selection criteria that are derived from the time varying other signal 5 or the thermal signal itself. An example of such weighted sum is an average of thermal signals that have been collected with environmental temperature between 21 and 21 .5 °C. Another example is to weight the thermal image signal using the time synchronized other signal 5 to derive the weight factors. This is the case of using ECG or heartbeat phases to weight the corresponding parts of the thermal image signal and then summarize the signals acquired during different heart beats (aligned to the same phase of ECG) in order to differentiate veins and/or arteries from the rest of the signal to form the final image or images.

Alternatively multiplication can be used between the thermal signal 3 and other signal 5. This signal can be weighed and combined with summing above.

Also, signals 3 and 5 may be correlated with each other for forming the combination signal 6. Combination signal 6 may formed by using the other signal 5 for screening the time varying thermal image signal 3 for further data processing.

Alternatively, the combination signal 6 may be formed by using the other signal 5 for determining weighing factors for further processing of the time vary- ing thermal image signal 3.

Alternatively, the combination signal 6 may be formed as a combination of a time varying thermal image signal 3 and the time varying other signal (5) such that only such thermal image signals 1 ,3 are selected, when the other signal 5 meets a predetermined condition and from selected signals 1 , 3 is formed a weighted the thermal image signal 6.

In accordance with the invention the combination signal 6 is formed as a combination of a time varying thermal image signal 3 and the time varying other signal 5 such that for example algorithm 2 (summing) is used.

In accordance with the invention the combination signal 6 is formed as a com- bination of a time varying thermal image signal 3 and the time varying other signal 5 such that for example algorithm 3 (multiplying) is used.

The data processing means include the possibility to process more than one image of the same target. This is straightforward when the images are acquired in rapid succession and the target has not moved within the field of view of the imaging system. In practical situations this is not realistical however. Therefore the system additionally includes data/imaging processing procedures in its software that are capable of coaligning images obtained from the same target but with slightly different orientation/distance, or similar. In other words, this processing provides motion correction (alignment either ri- gid or elastic).

The device includes means for reproducing the images of the target at different time (seconds to days to months). This is accomplished by using the distance, humidity etc information either to calibrate each data set separately to analyze their difference, or to mimic as good as possible the measurement environment. One could for instance use a feedback loop that moves the camera to same distance from the object every time.

It is clearly possible to construct variations of the implementation described in the above text and figures. For example, those skilled in the art may in practical implementations wish to lead the external signals to data processing unit 23 instead of signal processing unit 21 . The detector can also be controlled by the feedback control from the data processing unit 23. It is naturally possible also to combine the data processing unit 23 and the signal processing unit 21 into one single unit that includes the necessary algorithms for the signal processing. Or they could be splitted into even more physical units. The Sync signal can be external or internal clock, and could happen in either of the processing stages.

Object 2 of the thermal imaging is typically and advantageously a human body. However, the invention may be used for any other object like thermal analysis of combustion engines in connection with quality control, quality control of thermal insulation of houses, quality control of welding process, as a control tool in chemical industry and power plants etc. There the other signal may be directly related to the object 2. And it can be for example a physical dimension or other measurement related to the object such as chemical, mechanical or electrical parameter.

Claims

1. Method for thermal imaging, where

- several thermal images (1 ) successive in time are formed from an object (2), forming a time varying thermal image signal (3), characterized in that

- at least one other signal parameter (4) is measured essentially synchronously with the thermal images (1 ), forming a time varying other signal (5), and

- a combination signal (6) is formed as a combination of a time va- rying thermal image signal (3) and the time varying other signal

(5).

2. A method in accordance with claim 1 , characterized in that the combination signal (6) is time varying.

3. A method in accordance with claim 1 or 2, characterized in that the other signal parameter (4) is directly related to the object (2) , the signal parameter (4) for example EEG-signal, ECG-signal, breathing rate or phase signal, heartbeat signal, pulse oximetry signal, oxygen saturation signal, pulsation signal etc.

4. A method in accordance with claim 1 or 2, characterized in that the other signal parameter (4) is related to ambient environment of the measurements, the signal parameter (4) being for example ambient temperature, humidity etc.

5. A method in accordance with any preceding claim or their combination, characterized in that more than one other signal parameter (4) is combined to the time varying thermal image signal (3).

6. A method in accordance with claim 5, characterized in that ambient temperature and humidity signals are combined with the time varying or static thermal image signal (3).

7. A method in accordance with any preceding claim or their combination, characterized in that the other signal parameters are used to as part of feedback loop that control the imaging parameters or location, orientation, based on features detected from the thermal image itself or based on the other signals.

8. A method in accordance with any preceding claim or their combination, characterized in that the combination signal (6) is used for determining the location of a device forming the image with respect to the object (2).

9. A method in accordance with any preceding claim or their combination, characterized in that movement of the object is corrected in order to align two or more images of the same object either based on analysis of the mutual characteristics of the images or based on the other signals such as video, laser scanning, etc.

10. A method in accordance with any preceding claim or their combination, characterized in that it includes a step for acquisition of reference temperature of the object, typically a human body.

1 1 . A method in accordance with any preceding claim or their combination, characterized in that the reference temperature of the human object is ac- quisited from mouth, eye or ear and this temperature is used to refine the method.

12. A method in accordance with any preceding claim or their combination, characterized in that it includes method steps for image, data, or signal processing.

13. A method in accordance with any preceding claim or their combination, characterized in that it includes a method step for visualization of vena and/or artery images.

14. A method in accordance with any preceding claim or their combination, characterized in that it includes at least one of method steps of

- data collection

- image processing, or

- averaging thermal images acquired in the same phase of ECG.

15. A method in accordance with any preceding claim or their combination, characterized in that it includes a method step for using color palettes to emphasize features of the thermal imaging related to blood circulation.

16. A method in accordance with any preceding claim or their combination, characterized in that the combination signal is used for diagnostic purposes.

17. A method in accordance with any preceding claim or their combination, characterized in that the object (2) is a human body.

18. A method in accordance with any preceding claim or their combination, characterized in that the combination signal (6) is formed by using the other signal (5) for screening the time varying thermal image signal (3) for further data processing.

19. A method in accordance with any preceding claim or their combination, characterized in that the combination signal (6) is formed by using the other signal (5) for determining weighing factors for further processing of the time varying thermal image signal (3).

20. A method in accordance with any preceding claim or their combination, characterized in that the combination signal (6) is formed as a combination of a time varying thermal image signal (3) and the time varying other signal (5) such that only such thermal image signals (1 , 3) are selected, when the other signal (5) meets a predetermined condition and from selected signals (1 , 3) is formed a weighted thermal image signal.

21 . A method in accordance with any preceding claim or their combination, characterized in that the combination signal (6) is formed as a correlation signal of the time varying thermal image signal (3) and the time varying other signal (5).

22. A method in accordance with any preceding claim or their combination, characterized in that images (1 , 3) of the object (2) are reproduced at dif- ferent times, (seconds to days to months) by using the distance, humidity etc information either to calibrate each data set separately to analyze their difference, or to mimic as good as possible the measurement environment.

23. Thermal imaging system comprising

- a thermal camera (10) adapted to form several thermal images (1 ) successive in time from an object (2), forming a time varying thermal image signal (3), characterized in that

- at least one other measurement device (12, 13) adapted to form a signal parameter (4) essentially synchronously with the thermal images (1 ), forming another signal (5), and

- a computing device adapted to form a combination signal (6) as a combination of a time varying thermal image signal (3) and the time varying other signal (5).

24. A system in accordance with claim 23, characterized in that it includes a computing device adapted to form a time varying combination signal (6).

25. A system in accordance with claim 23 or 24, characterized in that it includes a measuring device (12) adapted to form the other signal parameter (4) directly related to the object (2), the signal parameter (4) being for example EEG-signal, ECG-signal, breathing rate or phase signal, heartbeat signal, pulse oximetry signal, oxygen saturation signal or pulsation signal.

26. A system in accordance with claim 23 or 24, characterized in that it in- eludes a measuring device (12) adapted to form the other signal parameter (4,

5) related to ambient environment of the measurements, the signal parameter (4, 5) being for example ambient temperature, humidity etc.

27. A system in accordance with any preceding claim or their combination, characterized in that it includes means adapted to combine more than one other signal parameter (4) to the time varying thermal image signal (3).

28. A system in accordance with claim 23, characterized in that includes means adapted to combine ambient temperature and humidity signals with the time varying thermal image signal (3).

29. A system in accordance with any preceding system claim or their combina- tion, characterized in that includes means adapted to use the other signal parameters as part of feedback loop that control the imaging parameters or location, orientation based on features detected from the thermal image itself or based on the other signals.

30. A system in accordance with any preceding system claim or their combina- tion, characterized in that includes means adapted to use the combination signal (6) for determining the location of a device forming the image with respect to the object (2).

31 . A system in accordance with any preceding system claim or their combination, characterized in that includes means adapted to correct for the move- ment of the object (2) in order to align two or more images of the same object either based on analysis of the mutual characteristics of the images or based on the other signals such as video, laser scanning, etc.

32. A system in accordance with any preceding system claim or their combination, characterized in that includes means adapted to acquire reference temperature of the object (2), typically a human body.

33. A system in accordance with any preceding system claim or their combina- tion, characterized in that includes means adapted to acquisite the reference temperature of the human object form mouth, eye or ear and means adated to refine the method with this reference temperature.

34. A system in accordance with any preceding system claim or their combination, characterized in that includes means (23) adapted to process image, da- ta, or signal.

35. A system in accordance with any preceding system claim or their combination, characterized in that includes means adapted to visualize vena or artery images.

36. A system in accordance with any preceding system claim or their combina- tion, characterized in that includes means adapted to perform at least one of following steps

- data collection

- image processing, or

- averaging thermal images acquired in the same phase of ECG.

37. A system in accordance with any preceding system claim or their combination, characterized in that includes means adapted to use color palettes to emphasize features of the thermal imaging related to blood circulation.

38. A system in accordance with any preceding system claim or their combination, characterized in that includes means adapted to use combination signal (6) for diagnostic purposes.

39. A system in accordance with any preceding system claim or their combination, characterized in that the object (2) is a human body.

40. A system in accordance with any preceding claim or their combination, characterized in that it includes means for reproducing images (1 , 3) of the object (2) at different times, (seconds to days to months) by using the distance, humidity etc. information (5) either to calibrate each data set separately to analyze their difference, or to mimic as good as possible the measurement environment.