DE102011106814B4 - Method for image analysis and/or image processing of an IR image and thermal imaging camera set - Google Patents

Abstract

Method for image analysis and/or image processing of an IR image (5), wherein at least one object (7, 8) is distinguished in the IR image (5) by means of segmentation (10) and made available for further processing and for the segmented object (7) at least one further image analysis or image processing step (18) is carried out, characterized in that - in a learning mode for the segmented object (7) at least one feature is extracted and in the learning mode in a feature space at least one object-related information about the segmented object (7) is stored, whereby object-related information is assigned to individual object classes, so that certain settings for image processing are automatically made for the segmented objects that fall into the respective object classes, - and that in an application mode to at least one of the IR image (5) extracted feature at least one object-related information in the associated point or area of the feature space is read out and automatically selected for setting for the image processing.

Description

The invention relates to a method for image analysis and/or image processing of an IR image, wherein at least one object is distinguished in the IR image by means of segmentation and made available for further processing and at least one further image analysis or image processing step is carried out for the segmented object.

The invention further relates to a thermal imaging camera set with a thermal imaging camera and image processing means.

It is known to take IR images with thermal imaging cameras. The IR image is created by evaluating and processing the output signal of an IR radiation detector. An IR image can therefore consist of raw data from the detector, offset-corrected data from the detector or radiometric data. The IR image may have been captured as a single image or as part of an IR video sequence.

To visualize IR images, it is common practice to display the processed measurement results in false color representations.

For this purpose, a color palette is used, which specifies the conversion of temperature measurements into color values that are easy to visually detect.

It has been found that with high contrasts in the IR images, unsatisfactory display results are achieved. For example, this can occur when so-called hot spots are viewed in front of or behind significantly cooler environments.

The color palette, which often allows a more or less linear conversion of the temperature values into color values, then needs to be adapted to the individual case presented.

In order to obtain meaningful measured values, it is also necessary to know as accurately as possible the emissivity and reflectance of the scene under consideration. Taking IR images therefore usually requires the many years of experience of a specialist, without which measurement or processing errors can easily occur.

Out of US 2008 / 0 123 956 A1 a technique for imaging an angular field of view of a screen depending on a viewer position is known, in which red-eye regions are detected in a recorded image that belong to the current positions of one or more viewers.

From the US 2007 / 0 223 815 A1 There is known an apparatus for segmenting contours of objects in an image, comprising the steps of: acquiring means for receiving an input image containing at least one object, the image comprising pixel data sets of at least two dimensions; selection means for selecting a reference point within the object of the input image; and processing means for: generating a coordinate map of a distance parameter between the pixels of the input image and the reference point; processing the input image to generate an edge detected image from the input image; calculating at least one statistical moment of the distance parameter with respect to a pixel p of the input image, the weighting factors depending on the edge-detected image and on a filter kernel defined on a window function centered on the pixel p; and analyzing the at least one statistical moment to evaluate whether the pixel p lies within the object.

From the US 2006 / 0 098 870 A1 is known a method comprising: (a) receiving a user-selected point in an image; (b) performing a watershed transform on the user-selected point to determine a primary object; and (c) adding an adjacent watershed region to the primary object based on region contention and a smoothing constraint to form an updated object.

From Lehmann T. et al. 1997, Image processing for medicine, Springer Verlag, the use of infrared images in medicine is known.

Standard methods of object classification are known from Haasdonk B. 2005, Digital Image Processing, Unit 11, Classification, Albert-Ludwigs-Universität Freiburg.

The invention is based on the object of simplifying the creation of meaningful IR images.

According to the invention, the features of claim 1 are provided to solve the problem. In particular, according to the invention, in a method of the type mentioned at the outset, it is provided that at least one feature is extracted for the segmented object in a learning mode and at least one object-related information about the segmented object is stored in a feature space in the learning mode, with object-related information being assigned to individual object classes , so that certain image processing settings are automatically applied to the segmented objects that fall into the respective object classes - and that in an application mode for at least one feature extracted from the IR image (5), at least one object-related information in the associated point or area of the feature space is read out and automatically selected for setting for the image processing.

The invention thus offers the advantage that the image analysis or image processing steps for preparing the recorded measurement data, the raw IR image, can be intelligently adapted to the displayed content of the IR image. With the invention, it is no longer necessary to carry out the desired image analysis or image processing step for the entire IR image in the same way, but rather tailor-made and targeted processing is possible. The segmented objects can therefore be separated from one another and from the background and can then be manipulated individually using image analysis or image processing steps.

For example, it can be provided that the segmentation includes the selection of at least one pixel in the IR image. The advantage here is that the object can be distinguished for the image analysis or image processing step by first selecting an image point assigned to the object and that the object is selected for this image point in a segmentation algorithm known per se from the processing of photographs in the visible spectral range becomes.

It can be provided here that a VIS image is recorded in the visible spectral range parallel to the IR image and that the segmentation is carried out for the VIS image and that the segmented object is then recorded on the basis of a correspondence between the IR image and the VIS image IR image is awarded. However, the segmentation preferably takes place directly in the IR image.

It is beneficial if the segmentation is carried out automatically. This can be done, for example, by first defining an image point as a seed point or starting point and a starting model as the area of the IR image (or VIS image) and by defining a structural element, a so-called texon, in the starting model as the building block of the image for which the intensity distribution or other structural feature, when varied over the starting model, is most similar to the overall intensity distribution or the overall structural feature of the starting model. The region outside the starting model in the processed image that has a high similarity to the texon can then be determined, and the region determined in this way can be propagated using a development equation. The model created by the propagation can now be used again as a starting model until convergence of the process can be determined.

In one embodiment of the invention, it can be provided that at least two segmentable objects are provided for selection in the IR image. For example, the sensible possible segmentations in the IR image can first be automatically recognized with the aid of a computer and then offered for selection.

The advantage here is that the user can easily edit critical recording situations in which several objects represent candidates for measurement errors and/or require individual post-processing.

To automate the segmentation, it is advantageous if the segmentation includes at least one feature detection. The advantage here is that information about an object class to which the segmented object belongs can already be obtained during the segmentation. In addition to this information, at least one object-related parameter can be read out in an assignment table, with which the image analysis or image processing step can be parameterized or is parameterized, for example according to a presetting.

It is particularly favorable if the feature detection includes at least one edge and/or corner detection. For automated detection of objects, it can be provided that the segmentation includes at least detection of closed lines or contours. The advantage here is that fully depicted objects can be easily identified using closed lines. It is particularly advantageous if the detection is carried out using the polar log method, which is known from object recognition in visible photography and image processing. In this case, the so-called edge map is transformed into a polar space in order to be able to find the edges of an object more quickly. It is particularly advantageous if the center of the polar log representation is placed in the or an initially selected pixel.

In one embodiment of the invention it can be provided that the IR image is recorded with a thermal imaging camera.

The thermal imaging camera can be equipped with a sensor field (FPA) of bolometers or scanner technology.

A particularly simple further processing of the IR image is made possible if the segmented object is output via an output unit.

For example, it can be provided that the segmented object is displayed on a display unit so that a user can directly perceive or examine the result of an object-related image analysis or image processing step carried out according to the invention for the segmented object.

For example, it can be provided that the image analysis or image processing step includes carrying a cursor. The advantage here is that the movement of an object segmented from the IR image can be tracked via a sequence of IR images.

For example, it can be provided that the image analysis or image processing step includes a calculation of an exact temperature of a relative position related to the segmented object. The advantage here is that the selection of those image points for which the temperatures are not only determined approximately, but precisely using a complex calculation process, can be intelligently tailored to the specific image content.

It can be provided that the image analysis or image processing step includes a selection of a color and/or gray value palette of a false color representation. The advantage here is that large contrasts in the images can be prepared by using a different color palette for particularly hot or particularly cold objects than for the background of the image. Alternatively or additionally, this can be used to hide uninteresting image areas of the IR image or to display them with reduced intensity.

It can also be provided that the image analysis or image processing step includes an input or selection of at least one emissivity and/or reflectance value. The advantage here is that the different IR-optical or radiation-physical properties of the objects displayed can be individually adjusted in the processed IR image.

In a storage unit, the markable objects can be combined into object classes, with object-related information such as emission, reflection and/or transmittance values being stored for each object class.

For example, it can be provided that at least one feature is extracted in a learning mode for the segmented object. In this way, object classes can be created, each of which has its own set of parameters, so that a later segmented object can be automatically assigned to this object class based on extracted features.

The image analysis or image processing step can also include showing or hiding the segmented object or the complement to the segmented object in the recorded IR image.

Other types of image analysis or image processing steps can also be provided.

It can be provided that at least one object-related information about a segmented object is stored in a feature space in one or the learning mode. The advantage here is that object-related information, such as the degree of emission, reflection and/or transmittance, can be assigned to the individual object classes, so that certain settings for image processing can be made automatically for the segmented objects that fall into the respective object classes.

It can also be provided that the image analysis or image processing steps define individual analysis functions, for example the determination of a minimum temperature value, a maximum temperature value, an average value and the like, for the segmented object.

For example, a feature extracted in learning mode can be the geometric shape of the object.

The learning mode can have already been implemented at the factory on a thermal imaging camera or in the software for processing recorded IR images, or the user can also be given the opportunity to work on individual use cases in a learning mode.

To apply the learned rules, it can be provided that in an application mode at least one object-related piece of information is read out in the associated point or area of a feature space for at least one feature extracted from the IR image. For example, this can be set up using a support vector machine. The advantage here is that it is not only possible to decide which object class a segmented object belongs to, but also that the settings and procedures that apply to the associated object are available for the segmented object class to be applied can be selected automatically.

It is particularly advantageous if the dimension of the feature space is set up or expanded in such a way that the object classes can be separated from one another by hyperplanes.

To solve the problem, a thermal imaging camera set of the type mentioned above provides that image processing means are set up for segmenting at least one object in an IR image recorded with the thermal imaging camera. The advantage here is that objects can be distinguished using segmentation, for which an individual image analysis or image processing step can be carried out using the image processing means.

In one embodiment of the invention, it can be provided that the image processing means are designed and set up in a PC connected or connectable to the thermal imaging camera and are operated in online mode or (after the data transmission of the IR images) in offline mode.

However, a particularly compact embodiment of the invention can provide that the image processing means are integrated into the thermal imaging camera. The advantage here is that the recorded IR image can already be processed in the thermal imaging camera.

It can be provided that an output unit has a touch-sensitive display, which is set up to select a segmented and/or segmentable object and/or an image point. The advantage here is that the user can easily specify which object should be segmented. It is particularly advantageous if the output unit is designed on the thermal imaging camera. A further advantage is that when an image point is selected, the image processing means can automatically determine the associated object through segmentation by automatically, i.e. computer-aided, determining those points that are related to the content of the selected image point. The possibility of selecting a segmented and/or segmentable object from several objects enables the user to select between two or more segmented objects in the IR image. The intended image analysis or image processing step can then be carried out for the selected object, while the rest of the IR image remains unprocessed or is processed with another image analysis or image processing step.

For automatic, content-appropriate selection of the image analysis or image processing step, it can be provided that a feature space is set up in a storage unit, the points and/or areas of which can be assigned entries for object-related information. The advantage here is that the object-related information is provided and can be used directly to determine the object-related image analysis or image processing step, for example to select a color palette and/or to determine an analysis program. A storage unit is preferably integrated into the thermal imaging camera.

It is particularly advantageous if means for carrying out a method according to the invention are designed and set up in the thermal imaging camera set according to the invention. This can be achieved, for example, by suitable programming of the image processing means in a data processing unit.

The invention will now be described in more detail using an exemplary embodiment, but is not limited to this exemplary embodiment. Further exemplary embodiments result from combining individual or several features of the protection claims with one another and/or with individual or several features of the exemplary embodiment.

• 1: eine Wärmebildkamera eines erfindungsgemäßen Wärmebildkamera-Sets in einer Schrägansicht von vorn,
• 2: die Wärmebildkamera gemäß 1 in einer Schrägansicht von hinten und
• 3: eine Prinzipdarstellung zur Erläuterung des erfindungsgemäßen Verfahrens.

It shows:

• 1 : a thermal imaging camera of a thermal imaging camera set according to the invention in an oblique view from the front,
• 2 : the thermal imaging camera according to 1 in an oblique view from behind and
• 3 : a schematic representation to explain the method according to the invention.

1 and 2 show in three-dimensional oblique views different views of a thermal imaging camera set according to the invention, designated as a whole by 1, which includes a thermal imaging camera 2.

The thermal imaging camera has, in a manner known per se, IR optics 3 with an IR radiation detector 4 arranged behind the IR optics 3 inside the thermal imaging camera 2.

The thermal imaging camera 2 is therefore equipped with appropriate evaluation electronics for recording IR images 5 (cf. 3 ) furnished.

Inside the thermal imaging camera 2, an image processing means 6 is designed and set up in a data processing unit.

The image processing means 6 is set up by programming so that objects in a recorded IR image 5 can be segmented.

To carry out the method according to the invention, an IR image 5 is first recorded with the thermal imaging camera 2, which is in 3 is shown as an example.

The IR image 5 represents a scene 6 with several objects 7, 8 in a false color representation, with a color and/or gray value palette 9 being used to calculate the false color values.

The objects 7, 8 shown in the IR image 5 require processing of the object 7 due to the selected color and/or gray value palette 9 in the IR image 5, while the objects 8, for example, do not require any further post-processing or are not relevant for further processing .

Therefore, in a segmentation step 10, the object 7 is automatically segmented.

This happens because the IR image 5 is first displayed on a display unit 11 and a user selects an image point 12, which the user can assign to the desired object 7 in terms of content.

The selection can be made, for example, by the display unit 11 (cf. 2 ) is designed as a touch-sensitive display, and the user selects the pixel 12 by correspondingly touching the display unit 11 at the corresponding position.

Alternatively or additionally, a cursor 13 with operating elements 14 (cf. 2 ) can be moved to the position of the pixel 12, and the pixel 12 located under the cursor 13 can be selected.

The image processing means 6 in the thermal imaging camera 2 now segments the object 7 belonging to the image point 12 with the aid of a computer by automatically, i.e. with the aid of a computer, determining a closed contour 16 which surrounds the object 7 in the IR image 5.

This can be done, for example, by searching for edges 17 in the IR image 5 using edge detection and by assembling the detected edges 17 to form closed contours 16.

In this case, the object 7 to be segmented results from the contour 16 in whose interior the previously selected image point 12 lies.

At least one image analysis or image processing step 18 is now carried out for the segmented object 7, while the other objects 8, which were not made available for further processing, are left unchanged or even reduced in display intensity.

In the illustrated embodiment according to 3 The image analysis or image processing step 18 consists in selecting a changed color and/or gray value palette 19, which is more suitable for the object 7 than the color and/or gray value palette 9.

For many applications it is useful that the parts of the IR image that are not subject to the image analysis or image processing step 18, in particular the objects 8, are displayed or output together with the processed segmented object 7.

Instead of or in addition to the selection of a color and/or gray value palette 19, a changed emissivity value and/or a changed reflectance value can also be used for the object 7, with which or with which a changed temperature value results for the object 7 compared to the initial value.

It can also be provided that a different magnification factor and/or a different brightness value is used for the object 7 than for the non-segmented image portion of the IR image 5.

The IR image 5 prepared in this way is then output again on the display unit 11.

In the thermal imaging camera 2 (or in further exemplary embodiments in a connected PC) there is also a storage unit 20 in which a feature space is set up, the points and / or areas of which are occupied by entries for object-related information.

The image processing means 6 extracts a predetermined number and type of features for the segmented object 7 and determines which object classes in a specified feature space belong to the set of extracted features.

The object-related information stored for the object class, for example an object-specific color and/or gray value palette, a magnification factor, a brightness or transparency value or an analysis program to be executed, is then automatically extracted from an assignment table is read out, and the image processing means automatically applies this information to the segmented object 7 in an image analysis or image processing step 18.

If the thermal imaging camera 2 is moved to take new images, the object 7 moves over the IR image 5.

The image processing means 6 recognizes this movement and tracks the cursor 13 so that its relative position in relation to the object 7 being displayed always assumes a fixed value.

Thus, for example, at the image position 12 of the cursor 13, the temperature of the segmented object 7 can be tracked over several IR images 5 of a video sequence, even if the object 7 moves or changes between the individual IR images 5

In the thermal imaging camera set 1, which comprises at least one thermal imaging camera 2 and an image processing means 6, a method for image analysis and/or image processing of an IR image 5 recorded with the thermal imaging camera 2 is set up, in which a selected object 7 in the IR Image 5 is automatically segmented with the aid of a computer and is subjected to an object-related image analysis or image processing step 18.

Claims (10)

Method for image analysis and/or image processing of an IR image (5), wherein at least one object (7, 8) is distinguished in the IR image (5) by means of segmentation (10) and made available for further processing and for the segmented object (7) at least one further image analysis or image processing step (18) is carried out, characterized in that - in a learning mode for the segmented object (7) at least one feature is extracted and in the learning mode in a feature space at least one object-related information about the segmented object (7) is stored, whereby object-related information is assigned to individual object classes, so that certain settings for image processing are automatically made for the segmented objects that fall into the respective object classes, - and that in an application mode to at least one of the IR image (5) extracted feature at least one object-related information in the associated point or area of the feature space is read out and automatically selected for setting for the image processing. Procedure according to Claim 1 , characterized in that the segmentation (10) includes the selection of at least one image point (12) in the IR image (5). Procedure according to Claim 1 or 2 , characterized in that the segmentation (10) is carried out automatically and/or that at least two segmentable objects (7, 8) are provided for selection in the IR image (5). Procedure according to one of the Claims 1 until 3 , characterized in that the segmentation (10) comprises at least one feature detection, in particular at least one edge and/or corner detection, and/or that the segmentation (10) comprises at least one detection of closed lines or contours (16), in particular according to the polar Log method includes. Procedure according to one of the Claims 1 until 4 , characterized in that the IR image (5) is recorded with a thermal imaging camera (2) and / or that the segmented object (7) is output via an output unit, in particular displayed on a display unit (11). Procedure according to one of the Claims 1 until 5 , characterized in that the image analysis or image processing step (18) includes carrying a cursor (13) and / or that the image analysis or image processing step (18) includes a calculation of an exact temperature at at least one relative position related to the segmented object (7). includes. Procedure according to one of the Claims 1 until 6 , characterized in that the image analysis or image processing step (18) comprises a selection of a color and/or gray value palette (9, 19) of a false color representation and/or that the image analysis or image processing step (18) includes an input or selection of at least one emission - and/or reflectance value. Thermal imaging camera set (1), with a thermal imaging camera (2) and image processing means (6), characterized in that the image processing means (6) for segmenting (10) at least one object (7) in an IR image recorded with the thermal imaging camera (2). Figure (5) is set up and that means for carrying out the method according to one of the Claims 1 until 7 are trained. Thermal imaging camera set (1). Claim 8 , characterized in that the image processing means (6) are integrated into the thermal imaging camera (2) and / or that an output unit (11) has a touch-sensitive display, which is used to select a segmented and / or segmentable object (7) and / or an image point (12) is set up. Thermal imaging camera set (1). Claim 8 or 9 , characterized in that a feature space is set up in a storage unit (20), preferably integrated into the thermal imaging camera (2), the points and/or areas of which can be assigned entries for object-related information.

Images