DE102011018263B4 - Process for the pulse thermographic quality and error classification of a joint connection such as a structural bonding in the automotive bodywork sector - Google Patents

Abstract

Process for the pulse thermographic quality and defect classification of a joint, such as structural bonding in the vehicle body area, in which a joined component (joint sample) is subjected to non-destructive pulse thermographic testing for defects by exciting the joint sample using ultrasound and the transient heating and / or cooling process in the joint sample is thermographically recorded by means of a thermographic camera before, during and after its excitation and alternative representations of the thermographic film are calculated by means of a computer connected to the thermographic camera, and in which the following process steps are also carried out: in a learning phase (calibration) (1st step to 10. Step), after the joining sample has been selected (1st step), from the thermographic film (2nd step) recorded by the thermographic camera in a pulse thermographic test of the selected joining sample (1st step), computer-standardized films based on time as differences Reference film, time-standardized film and locally standardized film are generated, then in addition to the images (representations) of the original film, the difference film and the standardized films from the data of the original film, the difference film and the data of the standardized films generated using FFT, HKA and NMF generated additional representations of the thermographic film (3rd Step), whereby the number of additionally generated representations depends on the respective procedure, the obtained alternative representations are stored in a comparator stage downstream of the computer (3rd step), then a horizontal section of the same joining sample is mechanically produced as a destructively tested reference, from which the The desired defects in the joint sample are to be detected optically (4th step), then with reproduced clamping of the horizontal section and reproduced positioning of the thermographic camera in the identical process structure using the thermographic camera, a thermogram (individual thermographic image) of the identical section of the destroyed joint sample is created with the same spatial resolution (5th step), and the thermographic film of this section is registered in the computer downstream of the thermographic camera, ...

Description

The invention relates to a method for pulse thermographic quality and error classification of a joint connection such as structural bonding in the motor vehicle body area, in which an assembled component (joining sample) is investigated by impulse thermometry nondestructive error by the joining sample excited by ultrasonic pulse and the transient heating and / or Cooling process in the joining sample by means of a thermographic camera before, during and after its excitation thermographically detected and using a computer connected to the thermographic camera alternative representations of the thermographic film are calculated.

Due to its property of being able to detect structures close to the surface being inspected, thermography, such as pulse thermography as an imaging measurement method, has great potential in non-destructive testing (zfP) of thin-walled structures, as described, for example, in US Pat. B. used in the automotive industry, namely z. B. as a non-destructive test method of welds and laser welds.

In pulse thermography, a body in thermal equilibrium is excited by a heat pulse, whereby the heat balance processes also taking place on its surface are detected with the thermographic camera. In this way, a time course is recorded for each pixel of the thermographic film, which qualitatively reflects the temperature profile at the surface of the body. These balancing procedures follow Fourier's law of heat conduction

den Wärmefluss
λ
die Wärmeleitfähigkeit und
T(x →)
die Temperatur.In this equation represent:

the heat flow
λ
the thermal conductivity and
T (x →)
the temperature.

If the body has inhomogeneities with regard to the material properties λ, this influences the heat propagation in the body and thus also the heat distribution visible on the surface of the thermographic camera. For thermal excitation z. B. photo flash, defocused laser, induction, cold or hot air or ultrasound used. Added to this is the thermographic inspection of adhesive seams.

Gluing is increasingly used in the automotive industry in combination with other joining methods such. B. when seam gluing or spot welding. Thus, the aim is to increase the structural strength of the component or to ensure the tightness of the compound through the use of epoxy adhesives.

By means of ultrasound-excited lockin thermography it is possible to use adhesive seams for air pockets, imperfections and conditional on connection errors such. B. explore and characterize kissing bonds. Especially in the determination of imperfections and air pockets is exploited that air, adhesive and steel attenuate the excited mechanical vibrations different degrees and thus convert varying degrees of mechanical energy into heat energy.

In order to realize an automated, nondestructive testing, it is necessary to reduce the entire information of the thermographic film step by step to the information error-free or error-prone. Thus, in a first step, the thermographic film is reduced to a single, information-bearing image, i. H. the representation of the film is made. In order to determine such representation, methods are used, which are partly based on physical approaches and partly based on heuristic approaches.

In physical approaches are, for. For example, the Fourier analysis in Lockin- and pulse thermography or the so-called. Thermal Signal Reconstruction.

In determining alternative representations of the thermographic film, Fourier Analysis (FFT), Principal Component Analysis (HKA), and Non-Negative Matrix Factorization (NMF) can be applied to all representations based on time.

The above-described, for each pixel associated time histories of the thermographic film can be represented in various basic systems, eg. B .: over time, spectral, in terms of the main components and based on a base system, which is determined by means of NMF, these representations are partly to be combined. The representations of the thermographic film in these cases represent all coefficient images belonging to the respective base.

From the DE 10 2009 031 605 A1 For example, there is known a method of classifying the quality of spot welds of galvanized sheet components by pulsed thermography, such as induction thermography, of the finish welded component in the area of the weld point, which is normally a nugget as a molten zone and a zinc fringe around it unmelted zone exists, wherein the surface temperature distribution, which results from the resulting heat flow in the component on the side facing away from the device, is thermally imaged by a thermographic camera in their local and temporal course, and evaluated by a coupled with the thermographic camera computer, the detected intensity time courses become. Here, the weld nugget diameter D _SL is calculated according to the following formula: D _SL = C ₀ + C ₁ · D · _SH + C ₂ H + C _BI + C ₃ · H _{RP 4} · D · _SH H _BI + C ₅ · E _OK where represent
D _{SL is} the nugget diameter,
D _{SH is} the thermographic diameter of weld nugget and zinc fringe,
H _BI the sheet spacing in the area of the weld nugget,
H _{RP is} the residual point thickness, ie the original sheet thickness of the galvanized sheets reduced by the sinking depth of the welding electrodes,
E _OK the surface emissivity of the side of the welding point away from the excitation,
C ₀ to C ₅ constants.

• – der thermografische Durchmesser D_SH von Schweißlinse und Zinksaum, indem die Zeitverläufe der Impulsintensität aus dem Thermofilm auf Standardnormalverteilung normiert werden, nach Maskierung von Bereichen, die weniger als sämtliche Blechlagen enthalten, eine Hauptkomponentenanalyse für die normierten Zeitverläufe der Impulsintensität erfolgt, dann ein binäres Bild mittels Schwellwertanalyse auf dem Koeffizientenbild der ersten Hauptkomponente bestimmt wird und daraus durch Hough-Transformation der thermografische Durchmesser D_SH von Schweißlinse und Zinksaum ermittelt wird,
• – der Blechabstand H_BI im Bereich der Schweißlinse, indem das Koeffizientenbild der ersten Hauptkomponente erstellt und die Koeffizienten außerhalb des Schweißpunktes und außerhalb der Maskierung gemittelt werden, wobei der erhaltene Mittelwert mit dem Blechabstand H_BI in diesem Bereich korreliert,
• – die Restpunktdicke H_RP, indem über die Zeitverläufe der Impulsintensität des Thermografiefilms Radius und Lage von Schweißlinse und Zinksaum auf Standardnormalverteilung normiert werden, in diesem Bereich ein repräsentativer Zeitverlauf der Impulsintensität durch Mittelung oder Wahl des Zeitverlaufes der Impulsintensität bestimmt wird, bei dem das Maximum zuerst erreicht wird, und dann die Totzeit berechnet wird, deren Wert mit der Restpunktdicke H_RP korreliert,
• – die Oberflächenemissivität E_OK der anregungsabgewandten Seite des Schweißpunktes, indem auf dem Koeffizientenbild zur ersten Hauptkomponente eine Schwellwertanalyse zur Bestimmung eines Bereiches durchgeführt wird, auf dem die Oberflächenemissivität auszuwerten ist, dann die Bestimmung der Maxima der Zeitverläufe der Impulsintensität in diesem Bereich erfolgt und über die Mittelung der Maxima die Kenngröße E_OK als Oberflächenemissivität erhalten wird, und
• – die Konstanten C₀ bis C₅ bestimmt werden, indem zuvor der Schweißpunkt eines Referenzbauteils aus verzinkten Blechen zerstört und der Referenzschweißlinsendurchmesser R_SL-RE gemessen und die mittlere quadratische Abweichung von D_SL zum gemessenen Wert D_SL-RE aus der zerstörenden Prüfung minimiert wird.

The parameters _D.sub.SH , _H.sub.BI , _H.sub.RP and _E.sub.OK are determined in advance via the time profiles of the pulse intensity from the thermographic film in each case as follows:

• - the thermographic diameter D _SH of weld nugget and zinc fringe, by normalizing the time-lapse of the momentum intensity from the thermofilm to standard normal distribution, after masking areas containing less than all sheet layers, a principal component analysis for the normalized pulse intensity timings, then a binary image is determined by means of threshold value analysis on the coefficient image of the first main component and the thermographic diameter D _SH of the weld nugget and zinc seam is determined therefrom by Hough transformation,
• The sheet spacing H _BI in the region of the weld nugget, by creating the coefficient image of the first main component and averaging the coefficients outside the weld point and outside the mask, the average obtained being correlated with the sheet spacing H _BI in this region,
• - the residual point thickness H _RP by normalizing over the time profiles of the pulse intensity of the thermographic film radius and position of weld nugget and zinc fringe on standard normal distribution, in this area a representative time course of the pulse intensity is determined by averaging or choice of the time course of the pulse intensity, where the maximum first is reached, and then the dead time is calculated, the value of which correlates to the residual point thickness H _RP ,
• The surface emissivity E _{OK of} the side of the welding point away from the excitation, by performing a threshold value analysis on the coefficient image for the first main component to determine a region on which the surface emissivity is to be evaluated, then determining the maxima of the pulse intensity variations in this region and Averaging the maxima the characteristic E _{OK is obtained} as surface emissivity, and
• - the constants C ₀ to C _{5 are} determined by previously destroying the welding point of a reference component made of galvanized sheets and the reference Welding lens diameter R _SL-RE measured and the mean square deviation of D _{SL is} minimized to the measured value D _SL-RE from the destructive test ,

Subsequently, the formula calculated weld nugget diameter D _{SL is} classified as:
i. O. if it results: D _RL ≥ D _SLmin or
ni O. if it results: D _SL <D _SLmin

Furthermore, from the DE 10 2006 061 794 B3 a method for the automatic non-contact and non-destructive testing of a welded joint at least second joining partners made of the same or different materials, in particular a welding point known. The weld point typically has a two-zone joint consisting of a molten zone called a weld nugget and a non-molten zone around the weld nugget called a weld adhesive. The method is designed as an examination of infrared images, so that a test object is excited by at least one source and the resulting heat flow is detected by at least one infrared sensor in a series of thermal images. The thermal images and the result images of various types obtained from the image series, which represent a heat flux in transmission and / or in reflection with temporal and spatial resolution, are investigated. Here, the resulting weld nugget in a result image representing the heat flow dynamics through the weld joint is determined as a range of regions of heat flow dynamics such that these regions consist of pixels above a dynamic threshold that is varied from the maximum to the minimum threshold the maximum threshold value corresponds to the peak value of the heat flow dynamics and the minimum threshold value lies above the heat flow dynamics of the background image. In this way, these areas of heat flow dynamics are examined after an abrupt increase in their circumference, which indicates the desired weld nugget, which is then evaluated for position and size.

The invention is therefore based on the object to provide a method of the type mentioned, with which effectively the quality and error classification of a joint connection such as a Strukturverklebung in the automotive sector is automated in mass production.

This object is achieved by the method of the type mentioned, in which the following further process steps are carried out:
In a learning phase (calibration) (1st step to 10th step), after selection of the joining sample (1st step) from the thermographic film recorded by the thermographic camera in a pulse thermographic examination (2nd step) of the selected joining sample (1st step) normalized films based on time as differential film, time normalized film and locally normalized film,
then, by means of the computer, in addition to the images (representations) of the original film, the difference film and the normalized films from the data of the original film, the difference film and the data of the produced normalized films by means of FFT, HKA and NMF, additional representations of the thermographic film are produced (3. Step), wherein the number of additionally generated representations depends on the respective procedure,
the obtained alternative representations are stored in a computer downstream comparator stage (3rd step),
Subsequently, as a destructively tested reference, a mechanical horizontal grinding of the same joining sample is produced, from which the sought defects in the joining sample are to be optically detected (4th step),
Then, with reproduced clamping of the horizontal grinding and reproduced positioning of the thermographic camera in the identical process structure by means of the thermographic camera, a thermogram (individual thermographic image) of the identical section of the destroyed joint sample at the same spatial resolution created (5th step), and the thermographic film of this section in the thermographic camera registered downstream computer,
the thus existing in the same position and resolution thermogram of Horizontalholzschliffs is used as a reference for a pixel-by-pixel classification of the thermographic film, wherein the reference each pixel of the thermographic film of one of the classes i. O., ni O. or "area not to be considered" (step 6),
the reference thus generated is referenced to the comparator stage and compared with all of the already plotted representations of the thermographic film of the joining sample to identify the thermographically searched for errored and error free areas of the representations by setting a threshold value for each representation of the thermographic film to distinguish them in the reference i. O. or ni O. pixels, the pixels not to be taken into account in this optimization and by counting the misclassified pixels and the threshold value is changed until the number of misclassified pixels is minimal and thereby the distinctness between intact and non-intact areas is optimized (7th step),
then the n (10) best representations are determined for the criterion of misclassified regions (the number of misclassified pixels) (8th step) and
all possible combinations of these n best representations are formed in such a way that for each combination a vector is assigned to each pixel whose coordinates contain the coefficients of the respective representation that belong to the pixel, averaged and scattered,
whereby sets of feature vectors of associated pixels (point clouds) are formed having as coordinates the coefficients of individual representations and subjected to a cluster analysis in which two clusters with respective cluster centroid i. O. (C ₁ ) and cluster centroid ni O. (C ₂ ) are determined, wherein the distance of each vector to the cluster centroid i. O. (C ₁ ) and to the cluster focus ni O. (C ₂ ) represents the fused representation of the thermographic film, which is evaluated for all possible combinations analogous to the individual representations by means of the number of misclassified pixels, and
wherein by forming a ranking of the individual representations and the merged representations corresponding to the misclassified area, the optimal representation is obtained (9th step) and the result in terms of an algorithm applied to the thermographic film of the joints, the indication of the individual representations to be merged and the threshold for the size D = l ₁ -l ₂ (difference of the Euclidean distances l ₁ and l ₂ to the center of gravity i.O. (C ₁ ) and to the center of gravity ni O. (C ₂ ) for each pixel) (step 10), and inasmuch as representations of HKA and NMF belong to these individual representations, also the corresponding analysis functions are used in the algorithm, and
following the learning phase (calibration) (1st step to 10th step), an as yet unclassified sample is selected in an automatic application phase (11th step) and, by analogy with the impulse thermographic examination of the learning phase (calibration), checked by pulse thermography and a thermographic film of this sample is created (12th step), from which the representations determined before (10th step) are calculated and combined into vectors (corresponding to the 9th step) (13th step), whereupon each individual pixel of this optimal representation is determined by means of the determined threshold value (10th step). Step) as i. O. or ni O. is classified (13th step), and thus results in the locally classified joining sample (14th step).

The 10 best representations are determined based on the misclassified area and joined by cluster analysis.

Preferably, the determination of all n ² - 1 fused representations and the associated focus i. O. (C ₁ ) and the associated center of gravity ni O. (C ₂ ) by means of known per se k-means Algorthmus,
wherein for evaluating each fused representation for each pixel, the Euclidean distances l ₁ and l ₂ are focussed i. O. (C ₁ ) or to the focus ni O. (C ₂ ) are determined,
then by subtracting the two distances l ₁ and l ₂ a size D = l ₁ - l _{2 is} formed, which is negative if the respective pixel closer to the center of gravity i. O. (C ₁ ), and is positive if the respective pixel is closer to the center of gravity ni O. (C ₂ ),
for each fused representation, the threshold value is determined and then it is counted how many pixels are considered to be intact or defective and thus the misclassified area is determined,
then, if this has been done for all n ² -1 representations, a ranking of all fused and single representations is made based on the misclassified area, and
then the obtained optimal threshold value for the quantity D = l ₁ -l _{2 is} applied to the optimal representation of the thermographic film of the joining sample detected by the thermographic camera, whereby for each pixel whose representation the result "adhesive present" or "adhesive not present" automatically results.

Preferably, several references are processed simultaneously to determine the representations to be merged and the threshold value for the quantity D = l ₁ -l ₂ .

Furthermore, a combination of the learning phase and the application phase and thus the integration of HKA and NMF can be carried out by also processing an unweighted thermographic film in addition to one or more references, which is assigned to the non-evaluated area during the evaluation and thus still in the calculation of the Analysis functions of HKA and NMF, but not in the determination of the optimum threshold value for each individual representation or the threshold value for the quantity D = l ₁ -l ₂ and subsequent classification of each pixel of the unweighted thermographic film on the basis of the found fusion representations and the threshold value.

Preferably, the image of the horizontal cut of the joining sample (step 1) is taken with any camera having a different resolution than the thermographic camera and a different distance and a different orientation for horizontal grinding of the joining sample as the thermographic camera, wherein an algorithmic orientation of the image destroyed joining sample to the thermographic film as well as the conversion of this image into the cutout and the resolution of the thermographic film done.

With the inventive method is with the help of ultrasound excited thermography and an evaluation of the resulting thermographic films using FFT, HKA and NMF in subsequent fusion of thermographic representations guaranteed to detect non-destructive areas missing adhesive automatically, especially in mass production.

The invention will now be explained with reference to the drawings. In these are:

1a a schematic experimental setup for ultrasonically excited thermography of a sample of two steel sheets,

1b a top view of the sample of the experimental setup according to 1a .

2a to 2d Representations for evaluation of the thermographic film by comparison of original data and the phase of the first Fourier coefficient,

3a to 3d Diagrams from which result images and time courses to pre-processing methods of the thermographic film emerge,

4a to 4d Phase and amplitude images of an FFT based on the same thermographic film but different preprocessing,

5a to 5d Result images and analysis functions of the thermographic film to the second coefficient of the HKA after different preprocessing,

6a to 6d Result images and analysis functions of the thermographic film to the second coefficient of the NMF after different preprocessing,

7a a photograph of the destroyed sample,

7b a binary image from the photograph according to 7a .

7c a binary image from the photograph according to 8a including regions of interest (ROIs),

7d above a representation of the thermographic film with reference classification and below the binary image after calculation of the optimal threshold,

8a to 8c Representations of analysis functions to the 3. coefficients of the HKA applied to T _ij (t _m ) of three thermographic films,

8d and 8e Illustrations of analysis functions resulting from the merging of the thermographic films of Samples 1 and 2 or 1 and 3,

9 a diagram for the evaluation of representations, wherein the amplitude of the 2nd coefficient of the FFT on time-normalized data and the abscissa of the amplitude of the 1st coefficient of the FFT are formed on temporally normalized data,

10 a representation of the classification of the adhesive seam by means of individual representations of the thermographic film,

11 a representation of the result of the cluster analysis,

12a to 12c Representations comparing classifications,

13 one of the 9 similar representation of the fused 1st and 2nd HKA of T _ij ^Δ (t _m ) for a section of the representations as sets of feature vectors of assigned pixels (point cloud),

14a a diagram showing the separation of the two areas (intact area, area "missing adhesive") by means of dk-means algorithm,

14b a schematic representation for the calculation of the size D,

15 a flowchart showing a first embodiment of the method according to the invention and

16 a flowchart showing a second embodiment of the method according to the invention.

1a schematically shows a test setup for ultrasound-excited thermography, wherein in section a side view of a sample 1 You can see that by means of an ultrasonic converter 2 an ultrasonic excitation is exposed.

The sample 1 exists, like 1b clarified, from two overlapping steel sheets 3 , where the overlap of the joint partners is 15 mm. The material of the two steel sheets is H320LA and the dimensions of the latter are: 200 mm × 50 mm × 1 mm.

The two overlapping steel sheets 3 are in the overlap area at three points 4 welded and at the same time glued together by means of an epoxy adhesive, the width of the adhesive strip is variable up to 15 mm. On the surface of the sample opposite to the excitation 1 is a thermography camera 5 having a recording frequency of 56 Hz, an integration time of 3 ms and a recording time of 3.1 sec. This corresponds to 176 images with a resolution of 384 × 56 pixels with a visible width of 29 mm.

For evaluation of the thermographic sequences, phase images are generated by means of Fourier analysis, which have a much stronger drawing of defects than is the case in the original data.

The 2a to 2d show representations for the evaluation of the thermographic film by comparing original data and the phase of the first Fourier coefficient.

In order to classify components for automated quality assurance based on such representations with regard to intact or defective, either image processing approaches are necessary or representations are to be determined which have an even clearer characterization of errors.

In this context, on the 3a to 3d from which result images and time courses are produced for different preprocessing methods (original film, differential film, temporally normalized film, locally normalized film) of the thermographic film.

In addition to the presentation of the original data on the basis of time, the representation on the basis of time provides three further possibilities of depicting the thermographic film over time, namely from the thermographic film of the joining sample recorded by the thermographic camera with normalized films on the basis of time as difference film, temporally normalized film and locally normalized film produced.

In the representation difference to image one, the first image of each image of the thermographic film is subtracted. In this way, the influence of emission differences and reflections should be reduced.

In the production of the time-normalized film (representation normalization over time), the temperature profiles, for non-calibrated images, the gradients of the digital levels are normalized to all pixels to a standard normal distribution. For this purpose, its mean value μ _ij and its standard deviation σ _{ij are} calculated for each time course x _ij (t) and the normalized results
x zn / ij (T)
are calculated as follows:

In this way, as in the case of the difference film, the influence of emission differences on the joining sample is reduced since the time-normalized film no longer has any amplitude information. All time courses now have an average value
μ ~ _ij = 0
and one standard deviation
σ ~ _ij = 1

Especially at time lapses with a low temperature lift, this leads to the amplification of the noise, with constant signal-to-noise ratio, as from the 3a to 3d is apparent.

Analogous to the procedure described above, in the representation normalization over the location, each image of the thermographic film is normalized to a standard normal distribution. The time courses
x on / ij (T)
now have different mean values and scatters, but for every image of the thermographic film:
Average
μ ~ (t) = 0
and standard deviation

In particular, in thermographic films with very small local differences in the temperature increase in this way an enhancement of these differences can be achieved, as from 3d can be seen.

• – FFT beim Differenzfilm: Amplitude des 0. Koeffizienten (konstanter Anteil)
• – FFT bei zeitlich normierten Daten: Amplitude sämtlicher Koeffizienten
• – FFT bei örtlich normierten Daten: Amplitude und Phase sämtlicher Koeffizienten

As far as the spectral representations are concerned, the Fourier analysis (FFT) is to be applied to all representations in the time domain. The results of these evaluations of the thermographic film can be regarded as phase and amplitude images. It should be noted that deviations from the results of the FFT applied to the original film are only to be expected for the following representations:

• - FFT at the difference film: amplitude of the 0th coefficient (constant proportion)
• - FFT for temporally normalized data: amplitude of all coefficients
• FFT for locally normalized data: amplitude and phase of all coefficients

In this context, on the 4a to 4d from which examples of phase and amplitude images of an FFT on the basis of the same thermographic film with different preprocessing (original data, time-normalized data, spatially normalized data) emerge.

Like Fourier Analysis (FFT), Principal Component Analysis (HKA) can be applied to all representations. In contrast to the FFT, the analysis functions are not specified in the HKA but calculated as eigenvectors of the covariance matrix of the thermographic film. From the 5a to 5d For example, result images and analysis functions of the second coefficient of HKA are given in the same thermographic film but with different preprocessings (HKA, original data, K2, HKA, differential film, K1, HKA, time normalized film, K1, HKA, locally normalized film, K2).

In the generation of representations on the basis of the NMF, like the HKA, it computes its analysis functions from the thermographic film, but not by solving an eigenvalue problem, but by solving a nonlinear optimization problem. NMF is applicable only to matrices with positive coordinates. Therefore, it is necessary to adjust the representation to be analyzed, if necessary, by offsetting the values of the entire thermographic film. By means of NMF positive approach functions are determined which optimally approximate the time courses to all pixels as a positively weighted sum.

This is therefore a nested optimization problem for determining suitable approach functions for which suitable weights are also sought. Since the quality function to be minimized is non-linear and usually has several minima, it can be assumed that it is not the global but a local optimum that is found.

Usually, the analyzed data set can not be exactly reconstructed from the coefficients and the mapping functions, even if the same number of mapping functions as in FFT or HAK is chosen.

The 6a to 6d show by way of example results images and analysis functions for the second coefficient of the NMF in the same movie, but for different preprocessing, namely 6a NMF, O- Data, K1; 6b NMF, difference film, K2; 4c NMF, temporally normalized film, K1; NMF, localized film, K2, 6d ,

FFT, HKA and NMF are applied to all representations based on time. These are the following:
Original film: T _ij (t _m ),
Difference to the first picture of the film:
T Δ / ij (t _m ) = T _ij (t _m ) - T _ij (t ₁ ),
Standardization of time courses with mean and variance:

Standardization of the individual images with mean and variance:

In the above formulas are:
T is a temperature or radiation intensity, measured as a digital level (DL).
i and j run indices of the position of the considered pixel.
m the running index of the time
T ~ _ij the mean of the time course to pixel ij
T ~ (t _m ) is the mean value of an image at time t _m
σ _ij and σ (t _m ) the corresponding scattering

In this way, a total of 2,124 partially redundant representations of the thermographic film result for a film with 176 images, in each case 176 thermograms for four representations on the basis of which the FFT applies 176 phase and amplitude images, the HAK 176 in each case Coefficient images and the NMF - three coefficient images each. With this multitude of partly redundant representations, a method is necessary to detect the defects sought, to evaluate them and to connect them on the basis of the best to new more meaningful representations.

In order to evaluate these in terms of their suitability to detect the missing defects in this multiplicity of representations and to combine them on the basis of the best representations to form new, more meaningful representations, a fusion method based on pixel-level cluster analysis is used in accordance with the invention an algorithm is used, which is based on a fusion of the information-bearing representations. What these are is learned on a reference sample, in which the optimal classification by a destructive test by means of horizontal grinding is known.

From a reference sample in which the optimal classification z. B. by a destructive examination by means of horizontal grinding in experience, the information-bearing representations are determined.

Thus, as a reference to be tested destructively a horizontal grinding of the same joining sample is produced, from which the sought defects in the joining sample are to be optically detected. From the destroyed sample, the thermally searched error-prone and error-free regions are identified and provided as a reference for classification in the form of a binary image. In order to ensure that both the same section is captured and the same resolution of the image is present, the identical section of the destroyed joint sample is created at the same local resolution with reproduced clamping of the horizontal section and reproduced positioning of the thermographic camera in the identical experimental setup by means of the thermographic camera , Based on these images, so-called Region Of Interest (ROIs) can be defined and error classes assigned.

From the 7a to 7c Illustrations of an adhesive sample, the derived binary image as well as the binary image including so-called ROIs emerge.

So shows 7a a representation of the destroyed sample at the same section as in the thermographic film, with areas are shown with and without adhesive, pores and welds.

7b shows the binary image from the illustration according to 7a , at the same resolution as in the thermographic film.

7c returns the binary image with the ROIs, with the intact areas and the areas with missing glue. Since the representation is slightly shifted and rotated relative to the thermographic film, the ROIs save the transition areas between defective and intact areas.

In order to evaluate each thermographic representation in terms of its information content, a pixel-by-pixel comparison with the destructive test takes place. For this purpose, an optimal threshold value for the respective representation is determined, so that the number of incorrectly classified pixels and thus the misclassified area becomes minimal. This area is used for a ranking of all representations on the basis of which the selection of representations to be merged takes place.

7d Figure 11 shows above a representation of the thermographic film with reference classification, where the hatched areas represent the so-called "region of interest" (ROI) in which adhesive is present, and the dotted ROIs represent areas in which no adhesive is present.

Out 7d below, a binary image emerges after calculating the optimum threshold, with the misclassified area being 8.25% of the area within the ROI.

Ideally, intact faulty areas can be separated by thresholding. As a rule, however, there are overlaps of intact and faulty areas. Pixels in this overlapping area can not be clearly classified. However, there is always a threshold that minimizes the number of misclassified pixels. The number of misclassified pixels corresponds to an area which in turn can be set in relation to the total area of the thermographically detected area. In 9 For example, these boundaries are diagrammed for two superimposed representations, the ordinates of the amplitude of the second coefficient of the FFT being based on time-normalized data and the abscissa of the amplitude of the first coefficient of the FFT being based on time-normalized data. In the middle of the diagram after 9 the amplitudes of the respective amplitudes are represented as sets of feature vectors of associated pixels (point cloud) having as co-ordinates the coefficients of individual representations, the optimal boundaries for both representations being plotted as follows: K1: 6.96% misclassified area; K2: 18.7% miscategorized area.

The minimum misclassified area is used for a ranking of all representations. Based on this ranking, the selection of merging representations takes place.

Since HKA and NFM do not specify the analysis functions as in the FFT, but rather calculate them from the representations on the basis of time, they can differ from film to film, as can the 8a to 8c showing analysis functions of the 3rd coefficient of the HKA applied to T _ij (t _m ) of three films. While the analysis functions for samples 1 and 2 are very similar, the analysis function for sample 3 shows a clear difference to the other two.

The coefficient images (representations) B _k are linked in the procedures with the analysis functions A _l and the representations of the thermographic film on the basis of F _kl via the equation B _k = F _kl A _l . To carry out HKA and NMF, the thermographic film is converted into a two-dimensional representation. For this reason, the thermographic film in this case has only two indices (pixel number and time) and the image only one index (pixel number).

The coefficient images - the representations - thus change depending on the analysis function from film to film, for whatever reason their position in the aforementioned order of precedence can change. Therefore, the selection of the information-carrying representations can not be made in a pre-learning phase, but the selection must be decided again for each thermographic film. To ensure this, reference to reference must be re-established in each evaluation of a movie. For this purpose, HKA and NMF are always applied to two films simultaneously, namely the film to be classified and the reference film for which the optimum classification result is known. This makes it possible to ensure that the reference and the film to be classified are evaluated with the same analysis functions. In this context is on the 8d and 8e which show analysis functions resulting from the merging of the films of Samples 1 and 2 and Samples 1 and 3, respectively. The above ranking is then based on the known classification target of the reference, and now that the analysis functions for reference and movie to be analyzed match, for both movies.

With the help of this review were how 10 shows, which determines the eight best representations. To generate the binary images, the optimal limits were used as thresholds for binarizing the respective representation. 10 proves that the least misclassification takes place on the basis of the second coefficient of the NMF, since the misclassified area here c. 6.69% of the total area.

Are the binary images in 10 compared with the reference (target) obtained from the destroyed test, it can be seen that the methods for generating the binary images are differentially sensitive in different regions of the sample. Thus, in the first, second and eighth representations according to 10 The spot welds are much easier to spot than in the rest of the representations, while many of the other representations allow a much better sketch of the outline of the defect-free areas and are less noisy.

In order to merge any number of representations on a pixel-by-pixel basis and in order to determine a new representation from a multi-dimensional representation, the procedure according to the invention proceeds as follows.

Each pixel is assigned a value in each representation. For the fusion of representations, these values are taken for each pixel as the coordinates of a point. In this way, the entire film can be represented as multi-dimensional sets of feature vectors of associated pixels (point cloud) having as coordinates individual coefficient representations, as shown 13 can be seen, in which the merger of two representations is shown.

If the two representations are considered in isolation by projecting the sets of feature vectors of associated pixels (point clouds) onto the respective coordinate axis, then both representations have overlaps of areas of missing glue and areas where glue is present. Only by the fusion of both representations can these regions be nearly separated into two sets of feature vectors of assigned pixels (in two point clouds).

For a corresponding partitioning of the space, the k-means algorithm known per se can be used. K-means determines their centers of gravity for a given number of clusters, so that the sum of the distances of all points to the corresponding center of gravity is minimal.

In order to effect an improvement of the classification accuracy in the sense of a reduction of the misclassified area, cluster analyzes are carried out for all combinations of the representations with the k-means algorithm known per se. With 8 analyzes there are 255 possible combinations.

The 20 best results of cluster analysis are in 11 It can be seen that a combination of several methods of analysis by means of k-means allows a reduction of the misclassified area from 6.69% to 5.05%. Table I below shows a summary of the representations leading to the results of the 11 have led.

Tabelle I

Table I

Given the diagrammatic representation of the merger of two representations of two as the number of clusters sought, namely: 1. cluster intact region, 2. cluster: region of missing adhesive, the boundaries between these clusters are those of the two emphases have the same distance, namely the plane that is perpendicular to the center of the connecting line of the two focal points. If the required partitioning is known, then the limit can be shifted accordingly. 14a shows a diagram illustrating the separation of the two areas "intact area" and "Lack of adhesive" using k-means.

Since the ranges of values of the representations differ strongly, however, before the fusion of the representations a normalization on mean value and scatter of the respective representation is made.

Table II below shows the 11 best individual results of the co-analyzed thermographic films of reference sample P1 and sample P2, all of which provide a misclassified area of over 8.24% for reference sample P1. The best classification result based on a merger with regard to the criterion "misclassified area" is an incorrectly classified area of 7.17%. method preprocessing Misclassified area [%] FFT, phase, 3rd coefficient normalized locally 8.25 NMF, 1st coefficient Difference to picture 1 8.48 FFT, amplitude, 2nd coefficient Difference to picture 1 9.1 FFT, phase, 1st coefficient normalized locally 9.86 52. Image of the film Difference to Figure 1 10.25 NKA, 3rd coefficient normalized locally 10.35 NKA, 3rd coefficient original data 10.59 NMF, 3rd coefficient standardized in time 12.17 NKA, 1st coefficient Difference to picture 1 12.51 FFT, phase, 6th coefficient normalized locally 13,30 NKA, 3rd coefficient Difference to picture 1 14.03 Table II

The following individual representations are involved in this result.
Representation on the basis of time, 52nd image, preprocessing: T ^Δ (t),
FFT phase, 1st coefficient, preprocessing: T ^on (t),
FFT phase, 3rd coefficient, preprocessing: T ^on (t),
HKA, 3rd coefficient, preprocessing: T (t),
HKA, 3rd coefficient, preprocessing: T ^on (t),
NMF, 3rd coefficient, preprocessing: T ^zn (t).

In the following Table III, for all 3 samples, the misclassified area is compared with the fusion of the misclassified area of the best single representation. sample Misclassified area when represented by merger / [%] Misclassified area at best single representation: FFT, phase, 3rd coefficient, preprocessing: T ^zn (t) / [%] Sample 1 (reference) 7.17 8.25 Sample 2 7.59 9.03 Sample 3 3.36 3.53 Table III

The calculation of the size D = l ₁ - l ₂ used for the fusion is in shown in principle.

• 1. Auswahl einer Probe
• 2. Impulsthermografische Prüfung
• 3. Ermittlung alternativer Repräsentationen auf Basis des Originalfilms, des Differenzfilms, örtlich normiertem Film, zeitlich normiertem Film mittels FFT, HKA, NMF
• 4. Zerstörende Prüfung der ausgewählten identischen Probemittels Horizontalschliff
• 5. Thermografische Aufnahme der zerstörten Probe
• 6. Ermittlung von i. O., n. i. O. und nicht zu berücksichtigender Pixel als Referenzklassifikation,
• 7. Eingabe der Ergebnisse aus 3. Schritt und 6. Schritt in Komparatorstufe zum pixelweisen Vergleich jeder einzelnen Repräsentation mit der Referenzklassifikation zur Bestimmung eines jeweiligen Schwellwertes, so dass die fehlklassifizierte Fläche minimal ist,
• 8. Bestimmung der 10 besten Repräsentationen hinsichtlich fehlklassifizierter Fläche,
• 9. Berechnung der optimalen Repräsentation aus den n besten mittels Clusteranalyse, wobei von allen mittels Clusteranalyse berechneten Repräsentationen diejenige mit der geringsten fehlklassifizierten Fläche die optimale Repräsentation ist,
• 10. Ergebnis der Lernphase (Kalibrierung) sind bei automatisierter Qualitäts- bzw. Fehlerklassifikation einer Fügeverbindung zu bestimmende Repräsentationen, die einer Fusion mittels Clusteranalyse zuzuführen sind sowie der Schwellwert für die Größe D = l₁ – l₂.
• 11. Auswahl einer Probe zur Anwendung des gewonnenen Algorithmus aus der Lernphase (Kalibrierung),
• 12. Impulsthermografische Prüfung der Probe aus 11. Schritt
• 13a. Berechnung der im 10. Schritt ermittelten Repräsentationen des Thermografiefilms aus 12. Schritt,
• 13b. Berechnung der optimalen Repräsentation mittels Clusteranalyse entsprechend Algorithmus aus 10. Schritt,
• 13c. pixelweise Klassifikation der optimalen Repräsentation entsprechend Schwellwert für die Größe D aus 10. Schritt,
• 14. Ergebnis: örtlich klassifizierte Fügeprobe

From the flow chart of 15 shows a first embodiment of the method according to the invention, in which initially a learning phase (calibration) for determining an algorithm and subsequently the application of the algorithm obtained, with the following method steps:

• 1. Selection of a sample
• 2. Impulse thermographic examination
• 3. Determination of alternative representations on the basis of the original film, the difference film, spatially normalized film, time normalized film by means of FFT, HKA, NMF
• 4. Destructive testing of selected identical samples by means of horizontal grinding
• 5. Thermographic image of the destroyed sample
• 6. Determination of i. O., ni O. and unobservable pixel as a reference classification,
• 7. Input of the results from the third step and the sixth step in the comparator stage for the pixel-by-pixel comparison of each individual representation with the reference classification for the determination of a respective threshold value, so that the misclassified area is minimal,
• 8. Determination of the 10 best representations regarding misclassified area,
• 9. Calculation of the optimal representation from the best by means of cluster analysis, wherein of all the representations calculated by means of cluster analysis, the one with the least misclassified area is the optimal representation,
• 10. Result of the learning phase (calibration) representations to be determined in the case of automated quality or defect classification of a joint connection, which are to be supplied to a fusion by means of cluster analysis and the threshold value for the quantity D = l ₁ -l ₂ .
• 11. Selection of a sample for the application of the algorithm obtained from the learning phase (calibration),
• 12. Impulse thermographic examination of the sample from step 11
• 13a. Calculation of the representations of the thermographic film from step 12 determined in the 10th step,
• 13b. Calculation of the optimal representation by means of cluster analysis according to algorithm from 10th step,
• 13c. pixel-by-pixel classification of the optimal representation corresponding to the threshold value for the size D from the 10th step,
• 14. Result: locally classified joining sample

The flow chart according to 16 describes a second embodiment of the method with combined execution of the learning (calibration) and application phase.

Thus, according to the 21st step, a first sample A is selected and subjected to a thermographic examination in the 22nd step. According to the 23rd step, a second sample B is selected and likewise subjected to a thermographic test in the 24th step. The imagewise connection of the two thermographic films obtained is then in the 25th step, whereupon in the 26th step alternative representations on the basis of the original film, the difference film as well as the spatially normalized and temporally normalized films are jointly determined by FFT, HKA and NMF for both films ,

The first sample A selected in the 21st step is then subjected to a destructive test by means of horizontal grinding in the 27th step. Then, in the 28th step, a thermographic image of the destroyed sample A is created,

In the 29th step, the reference classification i. O., n. I. O. and unaccounted for pixels are determined, whereupon in the 30th step the enlargement of the image area to the size of the imagewise connected in the 25th step thermographic films takes place and pixels are classified in the range of the sample B selected in 23 step as not to be considered pixels.

The results from the 26th step and the 30th step are entered in the 31st step of a comparator stage for the pixel-by-pixel comparison of each individual representation with the reference classification for the determination of a respective threshold value, so that the misclassified area is minimal.

In the 32nd step, the 10 best representations are determined for the misclassified area, and in the 33rd step, the optimal representation is determined from the n best representations by cluster analysis, of which all representations calculated by cluster analysis are those with the least misclassified area is.

In the 34th step, the classification of the sample selected in step 23 is then performed by applying the threshold value calculated in step 33 to the pixels of the optimal representation in the region of the sample selected in step 23. Step selected joining sample classified pixelwise.

Claims (5)

Method for the pulse thermographic quality and error classification of a joint connection such as a structural bonding in the automotive body area, in which an assembled component (joint sample) is impulsively nondestructively investigated for failure by the joining sample excited by ultrasound and the transient heating and / or cooling process in the joining sample thermographically detected by means of a thermographic camera before, during and after its excitation and by means of a computer connected to the thermographic camera alternative representations of the thermographic film are calculated, and in which the following method steps are also performed: in a learning phase (calibration) (1st step to 10th. Step), after selection of the joining sample (1st step), from the thermographic film recorded by the thermographic camera in a pulse thermographic examination (2nd step) of the selected joining sample (1st step), normalized films on the basis In addition to the images (representations) of the original film, the difference film and the normalized films, from the data of the original film, the difference film and the data of the produced normalized films are generated by the time as difference film, time normalized film and locally normalized film additional representations of the thermographic film are generated by means of FFT, HKA and NMF (3. Step), wherein the number of additionally generated representations depends on the respective procedure, the alternative representations obtained are stored in a computer downstream comparator stage (3rd step), then as a destructively tested reference mechanically a horizontal grinding of the same joining sample is made, from which the sought errors in the joining sample are visually detected (4th step), then a reproduced clamping of horizontal grinding and reproduced positioning of the thermographic camera in the same process structure by means of thermography camera a thermogram (single thermographic image) of the identical section of the destroyed joint sample created at the same local resolution (Step 5), and the thermographic film of this section is registered in the computer connected downstream of the thermographic camera, the thus existing in the same position and resolution thermogram of Horizontalholzschliffs is used as a reference for a pixel-by-pixel classification of the thermographic film, wherein the reference each pixel of the thermographic film of one of the classes i. O., ni O. or "area not to be considered" (6th step), the reference thus generated is then fed to the comparator stage as a reference and with all in this already stored representations of the thermographic film of the joining sample to identify the thermographically sought faulty and error-free regions of the representations, by applying to each representation of the thermographic film a threshold for discriminating in the reference as i. O. or ni O. pixels, the pixels not to be taken into account in this optimization and by counting the misclassified pixels and the threshold value is changed until the number of misclassified pixels is minimal and thereby the distinctness is optimized between intact and non-intact regions (7th step), then the n (10) best representations are determined in terms of the criterion of the misclassified regions (the number of misclassified pixels) (8th step) and all possible combinations of these n best representations are thus formed in that, for each combination, a vector is assigned to each pixel whose coordinates contain the coefficients of the respective representation associated with the pixel, averaging and scattering, forming sets of feature vectors of associated pixels (point clouds) having as co-ordinates the coefficients of unity l representations and are subjected to a cluster analysis in which two clusters with respective cluster focus i. O. (C ₁ ) and cluster gravity ni O. C ₂ ) are determined, wherein the distance of each vector to the cluster centroid i. O. (C ₁ ) and to the cluster centroid ni O. (C ₂ ) represents the fused representation of the thermographic film, which is evaluated for all possible combinations analogous to the individual representations by means of the number of misclassified pixels, and wherein by forming a ranking of the individual representations and the fused representations corresponding to the misclassified area the optimal representation is obtained (9th step) and the result in the sense of an algorithm applied to the thermographic film of the joints the specification of the individual representations to be fused and the threshold for the size D = l ₁ - l ₂ (difference of the Euclidean distances l ₁ , l ₂ to the center of gravity i.O. (C ₁ ) and to the center of gravity ni O. (C ₂ ) for each pixel (10th step), and thus representations of HKA and NMF to these individual representations also use the corresponding analysis functions in the algorithm w and then to the learning phase (calibration) (1. Step to 10th step) is selected in an automatic application phase, a previously unclassified sample (11th step) and analogous to the pulse thermographic examination of the learning phase (calibration) pulse thermo graphically tested while a thermographic film of this sample created (12th step), from the previously (10th step) calculated representations and connected to vectors (corresponding to 9th step) (13th step), whereupon each individual pixel of this optimal representation using the determined threshold (10th step) as i. O. or ni O. is classified (13th step), and thus automatically results in the joining sample locally classified (14th step). A method according to claim 1, characterized in that the determination of all n ² - 1 fused representations and the associated focus i. O. (C ₁ ) and the associated center of gravity ni O. (C ₂ ) by means of a known k-means algorithm, wherein for evaluating each fused representation for each pixel, the Euclidean distances l ₁ and l ₂ to the center of gravity i. O. (C ₁ ) or to the focus ni O. (C ₂ ) are determined, then by subtracting the two distances l ₁ and l ₂ a size D = l ₁ - l _{2 is} formed, which is negative when the respective pixels closer to the center of gravity i. O. (C ₁ ), and positive, if the respective pixel is closer to the center of gravity ni O. (C ₂ ), the threshold is determined for each fused representation and then counting how many pixels are considered intact or defective and then the misclassified area is determined and then, if this has been done for all n ² -1 representations, a ranking of all fused and single representations is made from the misclassified area and then the obtained optimal threshold for the size D = l ₁ - l _{2 is} applied to the optimal representation of the thermographic film captured by the thermographic camera of the joining sample, whereby for each pixel whose representation the result "adhesive present" or "adhesive absent" automatically results. A method according to claim 1 and 2, characterized in that a plurality of references are processed simultaneously to determine the representations to be merged and the threshold value for the size D = l ₁ - l ₂ . Method according to any one of claims 1 to 3, characterized in that a combination of the learning phase and the application phase and thus the integration of HKA and NMF take place by adding one or more references also an unweighted thermographic film is co-processed, which is assigned during the evaluation of the non-evaluated area and thus still enters into the calculation of the analysis functions of HKA and NMF, but not in the determination of the optimum threshold for each individual representation or the Threshold for the size D = l ₁ -l ₂ and then classifying each pixel of the unweighted thermographic film based on the fusion representations found and the threshold. Method according to one of claims 1 to 4, characterized in that the image of the horizontal section of the joint sample (step 1) is taken with any camera having a different resolution than the thermographic camera and a different distance and a different orientation for horizontal grinding of the joint sample than the thermographic camera, and that an algorithmic alignment of the image of the destroyed joining sample to the thermographic film and the conversion of this image into the cut-out and the resolution of the thermographic film take place.

Images