DE102018133567A1 - Process for computer-aided optical status assessment of an object - Google Patents

Abstract

A computer-aided optical status evaluation of an object, in which an optical recording of an actual state of the object in the form of an actual pixel representation is associated with an optical recording of an initial state of the object or a comparable object in the form of an original pixel representation by the following steps: Determining a flat comparison area of the object, which can be changed due to an external influence and can be optically detected via an external accessibility, taking a picture of the comparison area of the object with an optical camera and storing the picture as an actual pixel representation of the comparison area, which shows an actual state of the Represented object in the comparison area, comparing the actual pixel representation of the comparison area with the original pixel representation of the comparison area, the comparison area of the object in the original state or the comparison area of one of the Includes the object of comparison corresponding to the original state, determining a correlation factor from the comparison of the actual pixel representation with the original pixel representation and assigning the determined correlation factor to a state value range, from which a state of the object can be determined on the basis of the evaluated comparison region of the object on the basis of the determined correlation factor .

Description

Field of the Invention

The present invention relates to a computer-aided optical status evaluation of an object, in particular a tool, such as a die or a stamp, a component, a joint or a connection, in which an optical recording of an actual state of the object in the form of an actual pixel representation with a optical recording of an initial state of the object or a comparable object in the form of an original pixel representation. In addition, the present invention relates to a neural network for monitoring the status of tools used, in which the above computer-aided optical status evaluation is used.

Background of the Invention

The large-scale production of parts, for example in mechanical engineering by machining processes, is determined by high efficiency. A basis of this efficiency is the cycle time within which a certain item is manufactured or its processing is completed. A reduction of the cycle time means at the same time a reduction in processing costs, machine working hours and often also working hours of the workers involved.

During this manufacture of objects, tools such as drills, milling cutters, saw blades or the like are used, for example, in combination with machining processes. If adhesive processes are used on an industrial scale, a nozzle for applying an amount of adhesive forms the basis for a time-efficient adhesive process. The nozzle can thus also be viewed as a tool.

If a tool is damaged or destroyed in the machining process mentioned above, this results in downtimes of the machine tool for changing the tool. These downtimes reduce the efficiency of the processes running and, when viewed as a whole, increase the cycle times and costs for the production of individual items.

Describes how to avoid downtimes EP 0 127 751 A1 a tool breakage monitoring device. This is to prevent a workpiece from being machined with a damaged tool, for example a broken drill. In addition to the necessary replacement of the broken drill, it would also be necessary to remove an additionally damaged workpiece from the machining cycle. Accordingly, the tool is to be monitored with the described device without loss of working time by checking the length of the tools outside the work spindle, for example in a tool magazine. This system is based on the fact that the drills are removed from the machine during a machining cycle and temporarily stored in a magazine until a new cycle requires the use of this drill. Within this intermediate magazine it is possible to check the length of the tool, here the drill. If the examined drill exceeds a permissible length difference, this drill is eliminated from the further machining cycle.

WO 2000/15082542 describes a method for monitoring a cutting tool, which is used in the cutting of mainly metallic materials. This cutting tool is a cutting tool in which the cutting force, the feed force and the passive force are meaningful for an optimal execution of a machining process. With the help of individual sensors, the forces mentioned are continuously recorded. If there is a sudden reduction in the force detected during machining, the machining process is automatically stopped. Although this leads to a stoppage of the machining processes, it avoids major damage to the machined workpiece. After it has been recognized which tool damage is present, the tool is replaced and the machining process is continued.

The monitoring methods summarized above are characterized by a high expenditure of equipment and time. In addition, they only intervene in the machining cycle when there is actually damage to a tool. As a result, the machining cycle has to be interrupted in order to replace a tool.

Based on these known monitoring methods, it is the object of the present invention to realize more effective monitoring, which reduces the loss of machining times and enables a more efficient evaluation of tool states.

Summary of the invention

The above object is achieved by a computer-aided optical status evaluation of an object according to independent claim 1. In addition, the above object is achieved by a neural network according to independent claim 13, with which a condition assessment of the same tools, preferably a punch rivet die, is carried out for at least one user in operation and a result of the condition assessment is processed.

The computer-aided optical status evaluation of an object is used, in particular, to monitor a tool, such as a die or a stamp of a punch rivet setting device, to monitor a component, a joint or a connection that has been made. As part of the optical status evaluation, an optical recording of an actual state of the object in the form of an actual pixel representation is associated with an optical recording of an initial state of the object or a comparable object in the form of an original pixel representation. This condition assessment has the following steps: defining a flat comparison area of the object, which can be changed due to an external influence and can be optically detected via an external accessibility, creating a picture of the comparison area of the object with an optical camera and storing the picture as an actual pixel representation of the Comparison area, which represents an actual state of the object in the comparison area, comparing the actual pixel representation of the comparison region with the original pixel representation of the comparison region, which includes the comparison region of the object in the original state or the comparison region of a comparison object corresponding to the object in the original state, determining a correlation factor from the comparison of the actual pixel representation with the original pixel representation and assignment of the determined correlation factor to a state value range, from which on the basis of the determined correlation a factor of the object can be determined on the basis of the evaluated comparison region of the object.

The basis of the preferred computer-aided optical status evaluation of an object is its optical accessibility and thus the possibility of capturing it with the aid of an optical camera. Such objects are, for example, tools such as a punch rivet die or a punch for clinching or joining punch rivets. It is also preferred to record the geometry of a nozzle with which defined adhesive patterns have to be applied to an adhesive surface. Because as soon as this nozzle is damaged or at least partially clogged by adhesive, the adhesive pattern required for the process is disturbed and the connection quality is thereby impaired.

The present invention is described below using the example of monitoring a punch rivet die as a tool. Such punch rivet dies are distinguished by a rotational symmetry in their shape. When using the punch rivet die, it is exposed to high mechanical loads during the establishment of a connection. A large number of joining processes can lead to contamination and thus to an impairment of the original shape of the die. Furthermore, the mechanical loads in the course of a large number of joining processes can damage the die, so that its rotationally symmetrical shape is disturbed.

For optical monitoring of the functionality of a die or a gradual impairment of the configuration of the die, regular recordings of the die are preferably made. For this purpose, part of the die is first defined as a comparison area. A characteristic illustration of a die preferably forms a plan view of a recess in the die, with the aid of which the closing head, for example a punch rivet connection, is formed. In the same way, it is preferred to monitor the structure of an anvil or a punch or the like.

In order to ensure that the status of the die is evaluated as quickly as possible and preferably independently of the location of the use of the tool, here the die, the comparison area is captured by an optical camera. After the acquisition, this image is saved in the form of an actual pixel representation. The storage and later evaluation preferably takes place in a computer connected to the camera. In order to be able to recognize possible damage to the matrix within the scope of the recorded actual pixel representation, a comparison is made with the original shape of this matrix. For this comparison, an original pixel representation is preferred which corresponds to the original state of the die used, but does not have to have been originally recorded with it. Accordingly, the original pixel representation represents an optical image of the comparison area of the die before it is used. In this context, it is preferred to record a comparable matrix of the same geometry in the comparison area as the original pixel representation instead of the matrix before use.

The pixels of the original pixel representation represent the rotationally symmetrical geometry features which characterize the die before it is impaired by use. A correlation factor between the actual pixel representation and the original pixel representation is determined in order to obtain an indication of possible damage or wear of the matrix under consideration in the comparison area. This correlation factor is a measure of the extent to which the actual pixel representation reflects optical changes compared to the original pixel representation.

The correlation factor with a value of 1 or close to 1 preferably signals an approximate identity between the actual pixel representation and the original pixel representation. This means that the die has undergone almost no changes due to its previous use.

If the correlation factor specifies values <1, then this shows the effects of the use on the configuration of the die in the comparison area. Accordingly, it is preferred to define a value range for the correlation factor between the actual pixel representation and the original pixel representation, from which a state of the matrix used can be derived. Thus, the correlation factor preferably represents an indicator of the functionality of the die used in comparison to the original form of the die. Depending on the shape of the die, a different correlation factor signals that the punch rivet die is not functioning properly. The correlation factor can be 0.8 or 0.72, which must be defined depending on the form of the matrix.

While damage to the die in the comparison area, which should indicate the avoidance of further use of the die, can preferably be read from the correlation factor, it is also preferred to recognize imminent failure of the die on the basis of the increasing decrease in the correlation factor. Such an indication from the correlation factor is preferably used to avoid machine downtimes. Because during a production-related break it is possible to replace a die with a critical condition in the comparison area before it fails during production. Correspondingly, in the context of the state assessment preferred according to the invention, the determined correlation factor is assigned to a state value range of the object or the die discussed here by way of example.

der eine Zustandsänderung zwischen den in Zusammenhang gebrachten Gegenständen der verglichenen Darstellungen repräsentiert.According to a preferred embodiment of the present invention, the status evaluation is carried out with the following further steps: representation of the actual pixel representation with n · m pixel values according to {x _1,1 ... xn, m}, representation of the original pixel representation with n · m pixel values according to {y _1,1 ... y _{n, m} }, calculate the correlation coefficient for both representations according to $$r=\frac{{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^m\left(x_{i,j}-\overline{x}\right)\left(y_{i,j}-\overline{y}\right)}}}{\sqrt{{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^m{\left(x_{i,j}-\overline{x}\right)}^{\mathrm{2nd}}}}}\sqrt{{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^m{\left(y_{i,j}-\overline{y}\right)}^{\mathrm{2nd}}}}}},$$

which represents a change in state between the related objects of the compared representations.

The preferred optical recording of the comparison area and its digital acquisition and storage in a plurality of pixels ensures that the data volume of a pixel is assigned to each pixel of the comparison area under consideration. This amount of data includes an indication of the proportions of the primary colors red, green and blue, an indication of the saturation and the brightness of the pixel. Since the optically viewed and digitally stored comparison area is composed of the majority of the pixel data, these can be evaluated specifically according to regularities, preferably in terms of brightness. If a change in this regularity is recognized by the comparison between the actual pixel representation and the original pixel representation, this indicates a changed state of the matrix in the comparison area. A change in the state of the die in the comparison area is preferably due to the mechanical loads due to the joining processes carried out. In addition, these mechanical loads have the effect of damage to the die in the comparison area or contamination of the die in the comparison area.

According to a further preferred embodiment of the method for status evaluation, the further steps are provided: recording temporally spaced and successive actual pixel representations of the comparison area as at least the current actual pixel representation and previous actual pixel representation and calculating the correlation factor from a comparison of the current actual pixel representation and the one that preceded the current actual pixel representation, preferably the directly preceding actual pixel representation.

According to the invention, the comparison of the actual pixel representation and the original pixel representation preferably takes place with the aid of a computer. Sufficient computing power is now available with conventional computers that such a pixel comparison between the actual pixel representation and the original pixel representation is not associated with any delay in the processing of workpieces. In this context, it was found preferred that in addition to the comparison between the actual pixel representation and the original pixel representation, a comparison between the actual pixel representation and one of these preceding actual pixel representations is also meaningful. Because in a temporal sequence or a succession of actual pixel representations, step-by-step changes to the tool or the die can be seen in the comparison area. It is often the case that the mechanical load on the tool or the die gradually leads to damage in the comparison range, which increases in its degree of damage with the number of joining processes. If you therefore consider a sequence of actual pixel representations, which have preferably been recorded one after the other in time, then an impending damage is preferably already recognizable, which denies further use of the tool or the die.

On this basis, it is preferred to extend the condition assessment to include an increasing probability of failure of the die. Because as soon as an increasing damage in the comparison area of the die is predictable or recognizable on the basis of the successive actual pixel representations, then this indicates an imminent failure of the die.

According to the invention, a rotationally symmetrical object, here preferably a die, is preferably evaluated as part of the condition assessment. Furthermore, according to the invention, the rotationally symmetrical object is preferably a rotationally symmetrical die or punch rivet die, a rotationally symmetrical joint or a rotationally symmetrical stamp surface.

basierend auf den Pixelwerten, Vergleichen der Linien £_k untereinander und Erkennen von Abweichungen der Linien £_k untereinander, um Schäden an der Matrize zu qualifizieren.According to a further preferred embodiment of the present invention, the following further steps are provided: Defining a plurality of lines running in a star shape £ _k with k = 1 ... n in the actual pixel representation based on the Bresenham algorithm, which each have a starting point at a symmetrical center of the actual pixel representation and a brightness curve based on the pixel data per line £ _k deliver. In addition, the following further steps are preferred according to the invention: Representation of the plurality of lines £ _k each as a brightness curve ${\text{£}}_k^{\beta }=\left\{{\mathrm{<figure-callout\; id="1"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0055.png"\; state="\{\{state\}\}">p</figure-callout>}}_1^{\beta },...,p_l^{\beta }\right\}$

based on the pixel values, comparing the lines £ _k with each other and recognizing deviations of the lines £ _k among themselves to qualify damage to the die.

The rotationally symmetrically arranged comparison area of the die is characterized by characteristic geometry features that are arranged concentrically. Accordingly, it is preferred to place lines starting in a star shape in the center of the comparison area and extending radially outward. These lines are characterized by the brightness values of the pixels forming them. This is because the brightness values of the pixels in the representation of the comparison area preferably reflect a change in height and depth in the cavity of the matrix under consideration. Due to the rotational symmetry of the comparison area, i.e. the die itself, each centrally starting line should be characterized by the same brightness curve. However, as soon as there is geometrical damage in the die, a line crossing this damaging area will show a different brightness curve compared to the other lines. If one compares the plurality of star-shaped lines directly with one another, then damage or soiling or both can be recognized from the discernible differences between the lines within the comparison area of the die.

According to a further preferred embodiment of the present invention, the condition assessment is characterized by the further steps: calculating a mean value line in each case £ ^β from the plurality of lines £ _k as a brightness curve for the matrix in the initial state on the basis of the original pixel representation and for the matrix at least in the actual state on the basis of the actual pixel representation and comparing the mean value lines £ ^β for the matrix in the initial state and for the matrix at least in the actual state.

According to an alternative embodiment of the present invention, it is not absolutely necessary to carry out an absolute comparison between the plurality of star-shaped lines of the pixel representation. It is also preferably possible to calculate an average value line from the plurality of star-shaped lines for the actual pixel representation. This mean value line then represents an average brightness curve for the comparison area of the matrix under consideration or of the object under consideration. In comparison, a mean value line is also calculated from a similar plurality of star-shaped lines of the original pixel representation. A preferred comparison of the mean value lines for the actual pixel representation and the original pixel representation signals, if they match, a preferred functionality of the matrix under consideration. Should the comparison of the mean value lines of the actual pixel representation and the original pixel representation show recognizable deviations, then this indicates dirt and / or damage to the die in the comparison area.

On the basis of the differences between the compared mean value lines, a decision criterion is preferably defined whether a matrix under consideration may continue to be used or not.

In addition, it is preferred that average lines of the actual representation and a previous actual representation determined in this way are compared with one another. From this comparison it can be seen whether increasing soiling and / or damage to the die in the comparison area indicates an impending malfunction of the die in question.

als Helligkeitsverlauf für eine Matrize im Ausgangszustand anhand der Ursprungs-Pixeldarstellung und für eine Matrize zumindest im Ist-Zustand anhand der Ist-Pixeldarstellung und Berechnen des Korrelationsfaktors $r_{\overline{{\text{£}}_{neu}^{\beta }},\overline{{\text{£}}_{def.}^{\beta }}},$

zwischen der Mittelwert-Linie ${\text{£}}_{neu}^{\beta }$

der Matrize im Ausgangszustand und der Mittelwertlinie ${\text{£}}_{def}^{\beta },$

der Matrize im Ist-Zustand und Bestimmen eines Grads der Schädigung der Matrize basierend auf dem berechneten Korrelationsfaktors $r_{\overline{{\text{£}}_{neu}^{\beta }},\overline{{\text{£}}_{def.}^{\beta }}}.$

According to a further preferred embodiment of the condition assessment, the following further steps are carried out: Calculating an average line £ ^β from the plurality of lines ${\text{£}}_{neu}^{\beta },{\text{£}}_{def}^{\beta },$

as a brightness curve for a matrix in the initial state using the original pixel representation and for a matrix at least in the actual state using the actual pixel representation and calculating the correlation factor $r_{\overline{{\text{£}}_{neu}^{\beta }},\overline{{\text{£}}_{def.}^{\beta }}},$

between the mean line ${\text{£}}_{neu}^{\beta }$

the matrix in the initial state and the mean value line ${\text{£}}_{def}^{\beta },$

the die in the actual state and determining a degree of damage to the die based on the calculated correlation factor $r_{\overline{{\text{£}}_{neu}^{\beta }},\overline{{\text{£}}_{def.}^{\beta }}}.$

According to a further preferred embodiment of the state evaluation according to the invention, a mean value line is determined from the brightness profiles for the actual pixel representation and for the original pixel representation (see above). While it is preferred according to the embodiment described above to compare these mean value lines with one another, it is also alternatively preferred to calculate the correlation factor between the mean value line of the actual pixel representation and the mean value line of the original pixel representation. The correlation factor is preferably between 1 and 0. Depending on the deviation of the correlation factor from 1, ie the assumption of values <1, possible damage to the die can be determined in the comparison range. In this context, it is preferred according to the invention to specify a range of values for the correlation factor from which the degree of damage to the die in the comparison range can be read.

According to a further preferred embodiment of the condition assessment according to the invention, the following further steps are provided: calculating the correlation factor between each line £ _k and the other lines £ _k the majority of lines £ _k for the matrix in the initial state and in the current state, calculating the mean value of the calculated correlation factors for the matrix in the initial state and calculating the average value of the calculated correlation factors for the matrix in the current state and determining a degree of damage to the matrix in the current state using a Comparison between the mean correlation factors of the matrix in the current state and in the initial state.

According to a further preferred embodiment of the state evaluation according to the invention, the pixel representation of the current state and the original state or a previous current state is broken down into the plurality of lines already described above. In addition to the method alternatives described above, according to another method alternative, it is preferred to calculate the correlation factor between each line and the rest of the set of lines for each pixel representation. Correspondingly, a plurality of correlation factors result, which corresponds to the number of lines of the pixel representation in each case. From this plurality of correlation factors, the mean value of the calculated correlation factors for the matrix in question is then preferably calculated. This results in an average of the calculated correlation factors for the matrix in the initial state and for the matrix in the actual Status. If one compares these mean values of the calculated correlation factors, a degree of damage to the die in the comparison range can also be derived therefrom. Because as soon as there are geometric differences between correlated lines, this leads to a reduction in the correlation factor. This reduction is also reflected in the averaging of the correlation factor. Accordingly, it is preferred, based on a previously defined range of values for the averaged correlation factors of the die in the actual state, to determine a degree of damage in comparison to the averaged correlation factor of the die in the initial state.

According to a further preferred embodiment of the condition assessment according to the invention, the following further step is provided: determining the center point of the matrix in the actual pixel representation and in the original pixel representation.

The center point of the respective pixel representation preferably forms a starting point in order to carry out the radially outward running star-shaped lines for evaluating the geometry features in the original state and in the actual state. Since the objects to be evaluated are preferably constructed to be rotationally symmetrical, a center point arranged in the center or in the concentric center point of the comparison region under consideration forms an optimal starting position for comparing the star-shaped lines with one another. This applies equally to a direct comparison of such lines of the actual pixel representation and the original pixel representation as well as to a comparison of the mean value lines and / or a comparison of the calculated correlation factors of the different embodiments described above.

To support the determination of the center point of the die, it is also preferred to arrange the recording optical camera concentrically to the comparison area to be recorded and in particular to the center of the concentrically formed matrix. Such an accuracy of arrangement between the die and the camera can preferably be achieved by using a precisely working movement mechanism, such as an industrial robot.

basierend auf den Pixelwerten und dem Bresenham-Algorithmus, Bestimmen von drei Pixeln pro Linie £_k ausgehend von der Mitte, an denen die Helligkeitsdifferenz zum darauffolgenden Pixel maximal ist gemäß $p_{max}^{grad}=\underset{i=1\dots l-1}{\text{max}}\left|p_{i+1}^{\beta }-p_i^{\beta }\right|,$

Segmentieren der Mehrzahl von Pixeln zu unterschiedlichen Kreiskonturen der Matrize, indem für jede Kreiskontur ein Cluster mit Hilfe einer Menge von Gaußfunktionen nach dem Gaußschen Mischungsmodell modelliert wird, danach Ermitteln der dominanten Gaußverteilung mit der geringsten Varianz und daraus Bestimmen des Kreises aus den Pixelkoordinaten und dessen Mittelpunkt und Festlegen des Mittelpunkts als Matrizenmittelpunkt.In addition, according to the invention, the following steps are preferably provided for determining the center point in the respective pixel representation: Defining a plurality of lines running in a star shape £ _k with k = 1 ... n in the actual pixel representation as the brightness curve ${\text{£}}_k^{\beta }=\left\{{\mathrm{<figure-callout\; id="1"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0055.png"\; state="\{\{state\}\}">p</figure-callout>}}_1^{\beta },...,p_l^{\beta }\right\}$

based on the pixel values and the Bresenham algorithm, determining three pixels per line £ _k starting from the center, at which the difference in brightness to the next pixel is maximum according to $p_{max}^{Grad}=\underset{i=1...l-1}{\text{Max}}\left|p_{i+1}^{\beta }-p_i^{\beta }\right|,$

Segmentation of the plurality of pixels into different circular contours of the matrix by modeling a cluster for each circular contour using a set of Gaussian functions according to the Gaussian mixture model, then determining the dominant Gaussian distribution with the least variance and determining the circle from the pixel coordinates and its center and setting the center point as the matrix center point.

The present invention also discloses a neural network with which a condition assessment of the same tools, preferably a punch rivet die, is carried out for at least one user in the company, in which a result of a condition assessment according to one of the preceding claims can be processed.

A plurality of similar tools, in particular punch rivet dies of the same type, are preferably used by one or more users. These punch rivet matrices are regularly evaluated and monitored based on the computer-assisted optical condition assessment described above. On the basis of the large number of joining processes which are realized with the same punch rivet geometry, data for describing the service life and the failure probability of the punch riveting die are collected depending on the joining processes. In this context, the joined materials, the ambient conditions and other conceivable accompanying circumstances of the respective joining process are preferably stored as further parameters. Taking the various environmental conditions into account, the neural network captures relationships between the external conditions and their influence on failure mechanisms of the tool used, in particular the punch rivet die. These collected data and the relationships that are preferably recognized in these support the possibility of predicting after how many joining processes a critical damage state of a punch rivet die used could occur. Thus, the evaluation of the data of the computer-aided optical condition assessment supports an economical but nevertheless timely exchange of a punch rivet die or a tool before its failure leads to a production shutdown.

Figure list

• 1 eine seitliche Schnittansicht einer Stanznietmatrize mit darüber angeordneter optischer Kamera,
• 2 eine Draufsicht auf die Stanznietmatrize aus 1 entlang der Mittelachse M,
• 3 eine Veranschaulichung eines bevorzugten Vergleichsbereichs V zur Auswertung der Draufsicht auf die Stanznietmatrize gemäß 2,
• 4 a-c drei vereinfachte Rastergrafiken a) einer Ursprungsmatrize, b) einer defekten oder beschädigten Matrize und c) einer gebrauchten oder abgenutzten Matrize,
• 5 a-c mehrere Ist-Pixeldarstellungen einer Matrize a) im Ursprungszustand, b) im benutzten Zustand und c) im beschädigten Zustand,
• 6 eine bevorzugte Mehrzahl von sternförmig von der Mitte der Pixeldarstellung der Stanznietmatrize M ausgehend radial auswärts verlaufende Linien £_k nach dem Bresenham-Algorithmus,
• 7 die bevorzugten Helligkeitsverläufe der Linien £_k aus 6,
• 8 die gemittelten und auf den Bereich 0...1 normierten Helligkeitsverlauf ${\mathcal{L}}^{\beta }$
  
  der einzelnen Linien £_k aus 7,
• 9 a-c die bevorzugte Darstellung unterschiedlicher Matrizen im Vergleichsbereich als Ist-Pixel darstellungen,
• 10 die bevorzugten Helligkeitsverläufe der Linien aus 9 a-c,
• 11 die bevorzugten gemittelten Korrelationsfaktoren in den 5 a-c,
• 12 die bevorzugten Linien ähnlich zu 6 ausgehend vom Mittelpunkt als Pixeldarstellung,
• 13 bevorzugte Darstellung des geometrischen Schwerpunkts,
• 14 ein bevorzugtes Histogramm zur Clusterung der Pixel der Ist-Pixeldarstellung,
• 15 bevorzugte Ergebnisse der Clusterung der Pixel,
• 16 bevorzugte berechnete kreise der drei Hauptcluster der Pixel, und
• 17 bevorzugte Clusterung von Pixel einer Ist-Pixeldarstellung einer beschädigten Matrize.

The preferred embodiments of the present invention are explained in more detail with reference to the accompanying drawing. Show it:

• 1 2 shows a sectional side view of a punch rivet die with an optical camera arranged above it,
• 2nd a plan view of the punch rivet 1 along the central axis M ,
• 3rd an illustration of a preferred comparison area V to evaluate the top view of the punch rivet die according to 2nd ,
• 4 ac three simplified raster graphics a) an original matrix, b) a defective or damaged matrix and c) a used or worn matrix,
• 5 ac several actual pixel representations of a matrix a) in the original state, b) in the used state and c) in the damaged state,
• 6 a preferred plurality of star-shaped from the center of the pixel representation of the punch rivet die M starting radially outward lines £ _k according to the Bresenham algorithm,
• 7 the preferred brightness gradients of the lines £ _k out 6 ,
• 8th the averaged brightness curve normalized to the range 0 ... 1 ${\mathcal{L}}^{\beta }$
  
  of the individual lines £ _k out 7 ,
• 9 ac the preferred representation of different matrices in the comparison area as actual pixel representations,
• 10 the preferred brightness gradients of the lines 9 ac ,
• 11 the preferred averaged correlation factors in the 5 ac ,
• 12th the preferred lines similar to 6 starting from the center as a pixel representation,
• 13 preferred representation of the geometric center of gravity,
• 14 a preferred histogram for clustering the pixels of the actual pixel representation,
• 15 preferred results of pixel clustering,
• 16 preferred calculated circles of the three main clusters of pixels, and
• 17th preferred clustering of pixels of an actual pixel representation of a damaged matrix.

Detailed description of the preferred embodiments of the present invention

In the field of mechanical engineering, different joint connections are realized with a rotationally symmetrical tool. These tools preferably do not include, but are not limited to, punches, punch rivet dies, adhesive nozzles and the like.

As soon as these tools are soiled or damaged, a disturbance in the rotational symmetry can be identified by means of an illustration of a comparison area of the individual tools. The disturbance of the rotational symmetry underlines a restricted functionality of the tool. This is explained below using a punch rivet as an example. It can be transferred to other rotationally symmetrical tools, joints, component geometries and the like.

As an example shows 1 a radial section through a preferred embodiment of a punch rivet die 10 . This includes a central deepening 12th as well as another concentric depression 14 . It is understood that the recesses 12th , 14 concentric around the central axis M are arranged.

In a top view D on the concave cavity of the punch rivet die 10 along the central axis M are the recesses 12th , 14 recognizable as rings. this shows 2nd .

For the status evaluation of a rotationally symmetrical tool preferred according to the invention, here the punch rivet die 10 , the top view D along the central axis M on the punch rivet die 10 used as a basis. In this context, a section of the top view D or the entire top view D as a comparison area V Are defined.

The comparison area V captures an optical camera K . This is preferably centered on the comparison area V in extension of the central axis M arranged. The optically captured comparison area V is sent to a computer in the form of analog or digital electrical signals C. transfer. In the computer, the image data of the optically captured comparison area V stored as a plurality of pixels.

The image data or the plurality of pixels are preferably sent to a monitor M transmitted and displayed there. Therefore, a visual and / or a computer-aided or arithmetic evaluation of the image data of the comparison area is preferably carried out V .

Images of the optical camera K are available in computer-readable form as raster graphics. This consists of the arrangement of pixels - the so-called pixels. In addition to the hue, the pixels also each contain the color saturation σ and the brightness β.

The color tone is reproduced by the additive mixing of the three primary colors red (R), green (G), blue (B) and abbreviated as the RGB color space. The color saturation describes the quality of the color effect, i.e. the strength of the colored stimulus compared to achromatic colors. The colors white, gray and black have a saturation of 0. In addition to hue and saturation, the brightness β is another color property. This describes how light or dark a color appears. Accordingly, a pixel p contains the properties E: $$p^{\text{E}}=\left(\text{R},\text{G},\text{B},\mathrm{\beta },\mathrm{\sigma }\right)$$

$$p^{\text{schwarz}}=\left(\mathrm{0,0,0,0,0}\right)$$

$$p^{rot}=\left(\mathrm{1,0,0,0.5,1}\right)$$

$$p^{grau}=\left(\mathrm{0.5,0.5,0.5,0.5,0}\right)$$

$$p^{\text{orange}}=\left(\mathrm{1,0.4,0,0.5,1}\right)$$

The properties mentioned can each take values between 0 and 1 (corresponding to 0% and 100%). This can be seen from the following examples of pixels of different colors: $$p^{\text{White}}=\left(\mathrm{1,1,1,1,0}\right)$$

$$p^{\text{black}}=\left(\mathrm{0,0,0,0,0}\right)$$

$$p^{rOt}=\left(\mathrm{1,0,0,0.5,1}\right)$$

$$p^{Grau}=\left(\mathrm{0.5.0.5.0.5.0.5.0}\right)$$

$$p^{\text{orange}}=\left(\mathrm{1.0.4.0.0.5.1}\right)$$

For two series of measurements or representations, each with n · m values {x _1.1 ... x _{n, m} } and {y _1.1 ... y _{n, m} }, a correlation coefficient r is preferred as a measure of the similarity of these used two series of measurements to each other. The correlation coefficient r takes values between -1 and +1 and is calculated according to the following mathematical definition $$r=\frac{{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^m\left(x_{i,j}-\overline{x}\right)\left(y_{i,j}-\overline{y}\right)}}}{\sqrt{{{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^m\left(x_{i,j}-\overline{x}\right)}}}^{\mathrm{2nd}}}\sqrt{{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^m{\left(y_{i,j}-\overline{y}\right)}^{\mathrm{2nd}}}}}}$$

The mean value x̅ - analogously to the mean value y̅ - is calculated according to $$\overline{x}=\frac{1}{n\cdot m}{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^m{x}_{i,j}}}$$

With a value for r of +1 (or –1) there is a completely positive (or negative) linear relationship between the measurement series. If the correlation coefficient r is the value 0 , the two series of measurements do not depend linearly on one another at all.

The correlation coefficient r is made for three example graphics 4th calculated as an example. In 4th show the representations simplified the comparison areas V for a) the punch rivet die in its original state, b) the punch rivet die in the damaged / defective condition and c) the punch rivet die in the used or worn condition. Per representation of the comparison area V 14 x 14 = 196 pixels are specified.

For a simplified representation of the calculated values, these are related to smaller but qualitatively similar pixel representations. These preferably comprise only 4 × 4 pixels

mit n = 4 und m = 4 lauten

1. a) $$\begin{array}{l}{p}_{links}^{\beta }\{{\mathrm{<figure-callout\; id="1"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0055.png"\; state="\{\{state\}\}"><figure-callout\; id="1"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0055.png"\; state="\{\{state\}\}">p</figure-callout></figure-callout>}}_{\mathrm{1,1}}^{\beta }=\mathrm{0,}{\mathrm{<figure-callout\; id="1"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0055.png"\; state="\{\{state\}\}"><figure-callout\; id="2"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}">p</figure-callout></figure-callout>}}_{\mathrm{1,2}}^{\beta }=\mathrm{1,}{\mathrm{<figure-callout\; id="1"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0055.png"\; state="\{\{state\}\}"><figure-callout\; id="3"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}">p</figure-callout></figure-callout>}}_{\mathrm{1,3}}^{\beta }=\mathrm{0,}{\mathrm{<figure-callout\; id="1"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0055.png"\; state="\{\{state\}\}"><figure-callout\; id="4"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}">p</figure-callout></figure-callout>}}_{\mathrm{1,4}}^{\beta }=\mathrm{1,}{\mathrm{<figure-callout\; id="2"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}"><figure-callout\; id="1"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0055.png"\; state="\{\{state\}\}">p</figure-callout></figure-callout>}}_{\mathrm{2,1}}^{\beta }=\mathrm{1,}{\mathrm{<figure-callout\; id="2"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}"><figure-callout\; id="2"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}">p</figure-callout></figure-callout>}}_{\mathrm{2,2}}^{\beta }=\mathrm{0,}{\mathrm{<figure-callout\; id="2"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}"><figure-callout\; id="3"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}">p</figure-callout></figure-callout>}}_{\mathrm{2,3}}^{\beta }=\mathrm{1,}{\mathrm{<figure-callout\; id="2"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}"><figure-callout\; id="4"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}">p</figure-callout></figure-callout>}}_{\mathrm{2,4}}^{\beta }=\mathrm{1,}{\mathrm{<figure-callout\; id="3"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}"><figure-callout\; id="1"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0055.png"\; state="\{\{state\}\}">p</figure-callout></figure-callout>}}_{\mathrm{3,1}}^{\beta }=\\ \mathrm{1,}{\mathrm{<figure-callout\; id="3"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}"><figure-callout\; id="2"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}">p</figure-callout></figure-callout>}}_{\mathrm{3,2}}^{\beta }=\mathrm{1,}{\mathrm{<figure-callout\; id="3"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}"><figure-callout\; id="3"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}">p</figure-callout></figure-callout>}}_{\mathrm{3,3}}^{\beta }=\mathrm{1,}{\mathrm{<figure-callout\; id="3"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}"><figure-callout\; id="4"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}">p</figure-callout></figure-callout>}}_{\mathrm{3,4}}^{\beta }=\mathrm{0,}{\mathrm{<figure-callout\; id="4"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}"><figure-callout\; id="1"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0055.png"\; state="\{\{state\}\}">p</figure-callout></figure-callout>}}_{\mathrm{4,1}}^{\beta }=\mathrm{0,}{\mathrm{<figure-callout\; id="4"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}"><figure-callout\; id="2"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}">p</figure-callout></figure-callout>}}_{\mathrm{4,2}}^{\beta }=\mathrm{0,}{\mathrm{<figure-callout\; id="4"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}"><figure-callout\; id="3"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}">p</figure-callout></figure-callout>}}_{\mathrm{4,3}}^{\beta }=\mathrm{1,}{\mathrm{<figure-callout\; id="4"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}"><figure-callout\; id="4"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0061.png"\; state="\{\{state\}\}">p</figure-callout></figure-callout>}}_{\mathrm{4,4}}^{\beta }=1\}\end{array}$$
   
2. b) $$p_{\text{mitte}}^{\beta }=\left\{\mathrm{0,1,0,1,1,0,1,1,1,1,1,0,0,1,0,1}\right\}\text{sowie}$$
   
3. c) $$p_{\text{rechts}}^{\beta }=\left\{\mathrm{1,0,0,0,1,1,0,1,0,1,0,0,0,1,0,0}\right\}$$
   
   mit den Mittelwerten 10/16, 10/16 und 6/16. Der Korrelationskoeffizient r zwischen der linken (4 a) und der mittleren Pixeldarstellung (4 b) beträgt r=0,66, zwischen der linken (4 a) und der rechten Pixeldarstellung (4 c) r=0,92 und zwischen der mittleren (4 b) und der rechten Pixeldarstellung (4 c) r=0,66. Dieses Ergebnis zeigt die Aussage des Korrelationskoeffizienten r zur Ähnlichkeit der gezeigten Vergleichsbereiche V der Stanznietmatrize M gemäß den Pixeldarstellungen der 4 a bis c.

The brightness values $p_{i,j}^{\beta }=\left\{p_{1.1}^{\beta },...,p_{n,m}^{\beta }\right\}$

with n = 4 and m = 4

1. a) $$\begin{array}{l}{p}_{links}^{\beta }\{p_{1.1}^{\beta }=\mathrm{0,}{p}_{1.2}^{\beta }=\mathrm{1,}{p}_{1.3}^{\beta }=\mathrm{0,}{p}_{1.4}^{\beta }=\mathrm{1,}{p}_{2.1}^{\beta }=\mathrm{1,}{p}_{2.2}^{\beta }=\mathrm{0,}{p}_{2.3}^{\beta }=\mathrm{1,}{p}_{2.4}^{\beta }=\mathrm{1,}{p}_{3.1}^{\beta }=\\ \mathrm{1,}{p}_{3.2}^{\beta }=\mathrm{1,}{p}_{3.3}^{\beta }=\mathrm{1,}{p}_{3.4}^{\beta }=\mathrm{0,}{p}_{4.1}^{\beta }=\mathrm{0,}{p}_{4.2}^{\beta }=\mathrm{0,}{p}_{4.3}^{\beta }=\mathrm{1,}{p}_{4.4}^{\beta }=1\}\end{array}$$
   
2. b) $$p_{\text{center}}^{\beta }=\left\{\mathrm{0.1.0.1.1.0.1.1.1.1.1.0.0.1.0.1}\right\}\text{such as}$$
   
3. c) $$p_{\text{right}}^{\beta }=\left\{\mathrm{1,0,0,0,1,1,0,1,0,1,0,0,0,1,0,0}\right\}$$
   
   with the mean values 10/16, 10/16 and 6/16. The correlation coefficient r between the left ( 4 a) and the average pixel representation ( 4 b) is r = 0.66, between the left ( 4 a) and the right pixel display ( 4 c) r = 0.92 and between the middle ( 4 b) and the right pixel display ( 4 c) r = 0.66. This result shows the statement of the correlation coefficient r on the similarity of the comparison areas shown V the punch rivet die M according to the pixel representations of the 4 a to c .

The above theoretical procedure is used as an example on images of a real punch rivet die, that is to say actual pixel representations of the comparison area V the punch rivet die M , applied. The serve as an example 5 a to c . In 5 a) is a recording of the comparison areas V a new punch rivet die M - So an original pixel representation - shown. 5 b) shows an illustration of a used punch rivet die M and 5 c) shows a picture of a defective punch rivet die M same type.

The camera recordings each preferably have 1624 * 1234 = 2004016 pixels, although other resolutions with more or fewer pixels can also be used at this point. According to the invention, all pixels of the comparison area are preferred V used to calculate the correlation coefficient r.

Between the original pixel representation of the punch rivet die ( 5 a) and the actual pixel representation of the used punch rivet die ( 5 b) each recorded in the comparison area V the correlation coefficient is 0.94. Between the original pixel representation of the new punch rivet die ( 5 a) and the actual pixel representation of the defective or damaged punch rivet die ( 5 c) each recorded in the comparison area V the correlation coefficient is 0.77. Between the actual pixel representation of the used punch rivet die ( 5b) and the actual pixel representation of the defective or damaged punch rivet die ( 5 c) each recorded in the comparison area V the correlation coefficient is 0.79.

The above results show that a broken punch rivet die M can be recognized from the correlation coefficient r. The detection of a dirty or worn punch rivet die can also be identified by the correspondingly decreasing correlation value r. For this purpose, it is preferred to define a work area for the correlation coefficient r in connection with a die or tool state.

According to a preferred embodiment of the present invention, the correlation factor r becomes an actual pixel representation of the comparison area V not just with the original pixel representation of the Punch rivet die determined when not in use. The actual pixel representation of the comparison area has also proven to be advantageous V the punch rivet die M with the previous, preferably the directly previous, actual pixel representation of the comparison area V the punch rivet die M to correlate via the correlation factor r. Due to the preferred correlation of temporally successive actual pixel representations of the punch rivet die M a change in the correlation factor r is discernible, for example in the event of a steady decrease or gradual decrease in the correlation factor r, the greater damage or an impending failure of the punch rivet die M in the comparison area V signals.

To be more precise on the condition of the punch rivet die M in the comparison area V to be able to conclude, the calculation of the correlation coefficient r described above is preferably not only the pixel representation of the comparison area V applied. It is also preferred to evaluate certain image features in the captured pixel representations.

The punch rivet die considered M is preferably circular and rotationally symmetrical with - depending on the die type - a characteristic surface profile. These properties are used by using the well-known Bresenham algorithm in a star shape from the center of the pixel representation of the punch rivet die M lines starting radially outward. These lines are preferred due to the respective pixel courses from the recordings of the comparison area V writable. They contain further evaluable information about the matrix status, such as preferably the brightness curve along a line. This is according to a preferred embodiment of the present invention in 6 shown.

In 6 are preferably the radial lines £ _k marked with k = 1 ... 10, each containing 1 pixel, i.e. £ _k = {p ₁ , ..., p ₁ }. These lines £ _k preferred start at the central midpoint of the comparison area V punch rivet die M .

der Pixel pro Linie £_k , erhält man aufgrund der Rotationssymmetrie der Stanznietmatrize M ähnliche Helligkeitsverläufe. Diese bevorzugten Helligkeitsverläufe sind für die jeweilige Stanznietmatrize M charakteristisch, da sich hell/dunkel Wechsel aus geometrischen Höhen und Tiefen des Matrizenprofils ergeben.According to a preferred embodiment of the present invention, the brightness values of the individual pixels of each line £ _k evaluated. Consider the resulting preferred brightness curve ${\text{£}}_k^{\beta }=\left\{{\mathrm{<figure-callout\; id="1"\; label="p"\; filenames="DE102018133567A1\_0050.png,DE102018133567A1\_0055.png"\; state="\{\{state\}\}">p</figure-callout>}}_1^{\beta },...,p_l^{\beta }\right\}$

the pixels per line £ _k , is obtained due to the rotational symmetry of the punch rivet die M similar brightness gradients. These preferred brightness profiles are for the respective punch rivet die M characteristic, since light / dark changes result from geometrical heights and depths of the die profile.

7 shows the preferred resulting brightness curves of the lines £ _k out 6 . These lines £ _k According to a preferred embodiment, each comprise l = 570 pixels. The number k of lines £ _k is preferably increased to get more accurate results. In the same way, it is preferred to have the number of pixels per line £ _k about the resolution of the recording in the comparison area V to increase.

The direct comparison of the brightness curves of the lines £ _k in 7 preferably shows possible geometric differences along the course of the individual lines £ _k compared to each other. For this purpose, the number of pixels is plotted on the x-axis. The pixel at the center of the punch rivet die is preferably located at the coordinate origin M . Increasing pixel values on the x-axis describe a position running radially outwards on the line under consideration £ _k . Accordingly, the brightness value for each pixel is plotted on the y-axis.

The blue line in 8th preferably shows the averaged brightness curve normalized to the range 0 ... 1 £ ^β of the individual lines £ _k out 7 . The yellow line preferably shows the likewise averaged and normalized courses of the individual lines £ _k from the actual pixel representation of the used punch rivet die according to 5 b . The red line also preferably shows the averaged and normalized courses of the individual lines £ _k from the actual pixel representation of the defective or damaged punch rivet die according to 5 c .

The red line in the area of pixels 350 to approximately 450 preferably shows the defective location. Due to the breakout in the punch rivet die profile, the reflection or brightness is significantly lower on average than in the brightness profiles of the averaged lines £ _k the punch rivet die M in original condition or in used condition.

Für den Vergleich zwischen der Ursprungs-Pixeldarstellung des Vergleichsbereichs V der ungebrauchten oder neuen Stanznietmatrize M (Bezeichnung „neu“) mit der Ist-Pixeldarstellung des Vergleichsbereichs V der defekten Stanznietmatrize M (Bezeichnung „def.“) ergibt sich $r{}_{\overline{{\text{£}}_{neu}^{\beta }},\overline{{\text{£}}_{def.}^{\beta }}}=\mathrm{0,75.}$

The results of the correlation calculation are also preferably confirmed on the basis of the averaged brightness profiles. If the averaged brightness profiles are correlated or linked, the following correlation coefficients r result. For the comparison between the original pixel representation of the comparison area V the unused or new punch rivet die M (Description "new") with the actual pixel representation of the comparison area V the used punch rivet die M (Term "used") results $r{}_{\overline{{\text{£}}_{neu}^{\beta }},\overline{{\text{£}}_{Gebr.}^{\beta }}}=\mathrm{0.92.}$

For the comparison between the original pixel representation of the comparison area V the unused or new punch rivet die M (Description "new") with the actual pixel representation of the comparison area V the defective punch rivet die M (Name "def.") Results $r{}_{\overline{{\text{£}}_{neu}^{\beta }},\overline{{\text{£}}_{def.}^{\beta }}}=\mathrm{0.75.}$

Another preferred use of these lines of brightness is also possible £ _k in the type of punch rivet die used M to recognize. As an example shows 9 a to c three different matrices. The punch rivet dies of the 9 a and b are of the same type.

The corresponding brightness gradients of the lines are preferred £ _k for the die geometries of the 9 ac in 10 shown. The correlation factor r of the matrix 9 b according to the die 9 a is preferably 0.98. The correlation factor r of the matrix 9 b according to the die 9 c is preferably only 0.32. The type of die 9 b corresponds to that of the matrix 9 a , but not that of the die 9 c . This is documented by the correlation factor r <0.35.

This method can preferably be expanded to compare several types of matrices. The correlation factors r are calculated for each individual matrix type. Based on the closest match, i.e. H. of the largest correlation factor r, the corresponding matrix type can be determined or verified. This method is also preferably applicable to used matrices.

According to a further preferred embodiment of the evaluation method according to the invention, the following method is used: instead of the brightness curves of the lines £ _k out 7 to average, the individual brightness gradients of the lines £ _k according to the following formula with the brightness curves of all other lines £ _k correlated and the correlation coefficients averaged: $$\overline{r_b}=\frac{1}{k}{\displaystyle \sum _{i=1}^k{r}_{{\text{£}}_b^{\beta },{\text{£}}_i^{\beta }},i\ne b,b=\left\{\mathrm{1,}...,k\right\}}$$

Accordingly, the ten brightness gradients of the lines result £ _k according to 6 preferably ten averaged correlation coefficients r according to the above formula.

This is the preferred method for punch rivet dies M of the 5 a to c carried out, the result is in 11 shown history. In 11 the blue curve represents the averaged correlation coefficient for the brightness curves of the lines £ _k in the original pixel representation (see 5 a) . The course of the blue curve also corresponds to the expectation. The punch rivet die M in the comparison area V the original pixel representation is uniform and undamaged, so that the brightness gradients are very similar to one another throughout. This is reflected in the high correlation factor of over 0.9.

It is similar with the actual pixel display for the used punch rivet die M according to 5 b . The averaged correlation coefficients are in yellow in 11 shown. This shows the wear in the form of uneven courses and therefore lower correlation coefficients with values around 0.87.

The brightness gradients of the lines provide another result £ _k the defective or damaged punch rivet die M according to 5 c . The averaged correlation coefficients r are in red in 11 shown. The averaged correlation factors r are due to the breakout (a significant drop at the location of the die defect) and the resulting different brightness gradients of the lines £ _k lower at around 0.7. This is because the brightness gradients that sweep over the break point are hardly similar to the other gradients on the lines £ _k have.

The described methods can not only be applied to punch rivet dies. It is also preferred to apply these to any surfaces or workpieces, preferably with rotationally symmetrical geometry, in which a course of the existing pixel properties (brightness, color, saturation) can be viewed and compared with known (course) patterns.

According to a further preferred embodiment of the methods described above, the center of the rotationally symmetrical punch rivet die becomes M determined more precisely in the pixel representations to be evaluated. In this way, the rotational symmetry of the punch rivet die can be compared V make optimal use of it. Would be the brightness gradients of the lines £ _k not from the center of the rotationally symmetrical pixel representation in the comparison area V start, these would be too different from the start to be able to use the correlation coefficient r to make statements about the state of the matrix.

12th shows the lines £ _k with k = 1 ... 400 similar to in 6 , except that they run radially outwards from the center of the pixel representation. The center of the pixel representation does not match the center of the punch rivet die M match.

Since the center of the die is unknown, we prefer to use it per line £ _k Starting from the center point of the pixel representation, the three pixels are determined at which the brightness difference to the next pixel is maximum. The corresponding mathematical rule for this is shown in the following equation $$p_{max}^{Grad}=\underset{i=1...l-1}{\text{Max}}\left|p_{i+1}^{\beta }-p_i^{\beta }\right|$$

The pixels obtained in this way are in 13 shown in green. The circle in the middle shows the geometric center of gravity p _{sp of} the point cloud. This is preferably calculated by averaging all the X and Y values of all the pixel coordinates of the point cloud.

The coordinates of the point cloud preferably result from a selected area in the pixel representation considered. According to different preferred embodiments of the present invention, this area has different sizes, which is included in the calculation of the geometric center of gravity.

In the preferred embodiment according to 12th 400 lines are considered. From these lines, three pixels with a maximum difference in brightness are determined. This evaluation of the 400 lines provides a total of 1200 pixels, which are used as an illustration or symbolically as small green dots in 13 are shown. Because only these green dots / pixels made up of 400 lines 13 form the point cloud, only the coordinates of these pixels are included in the calculation of the center of gravity of the point cloud.

Furthermore, according to the invention, only the pixels that are on a (circular) contour of the punch rivet die are preferably used for the circle calculation M lie. These pixels are preferably used to determine the center of the matrix. To determine which pixel lies on which circular contour, the pixels are preferably segmented, ie they are assigned to different clusters. A cluster thus preferably corresponds to a circular contour in the pixel representation of the punch rivet die. The distance from the pixel to the center is used as a measure of the membership of such a cluster. This is based on the assumption that pixels with a similar or even identical distance are likely to lie on the same circular contour.

Since the pixel coordinates of the true center of the matrix are unknown, the center of gravity of the point cloud is used as the center.

The distance of each pixel to the assumed center point is then preferably calculated. The colored upper bar in shows an example histogram of these distances 14 . The preferred assignment to the clusters is not always clear. Therefore, membership of a cluster is preferably modeled using a set of Gaussian functions. Each function forms such a cluster with a characteristic expected value µ, a standard deviation σ and a weighting ω. The weighting preferably takes on values between 0 and 1. This also preferably indicates in percentage terms how many pixels are assigned to this cluster or this Gaussian function.

The distance values are preferably assigned to the Gaussian function with the highest probability in this area. In 14 there are a total of four segments and the corresponding distance values are colored red, blue, green and black accordingly. The dominant distribution with the least variance is the Gaussian curve to which the blue values are assigned. With these pixel coordinates, the circle is then preferably determined and its center is defined as the center of the matrix.

This type of clustering is known as the "Gaussian mixture model". The properties of the individual Gaussian distributions are calculated iteratively using the so-called Expectation Maximization algorithm (EM algorithm for short). Overall, the method is referred to as the maximum likelihood estimation method (ML estimator for short).

Assume there are the distance values a _i with i = {1 ... n}, which are preferably assigned to k different clusters - that is, Gaussian distributions. Then the EM algorithm preferably runs in four steps (the letter m in brackets and superscript stands for the number of runs):

Initialization:

und berechne den Wert der sog. Likelihood-Funktion l. Diese gibt an, wie gut die Gaußverteilungen die gegebenen Abstandsdaten beschreiben: $${\mathcal{l}}^{\left(0\right)}=\frac{1}{n}{\displaystyle \sum _{i=1}^{n}log}\left({\displaystyle \sum _{j=1}^k{\omega }_j^{\left(0\right)}\varphi \left({\alpha }_i|{\mu }_j^{\left(0\right)},{\sigma }_j^{\left(0\right)}\right)}\right),$$

mit der Gaußverteilung $$\varphi \left({\alpha }_i|{\mu }_j,{\sigma }_j\right)\triangleq \frac{1}{\sqrt{2\pi \sigma }}{\text{e}}^{-\frac{{\left({\alpha }_i-{\mu }_j\right)}^2}{2\sigma }}.$$

Choose first estimates for the Gaussian distributions ${\omega }_j^{\left(0\right)},{\mu }_j^{\left(0\right)},{\sigma }_j^{\left(0\right)},j=\left\{1...k\right\},$

and calculate the value of the so-called likelihood function l. This indicates how well the Gaussian distributions describe the given distance data: $${\mathcal{l}}^{\left(0\right)}=\frac{1}{n}{\displaystyle \sum _{i=1}^{n}lOG}\left({\displaystyle \sum _{j=1}^k{\omega }_j^{\left(0\right)}\varphi \left({\alpha }_i|{\mu }_j^{\left(0\right)},{\sigma }_j^{\left(0\right)}\right)}\right),$$

with the Gaussian distribution $$\varphi \left({\alpha }_i|{\mu }_j,{\sigma }_j\right)\triangleq \frac{1}{\sqrt{\mathrm{2nd}\pi \sigma }}{\text{e}}^{-\frac{{\left({\alpha }_i-{\mu }_j\right)}^{\mathrm{2nd}}}{\mathrm{2nd}\sigma }}.$$

E step:

Dieser Wert bezeichnet den Schätzwert der Wahrscheinlichkeit bei der m-ten Iteration, dass der i-te Abstandswert a_i der j-ten Gaußverteilung zugehörig ist: $${\gamma }_{ij}^{\left(m\right)}=\frac{{\omega }_j^{\left(m\right)}\varphi \left({\alpha }_i|{\mu }_j^{\left(m\right)},{\sigma }_j^{\left(m\right)}\right)}{{\displaystyle {\sum }_{l=1}^k{\omega }_l^{\left(m\right)}}\varphi \left({\alpha }_i|{\mu }_l^{\left(m\right)},{\sigma }_l^{\left(m\right)}\right)},i=\left\{1\dots n\right\},$$

und $$b_j^{\left(m\right)}={\displaystyle \sum _{i=1}^n{\gamma }_{ij}^{\left(m\right)}}.$$

Calculate the value for j = {1 ... k} ${\gamma }_{ij}^{\left(m\right)}.$

This value denotes the estimated value of the probability during the m th iteration that the i th distance value a _{i belongs to} the j th Gaussian distribution: $${\gamma }_{ij}^{\left(m\right)}=\frac{{\omega }_j^{\left(m\right)}\varphi \left({\alpha }_i|{\mu }_j^{\left(m\right)},{\sigma }_j^{\left(m\right)}\right)}{{\displaystyle {\sum }_{l=1}^k{\omega }_l^{\left(m\right)}}\varphi \left({\alpha }_i|{\mu }_l^{\left(m\right)},{\sigma }_l^{\left(m\right)}\right)},i=\left\{1...n\right\},$$

and $$b_j^{\left(m\right)}={\displaystyle \sum _{i=1}^n{\gamma }_{ij}^{\left(m\right)}}.$$

M-step:

$${\mu }_j^{\left(m+1\right)}=\frac{1}{b_j^{\left(m\right)}}{\displaystyle \sum _{i=1}^{\text{n}}{\gamma }_{ij}^{\left(m\right)}{\alpha }_i,}$$

$${\sigma }_j^{\left(m+1\right)}=\frac{1}{b_j^{\left(m\right)}}{\displaystyle \sum _{i=1}^{\text{n}}{\gamma }_{ij}^{\left(m\right)}{\left({\alpha }_i-{\mu }_j^{\left(m+1\right)}\right)}^2}$$

Calculate the new estimates for j = {1 ... k}: $${\omega }_j^{\left(m+1\right)}=\frac{b_j^{\left(m\right)}}{b},$$

$${\mu }_j^{\left(m+1\right)}=\frac{1}{b_j^{\left(m\right)}}{\displaystyle \sum _{i=1}^{\text{n}}{\gamma }_{ij}^{\left(m\right)}{\alpha }_i,}$$

$${\sigma }_j^{\left(m+1\right)}=\frac{1}{b_j^{\left(m\right)}}{\displaystyle \sum _{i=1}^{\text{n}}{\gamma }_{ij}^{\left(m\right)}{\left({\alpha }_i-{\mu }_j^{\left(m+1\right)}\right)}^{\mathrm{2nd}}}$$

Checking the termination criterion:

und gehe zu Schritt 2, falls |l^(m+1) - l^(m) | > δ für eine zuvor gewählte Abbruchgrenze δ oder falls die maximale Anzahl der Iterationsschritte m noch nicht erreicht ist. Andernfalls beende den Algorithmus.Calculate the new likelihood function: $${\mathcal{l}}^{\left(m+1\right)}=\frac{1}{b}{\displaystyle \sum _{i=1}^{\text{n}}lOG}\left({\displaystyle \sum _{j=1}^{\text{k}}{\omega }_j^{\left(m+1\right)}\varphi \left({\alpha }_i|{\mu }_j^{\left(m+1\right)},{\sigma }_j^{\left(m+1\right)}\right)}\right),$$

and go to step 2nd if | l ^{(m + 1)} - l ^(m) | > δ for a previously selected termination limit δ or if the maximum number of iteration steps m has not yet been reached. Otherwise, exit the algorithm.

The result of this calculation based on the data 13 is preferably illustrated in a histogram. This serves for an overview and evaluation. The exemplary histogram 14 does not build on the data from the above evaluation, but shows the qualitative content of a usable histogram. It qualitatively illustrates the Gaussian mixture model, in which points of the same color are assigned to the same cluster.

Detached from the histogram in 14 shows 15 the different clusters from the pixel display also via the qualitative assignment of the colors.
With the points from the most significant respective clusters - i.e. the clusters to which the most pixels are assigned - circles are preferably calculated using the “Modified Least Squares Circle Fit” method. 16 shows the calculated circles of the three main clusters.

As already mentioned, the center of gravity of all points is preferably used at the beginning as a first estimate for the center point of the rotationally symmetrical pixel representation. Depending on the location of the points, this estimate is inaccurate. Therefore, the cluster with the most assigned points is preferably selected after the first execution of the EM algorithm and the MLS calculation is applied to it. The calculation provides a circle and thus a new value for the center. The ML clustering is preferably carried out again based on this.

Similar to the EM algorithm, this process is carried out until the position of the center point no longer changes. This can be seen in the 15 to 17th see with the different colored circles in the middle. The first center point (see above) is the calculated center of gravity (see above; shown here in green). In 17th it is under the red circle. After further iterations, the center point corresponds to the red point and then the point in magenta. The last iteration level is shown as a dot in cyan. With each more precisely calculated center, the subsequent clustering also improves. This procedure also works with defective dies, such as 17th shows.

It is also preferred to use the punch rivet die M for recording the actual pixel representation in the comparison area V using a robot or equivalent mechanism to position it under the camera. In this context, it is preferred to select the center point determined in the previous measurement as the starting point for this positioning. This preferably speeds up the calculation since fewer iteration steps are necessary. It also compensates for any inaccuracies in positioning.

Fixed limits are preferably specified for the correlation coefficients r. From these value ranges of the correlation coefficients it can be derived when a matrix is considered new, used, dirty or defective. In addition, it can preferably be derived which matrix type it corresponds to in comparison to known matrix types.

If you take the consideration off, for example 11 as a basis for decision-making, one would prefer to classify values> 0.9 as new matrices, values between 0.9 and 0.8 as used matrices and everything below as defective or at least as heavily soiled or damaged.

With this heuristic approach it can e.g. For example, incorrect evaluations can occur even if the threshold value is slightly undershot, which is why artificial neural networks (KNN) are preferably used to classify the state of the matrix.

As is known, artificial neural networks consist of individual elements, the neurons, which are linked to one another and together form the neural network. The "circuit technology" of artificial neurons usually knows several inputs and one output. If the sum of the Input signals exceed a certain threshold value, the neuron "fires", ie it sends a signal at the output to connected neurons (according to the neurobiological analogy of an action potential that a nerve cell emits when its membrane potential changes critically). How strongly this activation is received by the connected neuron depends on the weighting of the connection.

The topology of the network must be adapted to the task of the neural network; in our case, this is done by weighting the connections from neuron to neuron and by adapting the respective threshold values. This representation of a neuron is known under the name perceptron and is preferably used in the form of feed-forward perceptron networks as an artificial neural network.

The number of hidden, input, output and bias neurons must be determined in advance. Then the weights within the KNN are trained on the basis of known input and output states. This is achieved with the help of monitored learning processes, here with the so-called error propagation (English back propagation). This algorithm works as follows: (1) An input pattern is created and propagated forward through the network. (2) The network output is compared to the desired output. The difference between the two values is considered an error. (3) The error is now propagated back through the output to the input layer. The weights of the neuron connections are changed depending on their influence on the error. This guarantees an approximation to the desired output when the input is created again.

The input data are preferably adapted to the problem so that the artificial neural network can successfully carry out its task. In this case, for example, it is not sufficient to feed only the image data into the neural network and to derive a decision from this. Instead, the feature parameters, such as the above calculation of the correlation coefficient, are calculated and used as input data for the neural network.

The output layer consists of only two neurons, which stand for the classification "functional" or "defective". During the use of the punch rivet matrices, the network is trained with the calculated correlation coefficients and / or other parameters of defective, damaged, dirty and intact matrices in order to provide corresponding evaluation results.

According to a preferred embodiment of the present invention, a neural network in the form of a single-layer feed-forward perceptron with two input and two output neurons has been implemented. An input neuron is preferably assigned the value of the total correlation factor as defined above in the preferred embodiments of the invention. A vector with values from the correlation curve is preferably assigned to the other input neuron, as preferably described in claim 10. This curve is represented by the red curve in 11 .

The two output neurons stand for the states OK (OK) and NOK (not OK). They each assume values in the range between 0 and 1.

This neural network was preferably trained with 15 matrix test images by back propagation. Subsequently, five matrix mappings in the neural network unknown to the neural network were evaluated; all with correct results.

The higher value in the two output neurons was the preferred decision. This neural network could of course be expanded and improved. For example, if the results from the decision of the neural network are not clear, a worker at the customer could carry out the classification in OK or NOK himself and thus train the neural network further.

Reference list

10th

Self-piercing rivet die

12, 14

Indentations

M

Central axis

D

Top view

V

Comparison range

C.

computer

M

camera

QUOTES INCLUDE IN THE DESCRIPTION

This list of documents listed by the applicant has been generated automatically and is only included for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• EP 0127751 A1 [0005]
• WO 2000/15082542 [0006]

Claims (13)

A computer-aided optical status evaluation of an object, in particular a tool, such as a die or a stamp, a component, a joint or a connection, in which an optical recording of an actual state of the object in the form of an actual pixel representation with an optical recording of an initial state the object or a comparable object in the form of an original pixel representation is connected by the following steps: a. Definition of a flat comparison area of the object, which can be changed due to an external influence and can be optically detected via an external accessibility, b. Creating an image of the comparison area of the object with an optical camera and storing the image as an actual pixel representation of the comparison area, which represents an actual state of the object in the comparison area, c. Comparing the actual pixel representation of the comparison area with the original pixel representation of the comparison area, which comprises the comparison area of the object in the original state or the comparison region of a comparison object corresponding to the object in the original state, d. Determining a correlation factor from the comparison of the actual pixel representation with the original pixel representation and e. Assigning the determined correlation factor to a state value range, from which a state of the object can be determined on the basis of the evaluated comparison region of the object on the basis of the determined correlation factor. Condition assessment according to Claim 1 , with the further steps: representation of the actual pixel representation with n · m pixel values according to {x _1,1 ... x _{n, m} }, representation of the original pixel representation with n · m pixel values according to {y _1,1 ... y _{n, m} } Calculate the correlation coefficient for both representations according to $$r=\frac{{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^m\left(x_{i,j}-\overline{x}\right)\left(y_{i,j}-\overline{y}\right)}}}{\sqrt{{{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^m\left(x_{i,j}-\overline{x}\right)}}}^{\mathrm{2nd}}}\sqrt{{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^m{\left(y_{i,j}-\overline{y}\right)}^{\mathrm{2nd}}}}}},$$

which represents a change in state between the related objects of the compared representations. Condition assessment according to Claim 2 , with the further steps: recording temporally spaced and successive actual pixel representations of the comparison area as at least the current actual pixel representation and previous actual pixel representation and calculating the correlation factor from a comparison of the current actual pixel representation and the actual actual pixel representation preceding Pixel representation. Condition assessment according to one of the preceding claims, in which the object is rotationally symmetrical. Condition assessment according to Claim 4 , in which the rotationally symmetrical object is a rotationally symmetrical die, a rotationally symmetrical joint or a rotationally symmetrical stamp surface. Condition assessment according to Claim 5 in combination with Claim 2 , with the further step: defining a plurality of star-shaped lines £ _k with k = 1 ... n in the actual pixel representation based on the Bresenham algorithm, which each have a starting point and one at a symmetrical center of the actual pixel representation Provide brightness curve based on the pixel data per line £ _k . Condition assessment according to Claim 6 with the further steps: displaying the plurality of lines £ _k each as a brightness curve ${\text{£}}_k^{\beta }=\left\{p_1^{\beta },...p_l^{\beta }\right\}$

based on the pixel values, comparing the lines £ _k with one another and recognizing deviations of the lines £ _k with one another in order to qualify damage to the matrix. Condition assessment according to Claim 5 with the further steps: calculate a mean value line in each case £ ^β from the plurality of lines £ _k as the brightness curve for the matrix in the initial state using the original pixel representation and for the matrix at least in the actual state using the actual pixel representation and comparing the mean value lines £ ^β for the matrix in the initial state and for the matrix at least in the actual state. Condition assessment according to Claim 5 with the further steps: calculating an average line £ ^β from the plurality of lines ${\text{£}}_{neu}^{\beta },{\text{£}}_{def}^{\beta },$

as a brightness curve for a matrix in the initial state on the basis of the original pixel representation and for a matrix at least in the actual state on the basis of the actual pixel representation and calculating the correlation factor $r_{\overline{{\text{£}}_{neu}^{\beta }},\overline{{\text{£}}_{def.}^{\beta }}},$

between the mean line ${\text{£}}_{neu}^{\beta }$

the matrix in the initial state and the mean value line ${\text{£}}_{def}^{\beta },$

the die in the actual state and determining a degree of damage to the die based on the calculated correlation factor $r_{\overline{{\text{£}}_{neu}^{\beta }},\overline{{\text{£}}_{def.}^{\beta }}}.$

Condition assessment according to Claim 5 , with the further steps: calculating the correlation factor between each line £ _k and the other lines £ _{k of} the plurality of lines £ _k for the matrix in the initial state and in the current state, calculating the mean value of the calculated correlation factors for the matrix in the initial state and calculating the mean value of the calculated correlation factors for the matrix in the current state and determining a degree of damage to the matrix in the current state on the basis of a comparison between the average correlation factors of the matrix in the current state and in the initial state. Condition assessment according to one of the preceding Claims 5 to 10 with the further steps: determining the center of the matrix in the actual pixel representation and in the original pixel representation. Condition assessment according to Claim 11 , in which the center point is determined as follows: Defining a plurality of star-shaped lines £ _k with k = 1 ... n in the actual pixel representation as a brightness curve ${\text{£}}_k^{\beta }=\left\{p_1^{\beta },...,p_l^{\beta }\right\}$

based on the pixel values and the Bresenham algorithm, determining three pixels per line £ _k starting from the center, at which the difference in brightness with respect to the subsequent pixel is maximum according to $$p_{max}^{Grad}=\underset{i=1...l-1}{\text{Max}}\left|p_{i+1}^{\beta }-p_i^{\beta }\right|,$$

Segmentation of the plurality of pixels into different circular contours of the matrix by modeling a cluster for each circular contour using a set of Gaussian functions according to the Gaussian mixture model, then determining the dominant Gaussian distribution with the least variance and determining the circle from the pixel coordinates and its center and setting the center point as the matrix center point. A neural network with which a condition assessment of the same tools, preferably a punch rivet die, is carried out for at least one user in the company, in which a result of a condition assessment according to one of the preceding claims can be processed.

Images