DE10252340B4 - Device for detecting a structure to be applied to a substrate and suitable methods therefor - Google Patents

Abstract

Device for detecting a structure to be applied to a substrate, comprising a lighting module and a sensor unit, wherein the sensor unit (3) is provided on the device (1) for applying the structure,
wherein the lighting module is an LED lighting module which emits the areas red, blue, green, infrared and / or ultraviolet,
wherein the evaluation unit sets a set of calipers over the data set determined with the picture elements, the calipers being orthogonal to the substrate track, and
wherein the evaluation unit performs the structure determination according to at least one of the following criteria:
a. edge thickness
b. structure width
c. Target-position difference

Description

The following invention relates to a device for detecting a structure to be applied to a substrate according to claim 1 and to a suitable method therefor.

State of the art

Conventionally, so far for detecting a structure to be applied to a substrate structure optical measurements are performed, often different systems for fully automatic testing of the structure, u. a. Adhesive and Dichtmittelraupen be used. For this purpose, one or more video cameras are directed to the structure to be recognized. In addition, a lighting module is required, which serves to produce a high-contrast camera image. The structure is checked in staggered time, a few seconds after the structure has been applied to the substrate. Frequently, however, the check is only carried out after the complete structure application has taken place on the substrate. The disadvantage of this is that thus the review is carried out separately and independently of the application process, which is sometimes cumbersome and difficult to handle. So far, the verification systems have been too unstable and cumbersome to parameterize.

task

It is therefore an object of the present invention to further develop the known device for recognizing a structure to be applied to a substrate and a suitable method such that, on the one hand, a direct check of the applied structure is possible, and, on the other hand, the check simply with improved detection and evaluation of the applied structure to handle.

Furthermore, it is an object to develop the known method for detecting a structure to be applied to a substrate, even in the case of subsequent verification, in such a way that, on the one hand, a subsequent check is simply possible and, on the other hand, a precise error analysis of the structure to be applied is provided.

These objects are achieved with respect to apparatus with the features of claim 1 and procedurally with the features of claim 14.

According to the application, the sensor unit is provided on the device for applying the structure. With this measure, a vision system is provided in a compact design, wherein preferably the lighting module can also be provided on the device for applying the structure. In this way, it is possible to integrate the device according to the application into existing systems whose task is to apply a structure to a substrate. If, in procedural terms, the structure is determined during the application of the substrate, it is possible, in the event of an error, to intervene directly in the production process or to separate out the defective substrate. This provides increased efficiency in fabricating structures on a substrate. Furthermore, the inspection area of the structure to be determined is set on the basis of bases along the structure to be recognized, the handling is straightforward, since the interactive process between the user and the structure shown is easily produced by today's means. If, according to the application, the tolerance range is defined along the reference line defined by the interpolation points, inaccuracies in the structure are possibly taken into account and, in particular, the quality check of the structure to be determined can be determined individually with this measure. With this simplified user interaction even complex lanes of the structure can be taught in easily and efficiently. Also, with the existing representation with the structure to be recognized and the reference line created by the support points, the user will immediately be informed as to whether changes in the structure course are present.

Further advantageous embodiments of the present subject matter are the subject of the dependent claims.

If the sensor unit is positioned directly at the output of the device for applying the structure, a compact and highly integrated design of the device according to the application is possible. The sensor unit is thus able to carry out a fully automatic high-speed inspection of the structure almost immediately after the structure application.

If the sensor unit has a video sensor, then conventional image recognition methods can be used. If the video sensor preferably has one and / or several picture lines, a maximum of 15 lines, a high image recording rate of the structure can be achieved. In this way, the device continues to build small and the evaluation can be done in the sensor unit. An external data evaluation device is therefore not required.

If a white light illumination module is used as the illumination module, then conventional halogen lamps can also be used for white light generation.

If an LED illumination module is used as the illumination module, the illumination of the sensor can be made available by an intelligent combination of different spectral ranges in order to increase the contrast between the background and the structure. The evaluation as such can thus be stable and the effort for the evaluation logic is also minimized. The same applies in particular if a plurality of lighting modules are provided, which can thus provide increased contrast sharpness. If the evaluation unit is likewise integrated in the sensor unit, then the setting of the quality criteria can be supplied in an easy way to the device according to the application via an external operating unit. Advantageously, the transmission via radio, infrared data or cable.

If, in procedural terms, the structure determination is carried out by means of so-called calipers, which are preferably orthogonal to the substrate structure, special measures, preferably crossing regions, between the caliper line and a contrast structure in the region to be determined can be determined by this measure. If the calipers are orthogonal to the substrate structure, in particular a width determination can be brought about in a simple way for the structure. In conjunction with a corresponding visualization software, the structure history and corresponding error areas can then be displayed. The user thus recognizes right away whether the structure pattern corresponds to given tolerances or the structure is applied inaccurately. It is also advantageous if, for example, the given substrate data, such as recesses and elevations, are used for the structure determination and corresponding error analysis, since this makes it possible to make more exact statements about the structure profile.

It has proven to be advantageous if the structure determination takes place via the evaluation of the brightness characteristic of the gray values along the caliper. On the basis of the gray values, it is thus possible to determine where an area to be determined is to be used for the structure check, in particular a position can be established where the change of object and background is strongest. This is achieved by using the second derivative in the course of the gray values for structure determination. The values to be determined are determined exactly as subpixels. If a set of hypotheses is created for each caliper, in particular at four nodes of the caliper there will be a set of six possible variations which are different in each case due to the positional spacing of the individual nodes of the caliper.

If adjacent sets of hypotheses are now linked to one another, particular values can be assigned, in particular by using a heuristic function, by means of which the respective nodes which are suitable for the structure edge can be determined.

Further advantageous embodiments of the invention are the subject of the remaining dependent claims.

With reference to the following drawings advantageous embodiments of the invention will be illustrated.

1 schematically shows an advantageous embodiment of the device according to the application.

2 shows a subsection of in 1 applied structure.

3 shows an error analysis.

4 shows the use of calipers on an area to be determined which contains both structure and perturbations.

5 shows the crossing points of the relevant contrast lines and the caliper.

6 shows the generation of a hypothesis set from a caliper.

7 shows the structure determination from adjacent sets of hypotheses.

8th shows the method for the determination or elimination of interference edges or determination of the structure.

In 1 is a facility 1 for applying a structure 9 on a substrate 7 shown. The device 1 is conventionally adjustable in the x, y and z directions. It is also conceivable that the device is rigid and the substrate can be adjusted in the x-, y- and z-direction. The device 1 also has a sensor unit 3 immediately positioned in this embodiment at the exit of the device for applying the structure. In addition, in this schematic representation is also the lighting module 5 characterized, which provides the contrast sharpness when applying or when registering the areas to be observed. In this embodiment, it can be seen that a so-called adhesive bead 9 in a prefabricated recess 13 in the substrate 7 is applied or introduced. With the reference number 11 is marked with the hatched lines an image area that in 2 is reproduced more clearly.

In 2 is for example the recess 13 represented in which the structure or adhesive bead 9 is introduced. On the one hand, this selection area can be located in the evaluation unit in the sensor unit 3 However, it can basically be displayed during the application to the user, so that the user manually his bases 20 can set, based on which a reference line 22 can be generated. As in 2 is clearly visible, is to the reference line 22 , which approximately reproduces the structure pattern, defines a tolerance range, which in this case is equally spaced from the reference line. According to the application, it is thus checked whether the reference line defined by the interpolation points lies within the tolerance range. In addition to the tolerance range is in 2 a test area 26 represented, within which the structure is to be found.

In 3 For example, an error representation is reproduced which, in addition to the positioning of the defect of the structure order, also indicates the defect size to the user on the basis of the evaluation accuracy of the method according to the application. On the basis of the error size, the user can then decide whether the deviation from the target value is tolerable or the production process should be aborted. With the method according to the application can thus be decided on the basis of the immediate review of the structure order during the manufacturing process, and indeed fully automatically, if the manufacturing process must be interrupted and / or the faulty substrate must be discarded.

Based on 4 to 8th the evaluation method according to the application is described. In 4 the so-called edge extraction of the abnormalities in the inspection area is reproduced. For this purpose, a set of calipers, which are preferably orthogonal to the track of the structure, is laid over the inspection area, wherein the extraction of the edges by the evaluation of the brightness characteristic of the gray values along the caliper is thus orthogonal to the pattern track. This will detect a position that reflects the change of object and background where the change is strongest. This is achieved by calculating the second derivative in the course of the gray values. The values to be determined are precisely determined subpixels.

In 5 After the edge extraction, the tracking of the structure is shown, wherein for each line, all found edges are represented by the nodes.

6 shows that for every Caliper the 4 and 5 a hypothesis sentence is created, for example, for four nodes of a caliper altogether six position hypotheses are present. Subsequently, the Caliper hypotheses are linked step by step, preferably hierarchically, with the corresponding neighbors or adjacent hypothesis sets. This linkage is done as in 7 is shown iteratively. For this purpose, more and more left and right hypotheses are generated, which in turn are linked together or evaluated with a heuristic function. A selection criterion for determining the structure determination may be, for example, that the higher the value determined, the better the hypothesis used.

In 8th clearly illustrates how the iterative procedure of each set of hypotheses is applied. Here, for example, the hypotheses 2, 3, 4 in 6 combinatorially connected (I-II, I-III, II-III, II-II), wherein in each case the left hypothesis of Hypothesis 3 are connected in accordance with the right hypothesis. This again results in an assignment of the hypotheses, whereby a value is determined on the basis of the heuristic function. Due to the predetermined definition, the larger the value, the better the hypothesis is, then the structure determination can be done by taking out, if the number of hypotheses so developed exceeds the number of allowed hypotheses per existing node, the hypotheses set with a lower value of the heuristic function.

• 1. Kantenstärke
• 2. Strukturbreite
• 3. Soll-Ist-Positions-Differenz
• 4. Co-Lineraität der IST-Position
• 5. Soll-Ist-Strukturbreiten-Differenz
• 6. Co-Lineraität der IST-Strukturbreite
• 7. Soll-Ist-Strukturhelligkeit-Differenz
• 8. Co-Lineraität der Ist-Strukturhelligkeit
• 9. Soll-Ist-Hintergrundhelligkeits-Differenz
• 10. Co-Lineraität der Ist-Hintergrundshelligkeit

With these methods according to the application, it is possible to bring about a structural determination precisely and with small data records, so that an immediate structural determination, for example, during the application of the structure, is possible. It should be noted that the heuristic function uses the following criteria to determine the specified value.

• 1. edge strength
• 2. Structure width
• 3. Target / actual position difference
• 4. Co-Linerity of the actual position
• 5. Target-actual structure width difference
• 6. Co-Linerity of the actual structure width
• 7. Set-actual-structure brightness difference
• 8. Co-Linerity of the actual structural brightness
• 9. Target-actual background brightness difference
• 10. Co-Linerity of the actual background brightness

On the basis of the concrete realization that the device according to the application is used in the application of a bead of adhesive to a substrate, it is advantageous if the following is observed. According to an advantageous embodiment, the system or device according to the application essentially consists of a color line video sensor with integrated evaluation unit and illumination for highlighting and illuminating the sealing or adhesive bead. The components are housed in a compact protective housing. The visual inspection system is mounted directly behind the adhesive application system (application nozzle) and is aligned with the area just behind the adhesive nozzle in order to carry out a test directly after the bead application. The test is thus carried out directly after the application of the sealing material or the adhesive, so that even during the application, an evaluation of the quality of the bead (outlines, position and position, thickness) can take place.

In contrast to the known approaches, a video sensor is used in this invention, which takes only one or more picture lines (maximum 15 lines) in order to achieve a high image pickup rate. The evaluation takes place in the color line video sensor with an integrated evaluation unit. An external data evaluation device (evaluation PC) is not required because a miniaturized evaluation computer is already available in the video sensor. The quality criteria (IO / NIO limits) are set by means of an external control unit which is connected to the sensor via a radio link, infrared data transmission connection (IrDa) or a cable connection (serial or network).

• • Weißlichtbeleuchtungsmodul, z. B. Halogenlampe
• • LED-Beleuchtungsmodul in verschiedenen Farben

eingesetzt.To illuminate the adhesive trace, depending on the surface properties of the adhesive or sealant one or more

• • white light illumination module, z. B. halogen lamp
• • LED lighting module in different colors

used.

The lighting modules have a compact design so that they can be installed in a compact system structure (image sensor and lighting in a common housing). It is intended to combine different different lighting modules (design, color) with each other to produce a high contrast between background and adhesive by a clever combination of different spectral ranges of the illumination and the sensor. Thus, the evaluation can run stable and the cost of the evaluation logic can be kept low.

The visualization software is used to display errors when applying adhesive beads. For this purpose, the adhesive track to be removed is saved as a 3D track and the corresponding error areas are marked in it. The corresponding errors are highlighted with a different color and described with additional text.

The software or the sensor communicates with a robot or another control unit via all standard fieldbuses (Profibus, Interbus, Devicenet), Ethernet, serial interface, OPC server or other available communication interfaces.

In the offline version, the robot path is taught in advance and saved. After the adhesive application process, the visualization software can be selected. This fetches the corresponding error areas from the robot.

In the online version, the visualization software always receives the current position along the robot path while driving and, in the event of an error, an additional error code.

In addition, data can be transferred from CAD files. The data contained therein from the component, from the adhesive track o. Ä. Are processed with and displayed together with the corresponding error 3 dimensional or 2 dimensional.

To simplify the user interaction, a GUI was developed specifically for the adhesive bead inspection. With simple mouse clicks, complex lanes can now be taught in easily and efficiently. The graphic elements are designed in such a way that you can see the set limit values such as Min / Max ranges and tolerances at a glance ( 2 ). Track changes can also be made with a few mouse clicks. The adhesive track does not have to be taught in exactly, as the subsequent image processing operations are stable enough to compensate for the inaccuracies that have arisen during teaching. An additional view informs the operator about production errors. By clicking on the error that has occurred, the affected area is enlarged and the error description is output in plain text ( 3 ).

The following mathematical linkage is used to determine the heuristic function for the structure determination on the basis of the following criteria, namely a heuristic value for elementary hypotheses and a heuristic value for complex hypotheses.

A. Heuristic value for elementary hypotheses

x_weight1

Gewicht des ersten Punkts,

x_weight2

Gewicht des zweiten Punkts,

x_pos1

Position von erstem Punkt,

x_pos2

Position von zweitem Punkt,

x_br

Helligkeit von Strukrur,

x_bk

Helligkeit von Hintergrund,

für die soll Werte gilt: s = {s_width, s_br, s_bk} wobei:

s_width

soll Breite,

s_br

soll Helligkeit von Struktur,

s_bk

soll Helligkeit von Hintergrund,

mit den Heuristischen Koeffizienten: a = {a_const, a_weight, a_pos, a_width, a_br, a_bk} b = {b_pos, b_width, b_br, b_bk} For an input vector, the following applies: x = {x _weight1 , x _weight1 , x _pos1 , x _pos2 , x _br , x _bk }, in which:

x _weight1

Weight of the first point,

x _weight2

Weight of the second point,

x _pos1

Position of first point,

x _pos2

Position of second point,

x _br

Brightness of Strukrur,

x _bk

Brightness of background,

for which values should be: s = {s _width , s _br , s _bk } in which:

s _width

should width,

s _br

should brightness of structure,

s _bk

should brightness of background,

with heuristic coefficients: a = {a _const , a _weight , a _pos , a _width , a _br , a _bk } b = {b _pos , b _width , b _br , b _bk }

wobei:

e_width = abs(x_pos2 – x_pos1 – s_width), e_br = abs(x_br – s_br), e_bk = abs(x_bk – s_bk). The heuristic value h has the following form:

in which:

e _width = abs (x _{pos 2 -x pos 1 -s width} ), _ebr = abs ( _{xbr -sbr} ), e _bk = abs (x _bk - s _bk ).

B. Heuristic value for complex hypotheses

x_lpos

Position auf rechte Seite von linke Hypothese,

x_lwidth

Breite auf rechte Seite von linke Hypothese,

x_lbr

Strukturhelligkeit auf rechte Seite von linke Hypothese,

x_lbk

Hintergrundhelligkeit auf rechte Seite von linke Hypothese,

x_rpos

Position auf linke Seite von rechte Hypothese,

x_rwidth

Breite auf linke Seite von rechte Hypothese,

x_rbr

Strukturhelligkeit auf linke Seite von rechte Hypothese,

x_rbk

Hintergrundhelligkeit auf linke Seite von rechte Hypothese,

mit den Heuristischen Koeffizienten: a = {a_const, a_pos, a_width, a_br, a_bk} b = {b_pos, b_width, b_br, b_bk} For an input vector, the following applies: x = {x _lpos , x _lwidth , x _lbr , x _lbk , x _rpos , x _rwidth , x _rbr , x _rbk }, in which:

x _lpos

Position on right side of left hypothesis,

x _lwidth

Width on right side of left hypothesis,

x _lbr

Structural brightness on right side of left hypothesis,

x _lbk

Background brightness on right side of left hypothesis,

x _rpos

Left-side position of right hypothesis

x _rwidth

Width on left side of right hypothesis,

x _rbr

Structural Brightness on Left Side of Right Hypothesis,

x _rbk

Background brightness on left side of right hypothesis,

with heuristic coefficients: a = {a _const , a _pos , a _width , a _br , a _bk } b = {b _pos , b _width , b _br , b _bk }

wobei: e_pos = abs(x_lpos – x_rpos), e_width = abs(x_lwidth – x_rwidth), e_br = abs(x_lbr – x_rbr), e_bk = abs(x_lbk – x_rbk), und

h_left

heuristischer Wert von linker Hypothese

h_right

heuristischer Wert von rechter Hypothese

The heuristic value h has the following form:

in which: e _pos = abs (x _{lpos -x rpos} ), e _width = abs (x _{lwidth -x rwidth} ), e _br = abs (x _lbr - x _rbr ), e _bk = abs (x _{lbk -x rbk} ), and

h _left

heuristic value of left hypothesis

h _right

heuristic value of right hypothesis

Claims (21)

Device for detecting a structure to be applied to a substrate, comprising a lighting module and a sensor unit, wherein the sensor unit ( 3 ) on the facility ( 1 ) is provided for applying the structure, wherein the illumination module is an LED illumination module which emits the areas red, blue, green, infrared and / or ultraviolet, the evaluation unit laying a set of calipers over the data set determined with the picture elements, the calipers being orthogonal to the substrate track, and wherein the evaluation unit performs the texture determination according to at least one of the following criteria: a. Edge thickness b. Structure width c. Target-position difference Apparatus according to claim 1, characterized in that the sensor unit is positioned directly at the outlet of the device for applying the structure. Device according to one of claims 1 or 2, characterized in that the sensor unit comprises a video sensor, which registers one and / or a plurality of image lines. Device according to one of claims 1 to 3, characterized in that the illumination module includes a white light illumination module. Device according to one of claims 1 to 4, characterized in that a plurality of lighting modules are provided. Device according to one of claims 1 to 5, characterized in that an evaluation unit is provided, wherein the settings of the quality criteria via an external operating unit. Apparatus according to claim 1, characterized in that the structure is determined by the brightness curve of the gray values along the calipers. Apparatus according to claim 1, characterized in that the second derivative are used in the course of the gray values for structuring means. Device according to one of claims 1 to 8, characterized in that the evaluation unit for the Caliper creates a hypothesis set. Apparatus according to claim 9, characterized in that the evaluation unit linked adjacent hypothesis sets. Device according to one of claims 1 to 10, characterized in that the evaluation unit performs the structuring means according to at least one of the following criteria: a. Co-linearity of the actual position b. Target-structure width difference c. Co-linearity of the actual structure width d. Target-structure brightness difference e. Co-linearity of the actual structure brightness f. Target-background brightness difference G. Co-linearity of the actual background brightness Device according to one of claims 1 to 11, characterized in that on the position of the sensor unit and the Strukturermittelung a three-dimensional representation is made possible. Device according to one of claims 1 to 12, characterized in that via a connection to a network connection, via the Internet or intranet, a control and evaluation is provided. A method of detecting a structure and in particular for use in the device according to claims 1 to 13, comprising the steps of: a) providing a lighting module and a sensor unit provided on the device for applying the structure. b) determining the structure while the structure is being applied to the substrate, the illumination module being an LED illumination module emitting the areas red, blue, green, infrared and / or ultraviolet, wherein the evaluation unit sets a set of calipers over the data set determined with the picture elements, the calipers being orthogonal to the substrate track, and wherein the evaluation unit performs the patterning according to at least one of the following criteria: a. Edge thickness b. Structure width c. Target-position difference The method of claim 14, wherein the structure determination is carried out via a video sensor as a sensor unit with one or more image lines. Method according to one of claims 14 or 15, wherein the structure determination with at least one illumination module, which is a white light module and / or a LED lighting module with different colors, is performed. Method according to one of claims 14 to 16, wherein a visualization software is provided, with which the structure profile and corresponding error areas can be displayed. Method according to one of claims 14 to 17, wherein substrate data are used for the structure determination and corresponding error analysis. Method according to one of claims 17 or 18, wherein based on the visualization software different error ranges can be displayed separately. Method according to one of claims 17 to 19, wherein the structure determination takes place via the evaluation of the brightness characteristic of the gray values along the caliper. The method of any one of claims 17 to 20, wherein said patterning is performed according to at least one of the following criteria: a. Co-linearity of the actual position b. Target-structure width difference c. Co-linearity of the actual structure width d. Target-structure brightness difference e. Co-linearity of the actual structure brightness f. Target-background brightness difference G. Co-linearity of the actual background brightness

Images