DE102019120696A1 - Apparatus and method for tire testing - Google Patents

Abstract

The present invention shows a device for tire testing with a tire testing device, by means of which at least one recording of at least one area of a tire is generated, and with an evaluation device for evaluating the at least one recording. It is provided that at least one evaluation function of the evaluation device works by means of machine learning (ML).

Description

The present invention relates to a device for tire testing with a tire testing device by which at least one recording of at least one area of a tire is generated, and with an evaluation device for evaluating the at least one recording. For example, the tire testing device can be an optical tire testing device, in particular a shearographic tire testing device.

Such a device is for example from the publication EP 2 851 670 A2 known.

The evaluation of the recordings recorded by the tire testing device is in many cases still carried out manually by an operator who uses the recordings to identify defects in the tire and classifies the tire on the basis of the identified defects. Initial approaches to automating the evaluation of the recordings are based on algorithms for image evaluation and pattern recognition, which work, for example, on the basis of gray value statistics or edge recognition. However, these approaches have so far proven to be insufficiently reliable in the detection of defects and also have to be adapted to the respective measurement situation or the tire to be tested in a very complex manner.

The object of the present invention is therefore to provide an improved device and an improved method for testing tires. In particular, the evaluation should be improved.

This object is achieved by a device according to claim 1 and a method according to claim 12. Preferred embodiments of the present invention are the subject of the subclaims.

The present invention comprises a device for tire testing with a tire testing device by which at least one recording of at least one area of a tire is generated, and with an evaluation device for evaluating the at least one recording. It is provided that at least one evaluation function of the evaluation device works by means of machine learning (ML).

The inventors of the present invention have found that machine learning is particularly suitable for the special requirements that have to be met in the automatic evaluation of recordings during tire testing. The use of machine learning enables robust and precise evaluation on the one hand, and simpler adaptation to new tire types and / or measurement situations on the other.

In the context of the present invention, any machine learning method can initially be used for machine learning. For example, the use of statistical models is conceivable.

In one possible embodiment of the present invention, the at least one evaluation function works with deep learning (DL). In the context of the present invention, deep learning is understood to mean the use of an artificial neural network. An artificial neural network with a plurality of intermediate layers between the input layer and the output layer is preferably used here.

In one possible embodiment of the present invention, the artificial neural network used has an encoder-decoder architecture. Furthermore, the artificial neural network can include skip-connections, which make available results from earlier transformation layers, corresponding later layers and in particular make results from encoder layers directly associated decoder layers available, for example decoder layers with the same number of pixels in the input.

In one possible embodiment of the present invention, the artificial neural network used has residual blocks, which transform inputs from several layers and additionally offset the output with the input in order to obtain the result of the block. In one possible embodiment, several residual blocks can be connected in series.

The above-mentioned configurations of the artificial neural network can also be combined with one another.

In a possible embodiment of the present invention, the evaluation device separates a tire area and a background. This takes into account the fact that the recordings typically not only show the tire, but also a background, which can also include holding devices, for example.

In a first variant, the separation between a tire area and a background can be done by an algorithmic solution, e.g. B. by means of gray value statistics, and thus without the use of machine learning.

In a possible embodiment of the present invention, in addition to the algorithmic solution, a parameterization of the dividing line between the tire area and the background can be used. In particular, a fit with a primitive such as a spline, a circular arc or an elliptical arc can take place in the algorithmically generated pixel-by-pixel separation in order to incorporate known knowledge about the circular shape of the tire surface.

In a second variant, on the other hand, the separation between the tire area and the background takes place by means of at least one ML model.

For example, deep learning segmentation can be used to separate the tire area and background pixel by pixel.

In a possible embodiment of the present invention, in addition to the use of machine learning to separate the tire area and the background, a parameterization of the dividing line between the tire area and the background can be used. The parameterization can be done, for example, by a primitive such as a spline, an arc of a circle or an arc of an ellipse.

In a first variant, a fit with a primitive such as a spline, a circular arc or an elliptical arc can take place in the pixel-by-pixel separation generated by machine learning in order to incorporate known knowledge about the circular shape of the tire surface.

In a second variant, the parameters of the primitives can be estimated directly from the recording by means of machine learning.

The parameterization can be used in particular to evaluate recordings of the side surface of a tire, since here the tire area can always be approximated by a segment of a circle.

In a possible embodiment of the present invention, the evaluation device localizes defect areas and / or classifies defects and / or the tire.

In the context of the localization of defect areas, for example, pixel-precise segmentation and / or rough delimitation of defect locations on the recording can be carried out.

In the context of the classification of defects, these can be divided into classes that describe the type and / or size of the defects, for example. This can take place either on the basis of a localization of defect areas and / or directly from the recordings.

As part of the classification of tires, the tires can be divided into defect classes, for example into tires without defects and tires with defects, whereby the tires with defects can be further divided into several classes, which differ in terms of type and / or number and / or severity of the defects. The classification of tires can take place on the basis of a completed classification of the defects in the tires, on the basis of a completed localization of defect areas and / or directly from the recordings.

The manner in which the localization and / or classification is carried out is initially not restricted further within the scope of the present invention.

For example, one, several or all of the above-described localization and / or classification processes can be carried out algorithmically, i.e. without the use of machine learning, or manually by an operator. For example, the segmentation of defect regions can be done algorithmically, e.g. by detecting the edges.

In a preferred embodiment of the present invention, however, the localization and / or classification is carried out using at least one ML model. In particular, at least one of the processes described above for localization and / or classification can take place using an ML model. In a possible embodiment, several or all of the provided, above-described processes for localization and / or classification can also take place by means of at least one ML model.

In a possible embodiment of the present invention, the localization, classification and / or separation between the tire area and the background is solved simultaneously by a single ML model.

In particular, the separation between the tire area and the background on the one hand and the localization and / or classification on the other hand can be solved simultaneously by a single ML model.
In one possible embodiment of the present invention, the evaluation device makes a decision on the basis of the localization of defect areas and / or the classification of defects in a tire as to whether the tire is scrap or not. This can be a yes / no statement, for example, or a reject probability can be specified. The statement can be used directly for control a sorting station of the production plant and for sorting out scrap tires, or checked again by a human tester.

In a possible embodiment of the present invention, the defect areas and / or defects are assessed with regard to their relevance to rejects by means of one or more tolerance rules.

In particular, within the scope of the present invention, the localization of defect areas and / or the classification of defects in a tire takes place by means of machine learning, whereas the decision based on these results as to whether the tire is scrap or not is rule-based. This makes the decision-making process traceable.

A tolerance rule can, for example, specify which defect areas and / or defects are tolerated and which defect areas and / or defects lead to the decision that the tire is scrap. For example, defect areas and / or defects can be compared with threshold values with regard to one or more of their properties such as, for example, size, radius, length and / or contrast.

In one possible embodiment of the present invention, different tolerance rules are used for different test areas of the tire. The tolerance rule which is applied to a defect area and / or a defect therefore depends in particular on the area of the tire in which the defect area and / or defect is located. For this purpose, the tire can be divided into test regions to which different tolerance rules are assigned.

In a possible embodiment of the present invention, different error tolerances are used depending on the test region. For example, the threshold values, with which one or more properties of the defect areas and / or defects are compared, can be different depending on the test region in which the defect area and / or defect is located.

For example, the sidewalls and the tread of the tire can form different test regions. Furthermore, the sidewalls of the tire can also be divided into different test regions.

In a possible embodiment of the present invention, the device comprises an operator interface, by means of which the tolerance rules can be specified and / or changed.

In a possible embodiment of the present invention, the device comprises an operator interface, by means of which the test regions can be specified and / or changed.

In one possible embodiment of the present invention, the device comprises an interface via which a user can process recordings recorded and / or simulated by the tire tester by annotation, in particular by manual separation, localization and / or classification, the device further comprising a training module, which allows the ML model to be trained using the recordings prepared by the user.

The interface thus allows the creation of training data for training the machine learning process.

In a possible embodiment of the present invention, the device comprises a pre-trained ML model, the interface and the training module allowing further training during installation and / or during operation of the device. This reduces the effort for training the ML model, since it has already been pre-trained. On the other hand, the device can nevertheless be adapted to the specific evaluation task, for example the tire types to be tested and / or the measurement situation, during installation and / or during operation. The device is preferably designed in such a way that the re-learning process can also be carried out several times, for example when the recording environment changes, new products are to be checked, new defect classes are to be recognized and / or other new information has to be provided to the ML model.

In a possible embodiment of the present invention, the device comprises an evaluation station for the manual evaluation of recordings, the evaluation station being supported by the ML model. In this embodiment, the final evaluation therefore takes place at least not fully automatically, but at least partially by an operator. However, this is supported by automated sub-processes.

In a first variant, the evaluation station can be designed in such a way that all tires are also checked by an operator. For example, the evaluation function can offer an evaluation, for example a classification, which is either confirmed or changed by the operator.

In a second variant, the evaluation station can be designed so that only some of the tires are also checked by an operator while another part of the tires is only checked by the evaluation function. In particular, only those tires for which the evaluation function outputs a high uncertainty factor can be presented to an operator for testing.

In a possible embodiment of the present invention, the evaluation station displays recordings differently depending on a localization and / or classification carried out with the aid of the ML model. For example, recordings which show defects are prioritized and / or highlighted in the display. Furthermore, an uncertainty factor of the ML model can be taken into account. For example, recordings whose evaluation the evaluation function outputs a high uncertainty factor can be prioritized and / or identified and / or highlighted in the display.

In a possible embodiment of the present invention, the device displays tires differently depending on a localization and / or classification carried out with the aid of the ML model. For example, tires in which no defects were detected and / or a low uncertainty factor with regard to the automatic classification is present, can no longer be displayed at all or is only displayed in a subordinate manner to tires in which defects are recognized and / or a high uncertainty factor is present with regard to the automatic classification.

In a further possible embodiment, the tire test is carried out at least in one operating mode of the device for all tires in an exclusively automated manner by the evaluation function and without an additional human test.

In a possible embodiment of the present invention, the tire testing device carries out an optical measuring method. In particular, an interferometric measuring method can be used.

In particular, the present invention can be used for evaluating recordings from a tire testing device which undertakes a shearographic measurement to generate phase difference images. The recordings evaluated according to the invention can therefore in particular be phase difference images.

In one possible embodiment of the present invention, the tire testing device subjects the tires to a pressure load. The recordings can reproduce a deformation of the tire due to the pressure load. In particular, the tires are exposed to a negative pressure.

In one possible embodiment of the present invention, the tire testing device comprises a pressure chamber, in particular a vacuum chamber. One or more measuring heads for shearographic measurement of the tread and / or the side surfaces of the tire are preferably arranged in the pressure chamber.

In one possible embodiment of the present invention, the tire testing device checks the tread and / or the side surfaces of the tire.

In one possible embodiment of the present invention, the tire testing device is arranged in a transport path for the tires. At least one tire turner can be provided so that both side surfaces of the tire can be checked from the outside.

• - Aufnehmen von Aufnahmen durch das Reifenprüfgerät; und
• - Auswerten der Aufnahmen mittels eines ML-Modells.

The present invention also comprises a method for evaluating recordings from a tire testing device with the following steps:

• - Recording of recordings by the tire testing device; and
• - Evaluation of the recordings using an ML model.

This results in the same advantages that have already been described in more detail above with regard to the device according to the invention.

• - Aufbereitung von Aufnahmen durch Annotation, insbesondere durch Trennung, Lokalisierung und/oder Klassifikation von Aufnahmen zur Erzeugung von ersten Trainingsdaten;
• - Trainieren des ML-Modells mittels der ersten Trainingsdaten.

In one possible embodiment of the present invention, the following steps are carried out beforehand:

• - Preparation of recordings by annotation, in particular by separating, localizing and / or classifying recordings for generating first training data;
• - Training the ML model using the first training data.

In possible alternative configurations, however, the preparation of the recordings can also be omitted. For example, in this case the ML model can only learn meaningful descriptions of the recordings, for example by means of unsupervised or self-supervised learning methods.

• - Aufnehmen von Aufnahmen durch das Reifenprüfgerät
• - Aufbereitung der Aufnahmen durch Trennung, Lokalisierung und/oder Klassifikation von Aufnahmen zur Erzeugung von zweiten Trainingsdaten;
• - Nachtrainieren eines ML-Modells mittels der zweiten Trainingsdaten.

In one possible embodiment of the present invention, the ML model is delivered pre-trained and after the device has been installed, the following additional steps are carried out:

• - Recording of recordings by the tire testing device
• - Preparation of the recordings by separating, localizing and / or classifying Recordings for generating second training data;
• - Retraining of an ML model using the second training data.

In one possible embodiment of the present invention, such retraining takes place several times, for example when the recording environment changes, new products are to be checked, new defect classes are to be recognized, or other new information has to be provided to the ML model

In a possible embodiment of the present invention, recordings of tires without defects are used to generate training data, and a separation between a tire area and a background is made in these recordings. Since defects in tires occur relatively rarely, the creation of a sufficient amount of training data with defects is time-consuming. This effort can be reduced by making use of recordings which show tires without defects, at least for learning about the separation between the tire area and a background.

In a further possible embodiment of the present invention, training data are generated by simulation.

The method according to the invention is preferably carried out as it has already been described in more detail above with regard to the device. In particular, the method can be carried out using the device described above.

The present invention further comprises software for implementing a device as described above. In particular, the software can include an ML model.

A method as described above can preferably be carried out by means of the software, in particular when the software runs on the device according to the invention.

The software can be stored on a data carrier or memory.

The device according to the invention preferably comprises at least one microprocessor and a memory on which the software according to the invention is stored, the software comprising commands by which the microprocessor controls the device for implementing the functions according to the invention.

The tires tested according to the invention are in particular rubber tires.

The present invention will now be described in more detail with reference to exemplary embodiments and drawings.

• 1 ein erstes Ausführungsbeispiel einer erfindungsgemäßen Vorrichtung,
• 2 eine Messung einer Seitenfläche eines Reifens und eine entsprechende Aufnahme in einem Ausführungsbeispiel der vorliegenden Erfindung,
• 3 eine Messung einer Lauffläche eines Reifens und eine entsprechende Aufnahme in einem Ausführungsbeispiel der vorliegenden Erfindung,
• 4 Beispiele für unterschiedliche Klassen von Defekten,
• 5 eine Prinzipdarstellung eines Ausführungsbeispiels eines erfindungsgemäßen Verfahrens,
• 6 Aufnahmen der Lauffläche und einer Seitenfläche eines Reifens mit einer durch ein Ausführungsbeispiel einer erfindungsgemäßen Vorrichtung vorgenommenen Trennung zwischen Reifenbereich und Hintergrund und einer Lokalisierung von Defekten,
• 7 ein zweites Ausführungsbeispiel einer erfindungsgemäßen Vorrichtung mit einem externen Interface,
• 8 ein erstes Ausführungsbeispiel für ein ML-Modell und
• 9 ein zweites Ausführungsbeispiel für ein ML-Modell.

Show:

• 1 a first embodiment of a device according to the invention,
• 2 a measurement of a side surface of a tire and a corresponding recording in an embodiment of the present invention,
• 3 a measurement of a tread of a tire and a corresponding recording in an embodiment of the present invention,
• 4th Examples of different classes of defects,
• 5 a schematic representation of an embodiment of a method according to the invention,
• 6 Recordings of the tread and a side surface of a tire with a separation made by an embodiment of a device according to the invention between the tire area and the background and a localization of defects,
• 7th a second embodiment of a device according to the invention with an external interface,
• 8th a first exemplary embodiment for an ML model and
• 9 a second embodiment for an ML model.

1 shows an embodiment of a device according to the invention with a tire testing device 1 .

In the exemplary embodiment, the tire testing device is arranged in a transport route from which transport sections 4th and 5 , which in front of and behind the tire tester 1 are arranged in 1 are shown. The tires 6 therefore pass through the tire testing device in the exemplary embodiment on the transport route. There is also a turning device 2 provided to turn the tire between two test runs in the tire tester.

In the exemplary embodiment, the tires are delivered lying on the transport route, if necessary turned by 180 ° by the turning device, and tested in the tire testing device in at least one test run. The testing of the tires in one test run is also preferably carried out horizontally. The tire testing device can have an internal transport route on which the tires on the one hand can be moved through the tire testing device, and on which the tires lie during the testing process.

However, other arrangements for loading the tire testing device with tires are also conceivable within the scope of the present invention. For example, the tire testing device can be loaded manually.

In one possible embodiment of the possible invention, the tire testing device is involved 1 around a test device in which the tire is subjected to a pressure load and deformations resulting from this are measured. In particular, the tire can be exposed to a negative pressure load.

The check is carried out in particular holographically and / or shearographically. One possible test run can consist in first recording a first phase image when the tire is subjected to a first pressure load, then changing the pressure load and recording a second phase image when the tire is subjected to a second pressure load. A phase difference image is created from the two phase images, the evaluation of which can be used to identify flaws and / or defects in the tire, since these deform under the changed pressure load. This procedure can be repeated several times as part of a test process in order to test different circumferential areas of the tire.

The tire testing device 1 comprises in the exemplary embodiment a pressure chamber 10 , especially a vacuum chamber. This preferably has at least one opening 11 through which the tires can be moved into and out of the pressure chamber, and which can be hermetically sealed to change the pressure in the pressure chamber. In the exemplary embodiment, the tires are placed in the pressure chamber on a first side 10 transported in, checked, and after the test carried out on the opposite side again out of the tire tester.

The tire testing device comprises one or more measuring heads 7th , which preferably via a traversing arrangement 8th are movable and / or rotatable. The measuring heads are in particular holographic and / or shearographic measuring heads. By rotating the measuring head or heads, different circumferential areas of a lying tire can be recorded. Alternatively or in addition, it would also be conceivable to rotate the tire in order to check different circumferential areas.

In alternative configurations, however, other optical test methods are also conceivable.

The tire testing system is preferably designed for a so-called bead-to-bead tire test, in which both side surfaces of the tire are tested at least from the outside and the tread of the tire is tested at least from the inside. Since one of the two side surfaces usually rests on a surface during a test run so that it cannot be tested from the outside, the entire tire test can be carried out in two separate test runs, between which the tire is turned through 180 °.

Alternatively, only the side surfaces, one of the side surfaces or the running surface, as well as any combination thereof, can be tested.

2 shows an examination of a side surface 12 of the tire 6 through a measuring head 7th with a measuring field 19th . The recording made by the measuring head 20th includes an outside background 13 , the upper side wall 12 from the outside, the upper bead 14th , the lower side wall 15 ' from the inside, the lower bead 14 ' and the inner background 13 ' .

3 shows an inner tread test 16 of the tire 6 through a measuring head 7 ' with a measuring field 19 ' . The through the measuring head 7 ' recorded recording 20 ' includes the upper bead 14th , the upper side wall 15th from the inside, the tread 16 from the inside and the lower side wall 15 ' from within.

4th shows the appearance of typical defects 21st to 23 which are to be recognized by a tire test on the recording made by the tire testing device, in particular a phase difference image.

The evaluation according to the invention of the recordings of the tire generated by the tire testing device serves to localize and / or quantify defects on the record of the tire.

1. 1. Data Recording mittels Shearographie
2. 2. Manuelle Inspektion

oder

1. 1. Data Recording mittels Shearographie
2. 2. Extraktion handkodierter Merkmale (z.B. Histogramm über Grauwerte, Kantenorientierungen, ...)
3. 3. Einfache Analyseverfahren (z.B. Schwellwert) auf der Merkmalsrepräsentation zur Unterscheidung Hintergrund vs. Reifen und Analyse der Defektregionen im Reifenbereich

The previous workflow used for this according to the state of the art was the following:

1. 1. Data recording using shearography
2. 2. Manual inspection

or

1. 1. Data recording using shearography
2. 2. Extraction of hand-coded features (e.g. histogram over gray values, edge orientations, ...)
3. 3. Simple analysis methods (eg threshold value) on the feature representation to distinguish background vs. tire and analysis of the defect regions in the tire area

• ■ Manuelle Prüfung ist nötig, welche zeitintensiv ist und in der Genauigkeit variieren kann und dadurch subjektiv ist.
• ■ Automatisierte Prüfungen auf Basis einfacher Schwellwert-Operationen sind in der Applikation für den vollautomatischen Einsatz zu ungenau.
• ■ Automatisierte Prüfungen auf Basis einfacher Image Processing Verfahren (z.B. handkodierte Merkmale) sind zeitaufwendig in der Implementierung, können nicht ohne signifikanten Aufwand auf neue Gerätetypen übertragen werden (im Gegensatz zu lernbasierten Verfahren) und sind häufig zu ungenau für die Applikation.
• ■ Die Position des Reifens im Analysetool kann variieren, was in einfachen Analyseverfahren nur mit erheblichem Aufwand zu berücksichtigen ist (zum Beispiel durch aufwendiges Kalibrieren und Vermessen der Anlagenparameter).
• ■ Reifen sind von oben und von der Seite zu sehen und zeigen variierender Hintergrund, eventuell mit Zusatzobjekten (Metallarmen etc.), was für einfache Analyseverfahren zu komplex ist

However, this had the following disadvantages:

• ■ Manual testing is necessary, which is time-consuming and can vary in accuracy and is therefore subjective.
• ■ Automated tests based on simple threshold value operations are too imprecise in the application for fully automatic use.
• ■ Automated tests based on simple image processing methods (eg hand-coded features) are time-consuming to implement, cannot be transferred to new device types without significant effort (in contrast to learning-based methods) and are often too imprecise for the application.
• ■ The position of the tire in the analysis tool can vary, which can only be taken into account with considerable effort in simple analysis processes (for example, through complex calibration and measurement of the system parameters).
• ■ Tires can be seen from above and from the side and show a varying background, possibly with additional objects (metal arms, etc.), which is too complex for simple analysis methods

In order to eliminate these problems, the device according to the invention therefore has an evaluation device for evaluating the recordings generated by the test device, in particular for evaluating the phase difference images of the shearographic test device, an ML model being used for evaluation.

In particular, the ML model can be used to predict the tire region to be checked, i.e. the separation between background and tire area, and / or the (subsequent) localization and / or classification of the defect regions. In particular, for this purpose the ML model can include a learned segmentation model and / or a learned classifier. The classification can include a quantification of the defect regions, for example the determination of the area and / or extent of a defect region.

• ◯ Robuste und präzise Vorhersage von Prüfregion und Defekten (insbesondere durch gemeinsames Lernen der Bildrepräsentation und des Prediktors/Klassifikators) .
• ◯ Berücksichtigung des globalen Kontexts kann ohne zusätzlichen Aufwand mitgelernt werden.
• ◯ Anpassung auf neue Reifentypen, Aufnahmeänderungen, etc. kann mit wenig Aufwand durch neues Anlernen erfolgen (z.B. durch Servicepersonal oder Kunde) und benötigt keine algorithmischen Anpassungen mit großem Implementierungsaufwand.

The advantages of the ML compared to known methods such as a simple image processing solution or a simple computer vision solution can be summarized as follows:

• ◯ Robust and precise prediction of test region and defects (especially through joint learning of the image representation and the predictor / classifier).
• ◯ Taking the global context into account can be learned without additional effort.
• ◯ Adaptation to new tire types, changes to mountings, etc. can be done with little effort through new training (e.g. by service staff or customers) and does not require any algorithmic adaptations with a large implementation effort.

5 shows an embodiment of a method according to the invention in which an ML model is used to evaluate the recordings.

A training phase 30th includes a creation 31 of recordings, an annotation 32 of errors in the recordings for generating training data and a training 33 an ML model or machine learning method.

To create the recordings, these are used in step 31 preferably recorded, especially by means of shearography. Alternatively, the recordings can also be created using a simulation. The annotation is done manually and can, if necessary, be supported by classic computer vision.

By training the ML model, its model parameters are determined.

In normal operation 40 the device is used for data recording 41 again by an optical measuring method, in particular shearography. The evaluation 42 the recordings are made with the help of the in the training phase 40 learned ML model. In particular, the output parameters are automatically determined using the learned model parameters.

In a possible embodiment of the present invention it is provided that the ML model not only receives a single input image, which is evaluated, but an image stack, e.g. an image stack, which shows the same area of the tire in different resolutions.

The evaluation device can, on the one hand, separate the background from the image areas to be checked, ie the tire area (step 1 ), as well as a defect region analysis and / or classification (step 2 ). In at least one of these steps, an ML method is used, in particular in the form of a neural network or deep learning. As part of step 1 the tire area can also be further divided into the different areas of a tire discussed in more detail above, which can be seen on the recording.

• - L1: Klassisch algorithmische Lösung (zum Beispiel via Grauwertstatistik);
• - L2: ML Lernbasierte Lösung (zum Beispiel mittels Deep Learning Segmentierung, um pixelweise Vordergrund vom Hintergrund und/oder unterschiedliche Reifenbereiche zu trennen);
• - L3: Kombination von L1 oder L2 mit Fit einer Primitiven (z.B. Spline), um bekanntes Wissen zur Kreisform der Prüffläche einzubringen;
• - L4: Direkte Schätzung von Parametern einer Primitiven (z.B. Splineparameter) aus der Aufnahme mittels eines Deep Learning Verfahrens (z.B. zur Auswertung von Seitenaufnahmen von Reifen, in denen der Vordergrund stets durch einen Kreisabschnitt approximiert werden kann).

For example, there are the following possible solutions for step 1 (especially the background separation):

• - L1: Classically algorithmic solution (for example via gray value statistics);
• - L2: ML learning-based solution (for example using deep learning segmentation to separate pixel-by-pixel foreground from background and / or different tire areas);
• - L3: Combination of L1 or L2 with the fit of a primitive (eg spline) in order to incorporate known knowledge about the circular shape of the test surface;
• - L4: Direct estimation of parameters of a primitive (e.g. spline parameters) from the recording using a deep learning process (e.g. for evaluating side recordings of tires, in which the foreground can always be approximated by a segment of a circle).

• - K1: Klassisch algorithmische Segmentierung von Defektregionen (z.B. via Kantenmerkmale o.Ä.);
• - K2: ML Lernbasierte pixelgenaue Segmentierung und ggf. Klassifizierung von Fehlerstellen (z.B. mittels Deep Learning zur Bildsegmentierung);
• - K3: ML Lernbasierte Detektion von Fehlerstellen und Beschreibung mittels rechteckförmiger Bounding Boxes oder anderer parametrisch beschreibbar, geometrischer Primitiva wie etwa Kreis oder Ellipsen (z.B. mittels Deep Learning Verfahren zur Objekterkennung).

Furthermore, there are, for example, the following possible solutions for step 2 (Defect region analysis and / or classification):

• - K1: Classically algorithmic segmentation of defect regions (eg via edge features or the like);
• - K2: ML learning-based, pixel-precise segmentation and, if necessary, classification of faults (eg using deep learning for image segmentation);
• - K3: ML learning-based detection of faults and description by means of rectangular bounding boxes or other parametrically describable, geometric primitives such as circles or ellipses (e.g. by means of deep learning methods for object recognition).

Options L2 and K2 can also be solved with a single ML model. The input here is the recording, in particular a sheraography image. The output is in the form of a recording on which a pixel-by-pixel differentiation has been made between the background, tires and one or more types of defects.

The options (L4 and K2) or (L4 and K3) can also be solved simultaneously with a common ML model (so-called multi-task learning, common prediction of a foreground-background distinction and a defect region segmentation).

6a and 6b show exemplary outputs of a device according to the invention for receiving a running surface ( 6a) and a side face ( 6b) .

There is a separation between the tire area 16 , 12 and background 13 , 13 ' by a primitive 50 , as well as a localization and classification of defects 21st , 22nd and 23 . The localization is done here by bounding boxes. Alternatively, the localization could also be done pixel by pixel.

According to the invention, in at least one of the steps 1 and or 2 an ML model is used, ie at least L2, L3 or L4 or K2 or K3.

The ML model can be used and designed independently of or in combination with the above-mentioned procedures as follows:

Solution 1 (only replacing the previous analysis)

description

• • Input: recordings, especially from the shearography process
• • Output: Defect map (binary or multi-class, scalar or class probabilities) and / or Region of Interest (ROI) - mask of the tire surfaces
• • The ML model learns the prediction of the output from the input

Solution 2 (direct prediction of the relevant metrics)

description

• • Input: recordings, especially from the shearography process
• • Output: Estimation of the defect size and defect position (e.g. in mm), ROI mask of the tire surfaces
• • The ML model learns the prediction of the output from the input.

Solution 3 (only learning the feature representations in the existing workflow)

description

• • Input: recordings, especially from the shearography process
• • Output: Defect map (binary or multi-class, scalar or class probabilities) and / or Region of Interest (ROI) - mask of the tire surfaces
• • An involved ML model only learns meaningful descriptions of the shearography data, eg using Unsupervised Learning (AutoEncoders, RBM), the rest of the workflow according to the state of the art remains unchanged, ie the descriptions learned by the ML model are replaced by hand-coded features used in a computer vision process. In this case, annotation can be omitted, which reduces the initial annotation effort.

There are different variants for generating training data:

Variant 1 (learning from measured data)

Associated pairs from {recording, annotation} are required as training data. Recordings obtained through measurements are therefore provided with a corresponding annotation. Depending on the solution (1-3), the annotation can be a pixel-by-pixel defect map, defect probability map, binary and multi-class version thereof, or just a scalar or vector of the target variable.

Variant 2 (learning from simulated data)

Same as variant 1, except that the recordings are obtained from a simulation method which appropriately simulates or approximates the shearographic image.

Variant 3 (learning from measured and simulated data)

Learning from measured and simulated data can be carried out cyclically, can be mixed, can e.g. run as a two-step process (first V1 then V2).

Regardless of the way in which the training data are generated, the ML model can be taught in a possible configuration with data from different "modalities", e.g. with data from several different shearography systems or different lighting settings.

• - Verschiedenen Zeitpunkte
• - Verschiedene physische Messmaschinen
• - Verschiedene Defekttypen
• - Verschiedene Defektausprägungen
• - Von verschiedenen Annotierern bewertet
• - Verschiedene Reifentypen

Dies gilt sowohl für tatsächlich aufgenommene als auch für simulierte Daten.The training data should be as representative as possible, ie they should have at least one and preferably several of the following properties:

• - Different times
• - Various physical measuring machines
• - Different types of defects
• - Different types of defects
• - Rated by various annotators
• - Different types of tires

This applies to both actually recorded and simulated data.

The annotations should be consistent, i.e. the annotations should be as consistent as possible or allow an assessment of the annotation inaccuracy and / or the acquisition process should be stationary.

The device can be provided with further configuration features in possible configurations. The options described below can be combined with all of the options and variants described above.

Design feature "continuous training + interface"

It is practically impossible to have representative data for all subsequent use cases at the time of development. In particular, the systems can be installed differently from customer to customer, different environments with background structures exist, different settings can be set for data acquisition, etc.

In a possible embodiment of the present invention, the device is therefore designed in such a way that the customer can relearn the ML model learned in advance. This can happen during an installation phase, for example, or later at regular or irregular intervals.

In particular, the re-learning process can also be carried out several times, for example if the recording environment changes, new products have to be checked, new defect classes have to be recognized, or other new information has to be provided to the ML model.

A corresponding embodiment is shown in 7th shown, the device initially the above with regard to 1 Has described components.
To enable re-learning, an interface can be provided in order to create annotations for newly recorded data, for example as an annotation GUI on an operator PC 60 , which is arranged on the tire testing device, or via an external computer 61 which communicates with the device over a network 62 is related, see 7th . The interface on the external computer 61 can for example be implemented via a browser solution.

In addition, a computing unit 63 (with GPU) are available on which the learning can take place, e.g. in the cloud or on a local PC.

Algorithmic or technical methods from the area of transfer learning can be used for retraining. Especially for deep neural networks, re-learning can take place via fine-tuning, for example, in which the previous parameter values of the originally learned model are used as a starting solution for re-training at the customer's. Fine-tuning enables the entire learning process to be carried out faster and with less data than was necessary when it was first learned during the development phase.

Design feature "prioritization of learning data"

During the above-described re-learning or continuous learning during the installation or operation and / or running-in of the device by adapting the ML methods, for reasons of time and costs, not any number of data can be annotated.

Therefore, in a possible embodiment of the present invention, it can be provided that the newly recorded data is automatically sorted by relevance and made available in a correspondingly sorted manner by the device. This makes it possible to annotate only the most relevant data. The uncertainty of the initially learned system can be a measure of relevance.

Hardware and / or software for annotating and training can also be provided here, either locally or via cloud-like connections

Design feature "ML method without pixel-by-pixel annotations"

The pixel-perfect annotation of images is often too expensive to be carried out in large quantities.

According to a possible embodiment of the present invention, the device is therefore designed in such a way that training data can be used in which the annotation is only implemented on the image label level. For example, the annotation can be made to determine whether or not an image contains a defect, or one or more defect classes are assigned to an image. A correspondingly designed interface is preferably provided for this.

With images annotated in this way, an image classifier is learned within the scope of an exemplary embodiment, which predicts this yes-no answer and / or classification for new images, and possibly then visualizes in the image which regions are decision-relevant for the ML -Classifier (e.g. Gradcam, GuidedBackprob). Thus, in addition to the classification result, a segmentation-like map is also generated without first having to use segmentation annotations.

Design feature "hybrid operation mode visual inspection + machine"

It is to be expected that ML methods will initially not be adequately accepted, especially since 100% accuracy cannot initially be achieved, but extremely low error rates are required in the application.

In one possible embodiment of the present invention, a hybrid solution comprising a machine and a visual inspector is therefore implemented. For this purpose, the device provides an interface through which a visual inspector can check and, if necessary, correct the prediction generated by the device using an ML model. In particular, an automated pixel-by-pixel ML analysis can be coupled with a manual verification.

In addition to the prediction generated on the basis of an ML model, the device preferably also outputs a value and / or a display for the confidence that the ML model had in the prediction.

For example, the visual inspector can accept the ML prediction for recordings without further examination if the model has a high confidence. The visual inspector only has to verify / correct predictions for examples for which the ML system has a high internal uncertainty.

Design feature "fully automated final inspection"

The previous workflow with manual visual inspection can be too time-consuming for high-frequency inspection processes, e.g. B. for 24/7 test sequences, where visual inspectors are not always available in 24/7 shifts. The overall workflow “hybrid operation mode visual inspection + machine” described in the previous feature can also be too time-consuming, even if only a few (unsafe) recordings have to be verified manually.

According to a possible embodiment of the present invention, the test is therefore carried out exclusively by an ML method which directly predicts the final test decision (e.g. as a classification model or with additional prediction of the segmentations as described above).

Design feature "rule-based committee decision"

As part of a possible embodiment, the evaluation device first uses an ML model to determine a segmentation map and / or a classification of the defects, for example a prediction of the location, size, type and / or number of defects.

Based on certain tolerance rules, this result is then transferred into the final statement as to whether the tire is scrap (in particular in the form of a yes / no decision).

For this purpose, the evaluation device assesses the determined defect regions and / or defects by means of rule-based tolerance (e.g. with regard to radius, length, contrast, ...) with regard to their relevance to rejects.

The assessment according to the defect relevance for OK or not OK can be dependent on test regions in one possible embodiment. In particular, error tolerances can be set depending on the test regions.

For example, different requirements for a bubble size can be specified on the tire side than on the tire running side.

Some ML models that can be used within the scope of the present invention are described below.

The easiest way to learn an ML model for transforming data is through statistical models (almost any parametric distribution can be imagined). In practice, however, this will often not lead to sufficient accuracy. For this reason, deep learning methods are preferably used in the context of the present invention.

In one possible exemplary embodiment of the present invention, an encoder-decoder architecture is used as the ML model. In particular, such an encoder-decoder architecture can be used for pixel-precise separation into the background and tire area and / or for pixel-precise segmentation of defects. For example, the recording can be transformed into a class probability map.

In one possible refinement, the encoder-decoder architecture can be expanded to include skip connections, which make it possible to accurately describe especially the smallest subtleties in image data. Well-known ML models are Resnet, DenseNet or RefineNet.

The encoder-decoder architecture can be designed as a U-Net, in which the encoder side reduces the sample size in several steps, while the decoder side increases the sample size again.

8th shows an embodiment of such a model for transforming recordings 70 into pixel-perfect segmentations 71 . The exemplary embodiment initially has an encoder path 73 from several levels 74 , 74 ' , 74 " , 74 ''' of layers 80 which reduce the sample size. The decoder path 75 also has corresponding levels 74 , 74 ' , 74 " , 74 ''' of layers 80 which increase the sample size again.

In the exemplary embodiment, this classic encoder-decoder architecture is generalized in that results from early transformation layers are also available to later layers through the skip connections (represented by horizontal arrows 72 ). In the exemplary embodiment, the skip connections connect layers of the encoder path 73 and the decoder path 75 the same level. As a result, the spatial information in particular is retained, which leads to greater accuracy. In addition, the intermediate result of each path can be interpreted as a feature, which generalizes the mapping to a central feature collection.

Instead of a U-Net, processing in blocks with skip connections can be provided.

9 shows an exemplary representation of a residual block, which transforms inputs with several layers and additionally offsets the output with the original input in order to obtain the final result of the block.

Several such residual blocks can be connected in series. The exclusive flow of data from one layer to the next is thus generalized

Dense blocks can also be used. With a dense block, inputs are transformed with several layers. Each layer receives the (calculated) results of all previous layers and forwards its output to the subsequent layer and to the final output

Processing in blocks can be combined with a U-Net.

Furthermore, DenseNet models can consist of several Dense blocks. It thus generalizes ideas from ResNet and U-Net by adding the Data flow and its processing is generalized, which has a positive effect on both the accuracy and learnability of the models.

Furthermore, a RefineNet block can be used, which simultaneously receives inputs of a modality at several resolution levels, processes them individually with Resnet blocks, fuses their results, and finally transforms this intermediate result in the manner of a set. A RefineNet model can also be used, which combines several RefineNet blocks in order to implement highly precise, learnable transformations.

In particular, encoder-decoder architectures, with or without skip connections, and in particular those encoder-decoder architectures as just described, can be used for the solution 1 described above.

In the context of solution 2 described above, the prediction of the target variable can be interpreted as a regression problem in one possible exemplary embodiment. Since several defects can occur in an image, it can also be provided that the number of defects is also predicted, i.e. a list of defects with the corresponding size and prediction probability for each image. Possible detection methods such as YOLO, SSD, YOL0-9000 or Mask R-CNN can be adapted accordingly by predicting the defect size in mm and the class affiliation instead of the predicted object size in pixels and the class affiliation.

In the context of solution 3 described above, AutoEncoder or Restricted Boltzmann Machines (RBM) can be used, which allow the unsupervised learning of image representations. Alternatively, monitored processes can also be learned as described for solution 1, in order to then only use their activations as a representation of image data and to integrate them into the previous workflow (e.g. replace the handcrafted features).

In summary, the present invention thus provides a device and a method by means of which recordings of tires, in particular phase difference images from shearography systems for tires, are automatically evaluated, defects are recognized, recorded in position and size and / or divided into defect classes, and in particular also of sensor artifacts can be distinguished.

Phase difference images from shearography systems for tires are available in different views on the tire: Sectors of the tread and / or the sidewall and at least one bead can be shown. A whole tire can also be seen. The image areas are divided into the individual areas of tread, side wall (near and far), bead area (near and far), background, which can also be partially or completely missing.

Existing methods recognize the different image areas and defects on the basis of specified criteria and threshold values for local or global differences in image contrast or pattern recognition parameters. The specification of the threshold values is flawed and complex.

In contrast, the invention uses a self-learning method that works without manual specification of threshold values and can therefore be used in a wide range of fields of application.

It is thus possible to automatically detect and classify defects in the manufacture and retreading of any pneumatic tire. The detection can support or even replace an operator of the system in evaluating the tested tire. The learning of the fault assessment is done by testing tires with known fault patterns.

Time-consuming and costly determination of parameters is no longer necessary, the error detection is continuously optimized and objectified and automatically adapted to the needs of the respective customer.

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• EP 2851670 A2 [0002]

Claims (15)

Device for tire testing with a tire testing device, by which at least one recording of at least one area of a tire is generated, and with an evaluation device for evaluating the at least one recording, characterized in that at least one evaluation function of the evaluation device by means of machine learning (ML), in particular by means of deep learning (DL), works. Device according to Claim 1 , the evaluation device separating a tire area and a background. Device according to Claim 2 , the separation between the tire area and the background being carried out by at least one ML model, with a parameterization of the dividing line between the tire area and the background also preferably being used. Device according to one of the preceding claims, wherein the evaluation device localizes defect areas and / or classifies defects and / or the tire. Device according to Claim 4 , the localization and / or classification being carried out by at least one ML model, localization, classification and / or separation between the tire area and the background preferably being solved simultaneously by a single ML model. Device according to Claim 4 or 5 , the evaluation device making a decision based on the localization of defect areas and / or the classification of defects in a tire as to whether the tire is scrap or not, for which purpose the defect areas and / or defects are preferably based on one or more tolerance rules are assessed with regard to their relevance to rejects, wherein different tolerance rules are preferably used for different test areas of the tire and in particular different error tolerances are used depending on the test region. Device according to one of the preceding claims, with an interface via which a user can process recordings recorded and / or simulated by the tire tester by annotation, in particular by manual separation, localization and / or classification, the device further comprising a training module which has a Training of the ML model through the recordings prepared by the user allowed. Device according to Claim 7 With a pre-trained ML model, the interface and the training module allowing further training during installation and / or during operation of the device, the interface and the training module preferably allowing the post-learning process to be carried out multiple times. Device according to one of the preceding claims, with an evaluation station for the manual evaluation of recordings, the evaluation station being supported by the ML model, wherein in particular all tires or some of the tires are also tested by an operator in addition to the test by the evaluation function, wherein the evaluation station preferably displays recordings and / or tires differently depending on a localization and / or classification carried out with the aid of the ML model, and in particular takes into account an uncertainty factor of the ML model. Device according to one of the preceding claims, the tire test being carried out at least in one operating mode of the device for all tires in an exclusively automated manner by the evaluation function and without an additional human test. Device according to one of the preceding claims, wherein the tire testing device carries out an optical measurement method, and in particular carries out a shearographic measurement to generate phase difference images, and / or wherein the tire testing device subjects the tires to a pressure load and in particular has a pressure chamber. Method for evaluating recordings from a tire testing device, in particular using a device according to one of the preceding claims, with the steps: - Recording of recordings by the tire testing device; and - Evaluation of the recordings using an ML model, the following steps preferably being carried out beforehand: - Preparation of recordings by annotation, in particular by separating, localizing and / or classifying recordings for generating first training data; - Training the ML model using the first training data. Procedure according to Claim 12 , whereby the ML model is delivered pre-trained and the following steps are carried out after the device has been installed: - Recordings by the tire testing device - Processing of recordings by separating, localizing and / or classifying recordings to generate second training data; - Retraining of an ML model using the second training data. Procedure according to Claim 12 or 13 , whereby recordings of tires without defects are used in the generation of training data and a separation between a tire area and a background is made in these recordings. Software for implementing a device according to one of the Claims 1 to 11 , by means of which a method according to one of the Claims 12 to 14th is feasible if this runs on the device.

Images