EP4238070A1 - Softwareverfahren zur opto-sensorischen erkennung, vermessung und bewertung von werkzeugzuständen - Google Patents
Softwareverfahren zur opto-sensorischen erkennung, vermessung und bewertung von werkzeugzuständenInfo
- Publication number
- EP4238070A1 EP4238070A1 EP21805449.2A EP21805449A EP4238070A1 EP 4238070 A1 EP4238070 A1 EP 4238070A1 EP 21805449 A EP21805449 A EP 21805449A EP 4238070 A1 EP4238070 A1 EP 4238070A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- target object
- computer
- implemented method
- image
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/478—Contour-based spectral representations or scale-space representations, e.g. by Fourier analysis, wavelet analysis or curvature scale-space [CSS]
Definitions
- the invention relates to a computer-implemented method for evaluating sensory information from a tool, in particular a tool suitable for soil cultivation.
- US Pat. No. 6,479,960 B2 discloses a machine tool in which it can be automatically determined whether the tool is faulty or not, even if the cutting load and/or the strength of a tool is low. Furthermore, the miniaturization of such a machine tool is described with the purpose of achieving cost savings.
- the machine tool in the primary aspect includes a photographing unit that photographs an image of the tool and a determination unit that determines whether or not the tool is defective based on the images obtained by the photographing unit. A faulty tool is detected without contact using an image comparison.
- DE 10 2008 045 470 A1 describes a method for determining the state of wear of a milling tool, in particular a chisel, a chisel holder and/or a chisel holder changing system. For this purpose, the position of at least one point of the chisel and/or the chisel holder is determined using a measuring method. The measurement result or an offsetting of the measurement result is compared in a switching unit with at least one reference value stored in a memory unit.
- AU 2014262221 B2 discloses a method and tool for monitoring the condition, condition and performance of wear parts used in tillage implements. The process and tool allow the operator to optimize the performance of the tillage implement.
- the tool has a clear positional relationship to wear parts during use and is used in conjunction with a shovel or blade on the tillage implement.
- the overall monitoring system includes a backhoe having walls defining an enclosure portion for collecting earthen materials, a digging edge, at least one wear part attached to the digging edge, at least one electronic sensor attached to one of the walls, and a programmable logic device .
- the logic device receives information from the at least one electronic sensor and determines the status of the presence of the bucket, the state of preservation or the degree of wear, the filling and performance of the bucket and/or the at least one wear part.
- the photo-optical methods mentioned are exclusively dependent on photographs of the machines/parts or tools to be monitored. In the case of moving machines/parts or tools, these photographs are often subject to major disruptive influences due to their movement. These interferences are amplified by additional image information, such as an uneven background.
- additional image information such as an uneven background.
- the use of operating hours as a measure of wear is a static variable that takes no account of the actual condition of the tool to be monitored.
- the object of the invention is therefore to overcome the obvious disadvantages of the prior art. Individualize the monitoring and evaluation of tool states and make them independent of interference.
- a computer-implemented method for the optical detection, detection and quantification of relevant states and/or changes thereto in at least one target object has the following steps: a) recording an image sequence, with each image of the image sequence showing at least the target object and/or at least contains the section of the target object, b) comparing at least two of the images in the image sequence with one another and determining similarities and/or determining differences between the at least two images in the image sequence, c) segmenting all images in the image sequence into background and foreground, or vice versa, d) identifying at least one target object in the foreground, e) determining relevant pixels of the target object, f) measuring at least one geometric property of the target object using relevant pixels, g) comparing the geometric properties of the target object with the geometric properties of a comparison object and h) classifying the state of the target object in
- the at least one target object being arranged at least temporarily opposite an optical sensor device and the states of the error classes deviating from the optimal state, characterized in that the distinction between background and target object is made possible by a higher rate of change of the image information of the background compared to the rate of change of the image information of the target object.
- digital image information is converted into the frequency domain by means of a fast Fourier transformation. Those parts of the frequency data set calculated in this way that describe the color and/or contrast changes with high fluctuation over time are removed.
- the background and target object are distinguished by a higher rate of change of the image information.
- the rate of change of the image information is understood here as the extent of the change in image information over a specific period of time in relation to the duration of this period of time; the rate of change of the image information is therefore a measure of how quickly image information changes.
- the duration if the unit of measure in the denominator contains a unit of time; the counter contains a unit of image information - for example bytes.
- the target object is that object which is subjected to the opto-sensory monitoring.
- tools for tillage are generally understood here.
- relevant states are understood to mean the states of the target object that are considered relevant for the specific application. For example, but not exclusively, the degree of contamination can be classified as relevant if this impairs the possibility of opto-sensory detection too much. Loss, destruction, deformation, displacement, contamination and/or wear and tear thus also fall within the category of relevant states. It is also conceivable that no change to the target object is classified as relevant.
- Image segmentation is a part of digital image processing.
- the generation of content-related regions by combining them into pixels or voxels according to a freely chosen criterion is referred to as segmentation.
- the position of the target object and/or the section of the target object does not change relative to the edges of the image on average over time. This makes it easier for the algorithm to carry out the image segmentation and to identify the target object or the section of the target object.
- determining the position of the target relative to the edges of the image is complicated by vibrational motion.
- By adapting the scanning rate of the camera to higher harmonics of the vibration and/or changing the sampling a state of relative constancy of the distances from the target object to the edges of the image is created.
- Software adjustments in the algorithm that controls the camera control and its sampling rate are often sufficient. This is advantageous because in this way no changes have to be made to the embodiment of the actual sensor system.
- the optical sensor system is supplemented by a stroboscopic light source.
- the frequency of the light flashes is selected in such a way that the illuminated target object appears relatively constant in relation to the edges of the image or with the drift movement distinguishable from the movement of the background. This is advantageous because in this way objects that are periodically guided past the optical sensor system can also be made accessible to the method.
- multiple image sequences are recorded according to the invention. These image sequences are divided into different sequences. After the subdivision into sequences, at least one sequence of the sequence of images is subjected to a method for averaging image information in order to improve the image quality of the sequences.
- the comparison object contains the measurement data of the geometric object information for comparison with the target object from a normal reference.
- the normal reference includes a model adaptation to possible state classes. For example, a target object—or tool—that is in operation is checked for its degree of wear. If a user finds this to be suitable despite clear signs of use, the model to be referenced and the associated model data are adapted to this condition.
- self-referenced model data is used in order to achieve the best possible state with correction algorithms and in this way to create a calibration standard yourself. This is advantageous, since an individual adjustment to the changed system can be carried out when a tool is changed.
- the comparison object is present as a virtual comparison object in all classifications of the state. For example, but not exclusively, a database is created in which all error messages that have occurred so far are stored with the associated sensory and virtual data. By comparing the data within an error class, it is advantageously possible to provide a detailed model of the respective error class for comparison or as a comparison object. Self-learning and self-improvement of these comparison objects is also advantageously made possible in this way.
- the geometric object information of the comparison object is available from model data.
- model data for example, but not exclusively, at least one true-to-scale model of the target object to be monitored is produced as a real embodiment.
- possible different error classes are represented and examined with regard to the resulting dimensions or, for example, but not exclusively, used by a direct image comparison of model and target object.
- data processing units are used, which are connected locally at the place of use and/or via remote transmission to the necessary sensor systems for data recording.
- computer-readable storage media on which a computer program product suitable for executing a computer-implemented method are used also, but not exclusively, to store the comparison data of the individual error classes.
- Output values of the computer-implemented method can serve as input parameters of a closed-loop and/or open-loop control unit.
- a model test setup is monitored by the method.
- the real equivalent of the system is in an environment that only allows limited sensory monitoring. If output values for specific error classes are now obtained from the model test, these can be transferred manually from the model test to the controller of the real system for the regulation or control, for example.
- Fig. 1 represents a possible embodiment of the invention in a schematic and sketchy manner.
- This is a main frame/carrier frame (1), the individual Components of the sensor system according to the invention are shown.
- the main frame serves as a basis for mounting the at least one camera (4) required according to the invention and for mounting the tool module (2).
- At least one tool (3) is located on the tool module, which according to the invention can be described as a target object.
- the measurement setup also includes the background of the machine system (6) and a computer (5), which is required to evaluate the sensor data.
- FIG. 3 shows possible tool shapes by way of example, the condition of which can be assessed using the method according to the invention.
- FIG 4 shows the sequence of the computer-implemented method according to the invention in its basic functions.
- the condition of the individual tools of an agricultural implement for tilling the soil is assessed on a carrier frame (1) using the method according to the invention.
- the optical sensor (4) is mounted on the support frame (1).
- the sensor (4) is connected to an evaluation unit, the computer (5), so to speak, in such a way that the sensor data can be transmitted according to the invention.
- a tool module (2) which carries the individual tools (3), is mounted on the support frame (1).
- the support frame (1) can also contain only a number of individual tools (3) which are connected directly to the support frame (1) without a tool module (2).
- the number of optical sensors (4) and individual tools (3) is at least one. This means that a different number of sensors are possible for different requirements from the measurement task.
- At least one tool (2) which forms the target object, is detected with the aid of the sensor (4).
- the background (6) is also recorded in the sensor section.
- the program flow is shown in FIG. Acquisition begins with the start command/trigger in the software.
- the start command can, for example, be issued automatically using position sensors of the tools or manually by an operator.
- An image is captured with a time stamp. Based on the recording of an image sequence, the image is segmented into foreground and background with the aid of averaging processes.
- the target objects (3) can be determined from this in the foreground.
- the background information can also be used to determine machine movement Aids for tolerance parameters of the measurement can be used.
- the foreground object is structured to identify characteristic geometries (skeleton or edges). Measuring points are then determined in the determined characteristic shapes and are compared with a model object, or direct measurements can be carried out based on the position of the measuring points.
- the status parameters When assigning the measuring points using a model object, the status parameters must then be determined indirectly. Finally, the individual tools and thus the machine condition can be classified. Classification can be understood as the determination of predefined states. The states of the tool or tools can either result in instructions and/or are quantified and output and stored as a measurement signal/state. The measured values can in turn be used as control parameters for the tools, such as setting the working depth, compensating for topological conditions.
- the status classes are defined as follows:
- the individual tool and/or the entire tool module meets the requirements and no measures are required.
- a wear limit is reached or exceeded, a status message is issued and a measured value is determined at the same time, which provides the user or a controller with information about, for example, the remaining working time until a measure becomes necessary.
- a broken tool for example, also represents a state of wear, but it occurs suddenly and does not progressively progress. Thus, this state is a special case of state 2.
- a status message with, if necessary, a measured value is output to the operator or the controller.
- Overloading the tool can also cause permanent deformation in the tool, which can have a negative impact on the desired work result. From here the system generates a status message that provides information as to whether the shape of the tool is within predefined limits.
- Condition 5 Tool dirty/not recognizable.
- the system must provide a status report about the contamination in order to initiate countermeasures.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Geometry (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Software Systems (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Linguistics (AREA)
- Evolutionary Biology (AREA)
- Mathematical Physics (AREA)
- Image Processing (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Measurement Of Mechanical Vibrations Or Ultrasonic Waves (AREA)
- Image Analysis (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| DE102020128759.3A DE102020128759B4 (de) | 2020-11-02 | 2020-11-02 | Softwareverfahren zur opto-sensorischen Erkennung, Vermessung und Bewertung von Werkzeugzuständen |
| PCT/EP2021/080268 WO2022090539A1 (de) | 2020-11-02 | 2021-11-01 | Softwareverfahren zur opto-sensorischen erkennung, vermessung und bewertung von werkzeugzuständen |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| EP4238070A1 true EP4238070A1 (de) | 2023-09-06 |
Family
ID=78536193
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP21805449.2A Withdrawn EP4238070A1 (de) | 2020-11-02 | 2021-11-01 | Softwareverfahren zur opto-sensorischen erkennung, vermessung und bewertung von werkzeugzuständen |
Country Status (7)
| Country | Link |
|---|---|
| US (1) | US20230419502A1 (de) |
| EP (1) | EP4238070A1 (de) |
| AU (1) | AU2021370993A1 (de) |
| CA (1) | CA3197238A1 (de) |
| DE (1) | DE102020128759B4 (de) |
| WO (1) | WO2022090539A1 (de) |
| ZA (1) | ZA202304710B (de) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102023131635A1 (de) * | 2023-11-14 | 2025-05-15 | Pöttinger Landtechnik Gmbh | Computerimplementiertes Verfahren zur Bestimmung des Verschleißzustandes eines an einem landwirtschaftlichen Arbeitsgerät, sowie landwirtschaftliches Arbeitsgerät, sowie Computerprogrammprodukt, sowie computerlesbarer Datenträger |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002018680A (ja) | 2000-07-10 | 2002-01-22 | Mitsubishi Electric Corp | 工作機械 |
| DE102008045470A1 (de) | 2008-09-03 | 2010-03-04 | Wirtgen Gmbh | Verfahren zur Bestimmung des Verschleißzustandes |
| AU2014262221C1 (en) | 2013-11-25 | 2021-06-10 | Esco Group Llc | Wear part monitoring |
| DE102018121997A1 (de) | 2018-09-10 | 2020-03-12 | Pöttinger Landtechnik Gmbh | Verfahren und Vorrichtung zur Verschleißerkennung eines Bauteils für landwirtschaftliche Geräte |
-
2020
- 2020-11-02 DE DE102020128759.3A patent/DE102020128759B4/de active Active
-
2021
- 2021-11-01 US US18/251,495 patent/US20230419502A1/en not_active Abandoned
- 2021-11-01 WO PCT/EP2021/080268 patent/WO2022090539A1/de not_active Ceased
- 2021-11-01 AU AU2021370993A patent/AU2021370993A1/en not_active Abandoned
- 2021-11-01 EP EP21805449.2A patent/EP4238070A1/de not_active Withdrawn
- 2021-11-01 CA CA3197238A patent/CA3197238A1/en active Pending
-
2023
- 2023-04-24 ZA ZA2023/04710A patent/ZA202304710B/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| DE102020128759B4 (de) | 2024-09-05 |
| DE102020128759A1 (de) | 2022-05-05 |
| AU2021370993A9 (en) | 2024-02-08 |
| WO2022090539A1 (de) | 2022-05-05 |
| AU2021370993A1 (en) | 2023-06-22 |
| ZA202304710B (en) | 2024-05-30 |
| US20230419502A1 (en) | 2023-12-28 |
| CA3197238A1 (en) | 2022-05-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| DE60211002T2 (de) | Verfahren und system zum analysieren der steuerschaltungsleistungsfähigkeit in einem industriellen prozess | |
| DE69625000T2 (de) | Schwingungsüberwachungsvorrichtung und Überwachungsverfahren | |
| WO2017167616A1 (de) | Verfahren zur schwingungsdiagnostischen überwachung einer maschine | |
| DE102017011740B4 (de) | Störungsermittlungsvorrichtung und Störungsermittlungsverfahren | |
| DE102017003165A1 (de) | Maschinelle Lernvorrichtung und maschinelles Lernverfahren zum Lernen der Fehlervorhersage einer Hauptwelle oder eines Motors, der die Hauptwelle antreibt, und Fehlervorhersagevorrichtung und Fehlervorhersagesystem umfassend eine maschinelle Lernvorrichtung | |
| DE10007972A1 (de) | Diagnosevorrichtung und -verfahren in einem Prozeßsteuersystem | |
| DE102007048602A1 (de) | Verfahren zum Diagnostizieren von Anomalitäten und Vorrichtung hierfür | |
| DE102019128177A1 (de) | Vorrichtung und verfahren zur zustandsbestimmung | |
| DE102018009309B4 (de) | Numerisches Steuergerät | |
| EP1123483A1 (de) | Unebenheiten in einer gewölbten oberfläche, wie reifenseitenwand, mit bandpassfilterung | |
| DE102012220222A1 (de) | Vorrichtung und Verfahren zur Zustandsüberwachung eines Wälzlagers | |
| DE10310116A1 (de) | Risikominimierung und Wartungsoptimierung durch Ermittlung von Schädigungsanteilen aus Betriebsdaten | |
| DE102018127844A1 (de) | Verfahren zur Regelung des Betriebs einer Maschine zum Ernten von Hackfrüchten | |
| DE102019100722A1 (de) | Erkennung von Spitzen und Fehlern in Schwingungstrendddaten | |
| DE102018124237A1 (de) | Abnormalitätsanzeichen-Meldesystem, -Verfahren und -Programm | |
| DE10348608A1 (de) | Spindelvorrichtung mit Zustandsüberwachung sowie entsprechendes Überwachungsverfahren und -system | |
| DE69713600T2 (de) | System zur überwachung und diagnose von automatischen regelschleifen | |
| DE102021132755A1 (de) | Verfahren und system zum überwachen von herstellungsvorgängen unter verwendung von maschinellem sehen für von menschen durchgeführte aufgaben | |
| DE102019110781A1 (de) | Verfahren und vorrichtung zur überwachung eines drehbaren elements | |
| EP4068018A1 (de) | Vorrichtung und verfahren zur überwachung eines stanzprozesses | |
| EP4238070A1 (de) | Softwareverfahren zur opto-sensorischen erkennung, vermessung und bewertung von werkzeugzuständen | |
| DE102022105681A1 (de) | Verfahren zur Ermittlung einer Schwingung eines Lüftungssystems | |
| EP4068019A1 (de) | Vorrichtung und verfahren zur aufbereitung der erfassten zustandsgrössen eines stanzprozesses | |
| WO2010046246A1 (de) | Verfahren und vorrichtung zur detektion von prozesszuständen in alternierenden produktionsprozessen | |
| EP1275772A2 (de) | Verfahren und Einrichtung zur Überwachung des Zustandes eines Filzes oder Siebes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
| 17P | Request for examination filed |
Effective date: 20230425 |
|
| AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| DAV | Request for validation of the european patent (deleted) | ||
| DAX | Request for extension of the european patent (deleted) | ||
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
| 17Q | First examination report despatched |
Effective date: 20250808 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
| 18D | Application deemed to be withdrawn |
Effective date: 20251209 |