DE102021002707B4 - Method and measuring device for determining the position of a movable machine part of a machine - Google Patents

Abstract

Method for determining the position of a movable machine part (20) of a machine (10), a plurality of optical sensors (122) being arranged on the machine (10), each of which has a fixed sensor field of view (126) in relation to its arrangement on the machine (10), and a plurality of reference objects (130) are arranged on the movable machine part (20) and each individual reference object (130) is fixed at a fixed reference position on the machine part (20), the respective sensor fields of view (126) of the optical sensors (122) are aligned in the direction of the reference objects (130), the method having the following steps: - providing a virtual model of the machine (10'), the virtual optical sensors (122`) with virtual sensor fields of view (126') and virtual reference objects (130'), wherein the orientation of the virtual optical sensors (122') corresponds to that of the optical sensors (122) and the positioning of the virtual reference objects (130') corresponds to that of the reference objects (130);- in an initialization step (t0):- Determining an initial sensor field of view position of the individual reference objects (130) in the respective sensor fields of view (126) of the respective optical sensors (122) and adapting the virtual model of the machine (10') to the initial sensor field of view positions of the individual reference objects (130) ;- in a first time interval (t1) following the initialization step:- activating the reference objects (130) and recording by the optical sensors (122);- determining the new sensor field of view position of the individual activated reference objects (130) in the individual recordings of the optical sensors (122);- adapting the virtual model of the machine (10') to the new sensor field of view positions of the individual reference objects (130) by performing a minimization function;- determining the position of the movable machine part (20) by reading the virtual position of the virtual Machine part (20') from the virtual model, wherein the determination of the initial sensor field of view position of the individual reference objects (130) has the following steps: - activating one of the reference objects (130) and taking pictures by all optical sensors (122) while the reference object ( 130) is activated;- evaluating the respective recordings of the optical sensors (122) in which the reference object (130) is activated, whereby the initial sensor field of view position of the reference object (130) is determined;- repeating the aforementioned method steps with all other reference objects (130) , whereby all initial sensor field of view positions of the reference objects (130) are determined.

Description

The invention relates to a method and a measuring device for determining the position of a movable machine part of a machine.

Industrial machines often have a machine part or a plurality of machine parts that are movably and controllably arranged in relation to the machine. Examples of this are construction machines such as earthmoving machines or drilling rigs with support arms. Machine tools with movable tool support arms can also be mentioned by way of example.

In machines of this type, it is important to determine the position of the movable machine part or machine parts or to determine the position of the machine itself in relation to a predefined reference coordinate system in order to be able to carry out the desired work, such as earthmoving work or drilling work, with the desired accuracy and quality, with a high level of temporal resolution is required. If, for example, a drill rig is used in tunneling to drill holes for the introduction of explosives in a working face of the tunnel structure, the exact positioning of the support arms carrying the drilling device is of great importance so that the holes can be drilled in such a way that the blasting can be carried out as desired .

Conventionally, the position of such machine parts, such as the support arms mentioned above, is determined by measuring the individual drives of the moving machine parts. If, for example, the position of the movable machine part is controlled by means of linear motors and/or stepping motors, the position of the individual drives can be monitored and determined, which means that the position of the movable machine part can then be deduced.

However, if the determination of the position of one of the drives that move the movable machine part has an error or is inaccurate, this can lead to an incorrect or inaccurate determination of the position of the movable machine part. In addition, such an error can multiply or continue due to the combination of several drives for moving the machine part, so that the position of the movable machine part can be determined increasingly imprecisely with an increasing number of movement motors.

The document JP 2020 - 97 821 A relates to a drilling system according to comprising: at least two temporary markers and at least three normal markers mounted in a longitudinal direction of a rod-shaped guide case pivotally protruding from a vehicle body of a drilling machine; a plurality of receiving means installed at intervals to acquire reception information received from each of the markers; means for calculating coordinate positions of the markers from the received information acquired by a plurality of receiving means; means for storing the calculated coordinate positions of the markers; and second calculation means for calculating coordinate positions of the temporary markers from the coordinate positions after movement of the normal markers after the temporary markers have been removed.

The document U.S. 2005/0 159 916 A1 relates to a method to correct the field angle of an image capturing device capturing an image of a real space containing a feature point whose position in the real space is known. The position of the feature point (real marker position) in the real space image captured by the image capturing device is detected. Then, the position of the feature point (virtual marker position) seen when the real space is viewed from a virtual viewpoint specified by the measured position, orientation, and field angle of the imaging device in this image is calculated. A correction amount for the position and/or a correction amount for the orientation of the image capturing device is/are calculated in such a way that a deviation between the real and the virtual marker position in the image is minimized.

The document U.S. 2011 10 169 949 A1 relates to a system for determining the orientation of an implement on a vehicle. The system includes a vehicle-mounted camera and a target located in a field of view of the camera and mounted on the implement. The target object contains markings that allow the orientation of the implement in relation to the camera to be determined.

The document DE 10 2019 220 557 A1 relates to a method for locating at least one component of a machine, with a large number of localization elements being arranged on structural elements of the machine that can be moved relative to one another according to predetermined kinematics, with at least two of the localization elements preferably having a respective RTLS module, with the steps of: determining a Variety of distances between any two of the locality tion elements, wherein determining the distances preferably comprises measuring at least one distance between RTLS modules, determining at least one angle, which is defined by three of the localization elements, using the determined distances; and locating the at least one component using the at least one determined angle, the determination of the plurality of distances and/or the determination of the at least one angle taking into account the specified kinematics.

The document U.S. 2015 10 168 136 A1 relates to a method and system for estimating the three-dimensional (3D) position and orientation (pose) of an articulated machine, such as. B. an excavator, includes the use of one or more markers and one or more imaging devices. Images of the marker(s) are captured with the image capturing device(s) and the captured images are used to determine the posture.

The document DE 10 2014 209 371 A1 relates to a system and method for controlling a working machine with a boom, with a control unit and a 3D camera for detecting the position of the boom, markers being arranged at at least one point of the boom, and the 3D camera and the control unit for monitoring the position of the markers are formed.

The object of the present disclosure is therefore to create a method and a measuring device with which it is possible to determine the position of moving machine parts of a machine with high accuracy and robustness.

The object is solved by the features of the independent patent claims. Advantageous developments of the present disclosure are specified in the dependent claims and the description.

• - Bereitstellen eines virtuellen Modells der Maschine, das virtuelle optische Sensoren mit virtuellen Sensorblickfeldern und virtuelle Referenzobjekte aufweist, wobei die Ausrichtung der virtuellen optischen Sensoren der der optischen Sensoren entspricht und die Positionierung der virtuellen Referenzobjekte der der Referenzobjekte entspricht. Das virtuelle Modell bildet die Maschine mit dem beweglichen Maschinenteil, ggf. mit mehreren beweglichen Maschinenteilen, virtuell ab und entspricht demgemäß einem virtuellen Zwilling der realen Maschine. Die Bewegungsmöglichkeiten des beweglichen Maschinenteils können in dem virtuellen Modell auch abgebildet werden. Gemäß der vorliegenden Offenbarung bildet das virtuelle Modell insbesondere auch die optischen Sensoren und die Referenzobjekte an der Maschine ab. In anderen Worten, so wie die optischen Sensoren und die Referenzobjekte an der realen Maschine angeordnet sind, sind sie auch in dem virtuellen Modell der Maschine angeordnet.
• - in einem Initialisierungsschritt:
• - Bestimmen einer initialen Sensorblickfeldposition der einzelnen Referenzobjekte in den jeweiligen Sensorblickfeldern der jeweiligen optischen Sensoren und Anpassen des virtuellen Modells der Maschine an die initiale Sensorblickfeldpositionen der einzelnen Referenzobjekte. Die Sensorblickfeldposition ist jene Position in dem Sensorblickfeld, an welcher das Referenzobjekt sich gerade befindet. Sie kann gemäß einer Ausführungsform mittels kartesischer oder polarer Koordinaten oder einer Pixelposition angegeben werden. Mittels der bestimmten initialen Sensorblickfeldposition wird das virtuelle Modell der Maschine entsprechend ausgerichtet / angepasst, indem die virtuellen Sensorblickfeldposition der ermittelten initialen Sensorblickfeldposition angeglichen wird. In anderen Worten, die virtuellen Referenzobjekte werden so verschoben, dass die virtuelle Sensorblickfeldposition der virtuellen Referenzobjekte der Sensorblickfeldposition der realen Referenzobjekte entspricht. Dies kann gemäß einer Ausführungsform mittels einer Minimierungsfunktion durchgeführt werden. Mittels dieses Verfahrensschritts wird die Initialisierung, die Bestimmung der initialen Position des beweglichen Maschinenteils, erreicht.
• - in einem dem Initialisierungsschritt nachfolgenden ersten Zeitintervall:
  • - Aktivieren der Referenzobjekte und Aufnehmen von Aufnahmen durch die optischen Sensoren. Die einzelnen Referenzobjekte werden aktiviert bzw. eingeschaltet, und gleichzeitig werden die Aufnahmen mit den optischen Sensoren gemacht, während die Referenzobjekte eingeschaltet sind. Gemäß einer Ausführungsform werden alle Referenzobjekte gleichzeitig oder nacheinander aktiviert und entsprechend von den optischen Sensoren die Aufnahmen gemacht. Nach der Initialisierung erfolgte beispielsweise eine Bewegung des beweglichen Maschinenteils, sodass mindestens eines der Referenzobjekte auch bewegt wurde.
  • - Ermitteln der neuen Sensorblickfeldposition der einzelnen aktivierten Referenzobjekte in den einzelnen Aufnahmen der optischen Sensoren. Die einzelnen Aufnahmen der optischen Sensoren werden dahingehend ausgewertet, dass die neue Sensorblickfeldposition der einzelnen aktivierten Referenzobjekte ermittelt wird. Aus dem Initialisierungsschritt ist die initiale Sensorblickfeldposition der jeweiligen Referenzobjekte bekannt. Hat sich das bewegliche Maschinenteil verschoben, also entsprechend auch das Referenzobjekt, ist in der neuen Aufnahme auch die Sensorblickfeldposition des verschobenen Referenzobjekts eine andere. Die zeitliche Auflösung wird derart gewählt, dass die Zuordnung der Referenzobjekte dauerhaft möglich bleibt.
  • - Anpassen des virtuellen Modells der Maschine an die neuen Sensorblickfeldpositionen der einzelnen Referenzobjekte, indem eine Minimierungsfunktion ausgeführt wird. Ähnlich zum Initialisierungsschritt wird auch gemäß dieses Verfahrensschrittes das virtuelle Modell der Maschine mittels der ermittelten Sensorblickfeldposition angepasst, indem die virtuelle Sensorblickfeldposition an die ermittelten neuen Sensorblickfeldposition angeglichen wird. In anderen Worten, die virtuellen Referenzobjekte werden so verschoben, dass die virtuelle Sensorblickfeldposition der virtuellen Referenzobjekte der neuen Sensorblickfeldposition der realen Referenzobjekte entspricht. Gemäß der vorliegenden Offenbarung wird dies mittels der Minimierungsfunktion durchgeführt. Die Minimierungsfunktion ermittelt demgemäß die wahrscheinlichste Position und Ausrichtung des virtuellen Maschinenteils. Dafür werden der Minimierungsfunktion als Eingangsgrößen die neuen Sensorblickfelspositionen zugeführt.
  • - Bestimmen der Position und Ausrichtung des beweglichen Maschinenteils durch auslesen der virtuellen Position des virtuellen Maschinenteils aus dem virtuellen Modell. Die Position und Ausrichtung des beweglichen Maschinenteils kann unmittelbar aus der virtuellen Position und Ausrichtung des virtuellen Maschinenteils bestimmt werden, indem die virtuelle Position ausgelesen wird. Die vorliegende Offenbarung schafft demgemäß ein sehr einfaches, robustes und genaues Verfahren zur Positionsbestimmung des beweglichen Maschinenteils.

According to the present disclosure, a method for determining the position of a movable machine part of a machine has the steps listed below. A plurality of optical sensors are arranged on the machine, each having a fixed sensor field of view in relation to their arrangement on the machine, and a plurality of reference objects are arranged on the movable machine part and each one of the reference objects is at a fixed reference position on the machine part fixed, with the respective sensor fields of view of the optical sensors being aligned in the direction of the reference objects. The process steps are:

• - providing a virtual model of the machine comprising virtual optical sensors with virtual sensor fields of view and virtual reference objects, the orientation of the virtual optical sensors corresponding to that of the optical sensors and the positioning of the virtual reference objects corresponding to that of the reference objects. The virtual model virtually depicts the machine with the moving machine part, optionally with several moving machine parts, and accordingly corresponds to a virtual twin of the real machine. The possible movements of the moving machine part can also be mapped in the virtual model. According to the present disclosure, the virtual model in particular also depicts the optical sensors and the reference objects on the machine. In other words, just as the optical sensors and the reference objects are arranged on the real machine, they are also arranged in the virtual model of the machine.
• - in an initialization step:
• - Determining an initial sensor field of view position of the individual reference objects in the respective sensor fields of view of the respective optical sensors and adapting the virtual model of the machine to the initial sensor field of view positions of the individual reference objects. The sensor field of view position is that position in the sensor field of view at which the reference object is currently located. According to one embodiment, it can be specified using Cartesian or polar coordinates or a pixel position. The virtual model of the machine is aligned/adjusted accordingly by means of the determined initial sensor field of view position, in that the virtual sensor field of view position is adjusted to the determined initial sensor field of view position. In other words, the virtual reference objects are shifted such that the virtual sensor field of view position of the virtual reference objects corresponds to the sensor field of view position of the real reference objects. According to one embodiment, this can be performed by means of a minimization function. The initialization, the determination of the initial position of the movable machine part, is achieved by means of this method step.
• - in a first time interval following the initialization step:
  • - Activating the reference objects and taking pictures through the optical sensors. The individual reference objects are activated or switched on, and at the same time the recordings are made with the optical sensors while the reference objects are switched on. According to one embodiment, all reference objects are activated simultaneously or one after the other and the recordings are made accordingly by the optical sensors. After the initialization, for example, the movable machine part moved, so that at least one of the reference objects was also moved.
  • - Determination of the new sensor field of view position of the individual activated reference objects in the individual recordings of the optical sensors. The individual recordings of the optical sensors are evaluated in such a way that the new sensor field of view position of the individual activated reference objects is determined. The initial sensor field of view position of the respective reference objects is known from the initialization step. If the moving machine part has shifted, i.e. the reference object as well, the sensor field of view position of the shifted reference object is also different in the new recording. The temporal resolution is selected in such a way that the assignment of the reference objects remains permanently possible.
  • - Adjusting the virtual model of the machine to the new sensor field of view positions of each reference object by performing a minimization function. Similar to the initialization step, the virtual model of the machine is also adapted according to this method step using the determined sensor field of view position by the virtual sensor field of view position being adjusted to the new sensor field of view position determined. In other words, the virtual reference objects are shifted such that the virtual sensor field of view position of the virtual reference objects corresponds to the new sensor field of view position of the real reference objects. According to the present disclosure, this is done using the minimize function. Accordingly, the minimization function determines the most probable position and orientation of the virtual machine part. For this purpose, the new sensor field of view positions are fed to the minimization function as input variables.
  • - Determining the position and orientation of the movable machine part by reading the virtual position of the virtual machine part from the virtual model. The position and orientation of the movable machine part can be determined directly from the virtual position and orientation of the virtual machine part by reading out the virtual position. Accordingly, the present disclosure provides a very simple, robust and accurate method for determining the position of the moving machine part.

The optical sensors are located on the machine in accordance with the present disclosure. According to one embodiment, the optical sensors can be arranged on a non-moving area of the machine or, according to another embodiment, can be arranged at least partially on one of the moving machine parts. For example, if the machine is a construction machine, the movable machine part can be a support arm. The support arm as a moving machine part is therefore movable in relation to the construction machine. In addition, the construction machine can be moved in relation to an external reference coordinate system.

The fixed sensor field of view of the optical sensors according to the present disclosure is determined by the orientation of the optical sensors and accordingly by the placement of the optical sensors on the machine. According to the present disclosure, the sensor field of view of the optical sensors is aligned in the direction of at least one of the reference objects. This should not be understood to mean that the sensor field of view of the optical sensors must always/permanently capture at least one of the reference objects. It is quite possible that at least temporarily the sensor field of view of one of the optical sensors does not detect a reference object. A sensible arrangement and alignment of the optical sensors ensures that at least one of the reference objects can be viewed by one of the optical sensors in every possible position and alignment of the moving machine part.

Accordingly, at least one of the reference objects is detected by the optical sensors. According to one embodiment, the sensor field of view of at least one of the optical sensors is aligned in the direction of several of the reference objects. In other words, in at least one position of the movable machine part, at least one of the reference objects can be detected by the corresponding optical sensor. Due to the fact that the moving machine part moves in relation to the fields of view of the optical sensors, the number of reference objects in the field of view of the respective optical sensors can change.

Accordingly, according to the present disclosure, it is possible to determine the position of the movable machine part very precisely by means of a comparatively simple structure consisting of the reference objects and the optical sensors.

In order to determine the position of the individual reference objects, it is advantageous for the individual reference objects to be in the sensor field of view of two or more optical sensors. The higher the number, the greater the redundancy, which can increase the accuracy of the position determination. In addition, if there are several recordings, a selection can be made from the recordings that are used to determine the position. As a result, the accuracy of the position determination of the reference objects can be increased.

It is conceivable that the individual reference objects leave the field of view of one of the optical sensors due to the movement of the movable machine part and move into the field of view of another of the optical sensors. According to one embodiment, this can be recognized and even calculated in advance by evaluating the individual recordings of the optical sensors by determining a predicted movement of the movable machine part. Because this is taken into account, the accuracy of the position determination of the optical reference objects can be further advantageously increased.

According to one embodiment, the position of the movable machine part is continuously determined by continuously repeating the method of the first time interval. This enables a continuous determination of the position of the individual reference objects. According to a further embodiment, the measuring rate is selected as high as possible and the measuring time as short as possible for the individual subsequent time intervals, so that the position determination can advantageously be carried out precisely, since the movable machine part can move less in a comparatively short time, so that the assignment is simple and the overall measurement error is small.

• - Aktivieren eines einzigen der Referenzobjekte und Aufnehmen von Aufnahmen durch alle optischen Sensoren während das Referenzobjekt aktiviert ist. Die weiteren Referenzobjekte sind dabei nicht aktiviert bzw. ausgeschaltet.
• - Auswerten der jeweiligen Aufnahmen der optischen Sensoren in denen das Referenzobjekt aktiviert ist, wodurch die initiale Sensorblickfeldposition des Referenzobjekts bestimmt wird. Während das eine Referenzobjekt aktiviert ist, werden mit allen der optischen Sensoren eine Aufnahme gemacht. Diese Aufnahmen werden ausgewertet, wodurch die initiale Sensorblickfeldposition des aktivierten Referenzobjekts der unterschiedlichen optischen Sensoren ermittelt wird.
• - Wiederholen der vorgenannten Verfahrensschritte mit allen weiteren Referenzobjekten, wodurch alle initialen Sensorblickfeldpositionen der Referenzobjekte bestimmt werden. Sobald die erste Sensorblickfeldposition des ersten Referenzobjekts ermittelt wurde, wird ein zweites Referenzobjekt aktiviert, eingeschaltet, während die anderen Referenzobjekte ausgeschaltet bleiben. Der Vorgang wird für das zweite Referenzobjekt und anschließend für die weiteren Referenzobjekte wiederholt, bis für alle Referenzobjekte die initiale Sensorblickfeldposition ermittelt worden ist. Gemäß dieser Ausführungsform ist die initiale Zuordnung der Referenzobjekte und die initiale Sensorblickfeldposition aller Referenzobjekte vorteilhaft einfach möglich.

According to the present disclosure, the initial sensor field of view position of each reference object is determined as follows:

• - Activating a single one of the reference objects and taking pictures through all optical sensors while the reference object is activated. The other reference objects are not activated or switched off.
• - Evaluation of the respective recordings of the optical sensors in which the reference object is activated, whereby the initial sensor field of view position of the reference object is determined. While one reference object is activated, a picture is taken with all of the optical sensors. These recordings are evaluated, whereby the initial sensor field of view position of the activated reference object of the different optical sensors is determined.
• - Repeating the aforementioned method steps with all further reference objects, whereby all initial sensor field of view positions of the reference objects are determined. Once the first sensor field of view position of the first reference object is determined, a second reference object is activated, turned on while the other reference objects remain off. The process is repeated for the second reference object and then for the further reference objects until the initial sensor field of view position has been determined for all reference objects. According to this embodiment, the initial assignment of the reference objects and the initial sensor field of view position of all reference objects is advantageously possible in a simple manner.

According to one embodiment, in addition to the recordings in which the respective reference object is activated, a further black recording is recorded with each of the optical sensors and compared with the recordings in which the respective reference object is activated, whereby environmental influences are filtered out. Black recordings are recordings of the optical sensors in which none of the reference objects is activated. Accordingly, these black recordings only show environmental influences such as radiation from radiators or lamps, for example construction site spotlights. According to this embodiment, such environmental influences are identified, classified as such and can be taken into account accordingly when determining the sensor field of view position of the activated reference objects. According to this embodiment, the position determination method becomes more accurate and robust. For example, if a radiation source, such as a lamp/radiator, is switched on in the field of view of one of the optical sensors, it can be detected in the black recording. Reference objects that are in a similar field of view position as the source of interference cannot be used for subsequent recordings, or only to a limited extent, until these reference objects have been moved to a different field of view position away from the source of interference by moving the movable machine part or the machine. In the meantime, the position can be measured using other reference objects. As a result, the increase in robustness/stability with respect to environmental influences when determining the position can be additionally advantageously increased.

According to one embodiment, a system of linear equations is first solved by means of the minimization function in order to find an optimal starting point for a subsequent numerical optimization tion, with the numerical optimization then determining the global minimum or the global maximum starting from the starting point, as a result of which the virtual model of the machine is adapted using the new sensor field of view positions of the individual reference objects. The currently determined sensor field of view positions are provided to the minimization function as input variables. The minimization function then tries to minimize a residue, first determining the position of the reference objects in the virtual model. The individual 3D core rays are then calculated from the sensor field of view positions of the optical sensors, after which the distances between the 3D core rays and the associated virtual reference objects are calculated. The root mean square of the individual distances is then calculated. The resulting root mean square represents the residual, which is minimized. As soon as certain parameters are reached, the minimization function is aborted and the result is returned by the minimization function.

According to one embodiment, at least one position sensor is arranged on the movable machine part, with the position sensor being an acceleration sensor and/or a gyroscope, with direction data relating to the gravitational vector being determined by means of the position sensor and these direction data also being used when determining the position of the movable machine part. According to one embodiment, several position sensors are arranged on the movable machine part. According to a further embodiment, at least one position sensor is arranged on a plurality of movable machine parts. The position sensors are fixed or screwed to the moving machine part. As an output, the position sensors provide detailed information on the direction of the gravitational vector and/or detailed information on a linear movement, acceleration and/or rotation of the movable machine part on which they are arranged. According to this embodiment, this information is supplied to the minimization function as additional input variables, which means that the accuracy of the result of the minimization function can be increased, which in turn can increase the accuracy and robustness of the determination of the position of the movable machine part. According to a further embodiment, the position sensor can also be integrated in one of the reference objects.

According to one embodiment, at least one position sensor is additionally arranged on the machine, by means of which reference data are determined, which are used to compensate for the position of the machine when determining the position of the movable machine part. For example, if the machine itself moves from a horizontal orientation to the gravitational vector into a different orientation to the gravitational vector, for example due to uneven terrain, this can be determined using the position sensor, which is arranged on the machine itself and is fixed. In other words, the deviation of the alignment of the machine itself from the gravitational vector can be determined using the position sensor on the machine. This information is used to correct the data from the position sensors on the moving machine part, thereby reducing errors and increasing the overall position determination of the moving machine part. According to a further embodiment, the information from the position sensor or sensors can be used to predict the future position of the reference objects on the basis of the movement of the movable machine part and/or the machine. As a result, the continuous position determination of the reference objects can be further advantageously increased.

• Der erste Abschnitt betrifft die Initialisierung. Es wird angenommen, dass sich die Maschine in Ruhe befindet, also werden Geschwindigkeiten und Beschleunigungen auf Null gesetzt. Die Position und Lage wird den aus der Initialisierungsroutine gewonnenen Werten gleichgesetzt.
• Der zweite Abschnitt umfasst eine Prädiktion. In diesem Schritt werden die vorhergesagten Zustände aus dem Kalman Filter zu einem bestimmten Zeitpunkt errechnet. Dieser Zeitpunkt kann sowohl in Zukunft, Gegenwart oder Vergangenheit liegen. Hierbei ist zu beachten, dass sich die Genauigkeit mit steigendem zeitlichem Abstand zur Gegenwart vermindert.
• Der dritte Abschnitt umfasst eine Korrektur: Liegt ein Messwert aus dem Messsystem vor, so werden die Zustände und die Kovarianzmatrix um den Messwert korrigiert. Neben dem Messwert selbst wird hierbei auch der Zeitstempels des Messwertes verwendet, um die Dynamik des Systems besser beschreiben zu können.

According to one embodiment, the direction data and/or the reference data of the position sensors of the minimization function are provided, whereby the adaptation of the virtual model of the machine is carried out by means of a sensor data fusion of the data from the position sensors and the new sensor field of view position data of the optical sensors. According to one embodiment, the sensor data fusion consists of a Kalman filter. A Kalman filter is used to calculate a measurement result from various measurement values in which the variance of the individual measurement values is minimized, i.e. optimal. It is irrelevant here whether the measured values are positions of the field of view, angular velocities of the position sensors or accelerations of the position sensors. This data is linked to the model equations that have already been set up and the individual variances are connected to one another via the so-called covariance matrix. The form of the covariance matrix itself, in turn, depends on the model equations. States of the Kalman filter used include the 3D position and 3D rotation of the virtual model, as well as their first and second derivatives, as (angular) velocity and (angular) acceleration. The status of the Kalmans filter can be divided into 3 sections, with sections 2 and 3 repeating themselves continuously:

• The first section concerns the initialization. The machine is assumed to be at rest, so velocities and accelerations are set to zero. The position and position are set equal to the values obtained from the initialization routine.
• The second section includes a prediction. In this step, the predicted states from the Kalman filter are calculated at a specific point in time. This point in time can be in the future, present or past. It should be noted here that the accuracy decreases as the time between the present and the present increases.
• The third section includes a correction: if there is a measured value from the measuring system, the states and the covariance matrix are corrected by the measured value. In addition to the measured value itself, the time stamp of the measured value is also used in order to be able to better describe the dynamics of the system.

According to one embodiment, when the reference objects are activated in the subsequent time intervals, a selection is made from the available reference objects, with the selection being based on the position of the reference objects known from the previous position determination and/or the speed of the reference objects and/or the acceleration of the reference objects and/or a geometric structure of the movable machine part and/or a geometric structure of the machine is taken into account. As already explained, the initial position of the reference objects is determined during the initialization step, ie their initial sensor field of view position of the individual optical sensors is known. Accordingly, it is known which reference objects can be detected by which optical sensors, ie are available. The accuracy of the position determination of the movable machine part can be increased by using reference objects that are clearly visible, for example, and are advantageously positioned in the overall view of the position of the movable machine part.

If, for example, the moving machine part is a long object and three reference objects are attached close to the optical sensors and one reference object is attached far away from the optical sensors, it is not expedient to only activate and evaluate the three close ones, since the measured angles between these three reference objects are small compared to the angle that the front reference object subtends together with the rear objects. So it depends on the combination of the activated reference objects. Large distances between the reference objects are always advantageous, as this results in large angles. In addition, the arrangement of the sensor field of view of the respective optical sensor is relevant. If an optical sensor is arranged in such a way that two reference objects are in a line at a large distance from each other, the measurable angle is very small and consequently the measurement of this sensor is disadvantageous. If the same measurement is carried out on an optical sensor which, for example, looks at the two reference objects rotated by approx. 90 degrees, the angle here is advantageously large. The same applies if two optical sensors are used. In this example, however, the angles are not the only decisive factor, since they can refer to two separate coordinate systems. Reference objects that are close to the sensors are generally better, but are not solely decisive for the value of the measurement. According to one embodiment, optimization is based on a weighted combination of the current parameters that depends on the structure of the measuring system, such as a distance matrix (distance of optical sensor N to reference objects M), a field of view matrix (estimated field of view position of optical sensor N to reference objects M, a field of view position in the center of the recording, the center of the image is preferred , since it is easier to measure here), an alignment matrix (alignment of the optical sensor N to reference objects M, since the optical sensor has directional sensitivity) and/or information from a measurement history (sources of interference, nearby overlaps, dirt, disturbing reflections.)

According to one embodiment, the measurement history is made up of a position known from the previous position determination and/or a speed of the reference objects and/or an acceleration of the reference objects and/or a geometry of the movable machine part and/or a geometry of the machine. The current speed of the reference objects and the current acceleration of the reference objects can be determined by means of the change in position of the reference objects. According to this embodiment, these values can be used to predetermine the future position of the reference objects by appropriately controlling the activation of the reference objects and the associated selection of the optical sensors for recording the recordings. If the change in position of the reference objects does not behave as predetermined, for example due to a different control of the movement of the movable machine part, the new current speed and new current acceleration can be re-determined and taken into account again in the subsequent position determination. The current direction of movement of the reference objects can also be taken into account. The geometry of the machine and the geometry of the moving machine part are known and help determine when which of the reference objects is in which sensor field of view of the optical sensors. If, for example, the moving machine part is moved, it is conceivable that one of the reference objects is out of the field of view of one of the optical sensors is turned or is covered by the moving machine part or by the machine. Since the geometry is known, it can be taken into account when activating the reference objects or when selecting the optical sensors, which means that faulty recordings or faulty assignment can be avoided, which in turn means that the position determination can advantageously be carried out precisely and robustly can. In addition, reflections from the reference objects on the moving machine part, which can interfere with the optical sensors, can be reliably detected and removed, which has an advantageous effect on the accuracy and robustness of the position determination.

According to one embodiment, a weighting factor is calculated between at least one of the reference objects or between at least one of the radiation transmitters and at least one of the optical sensors, which weighting factor is used to determine the position of the movable machine part. According to a further embodiment, it is conceivable that a weighting factor is calculated between each of the optical sensors and each of the reference objects or each of the radiation transmitters that are in the field of view of the optical sensors, which weighting factor is used to determine the position of the movable machine part. According to this embodiment, the weighting factor indicates how highly the results of the corresponding field of view position determination should be weighted in comparison to the results of other field of view position determinations. The accuracy and robustness of the position determination of the movable machine part can also advantageously be increased by means of the weighting factor.

According to one embodiment, each of the reference objects has at least one radiation emitter and the activation of the respective reference objects consists in activating the radiation emitter. According to one embodiment, the radiation transmitter is designed to emit radiation in the form of electromagnetic waves. According to this embodiment, the activation causes the electromagnetic waves to be emitted for a specific time. In the recordings of the optical sensors, the radiation can advantageously be seen well, whereupon the position of the field of view of the individual reference objects can advantageously be well determined. The position determination can accordingly be carried out very precisely.

According to a further embodiment, the at least one radiation emitter is an LED radiation emitter. Such LED radiation emitters are comparatively inexpensive and can advantageously be easily detected by means of the optical sensors.

According to one embodiment, each of the reference objects has multiple sides. In addition, one of the radiation transmitters is arranged on at least two of the sides. In addition, according to this embodiment, when activating the reference objects, a distinction can be made as to which of the radiation transmitters is switched on for each reference object. If, according to one embodiment, the reference objects are cuboid, for example, then they have six sides. On one of these sides they are arranged on the moving machine part. One of the radiation transmitters can be arranged on each of the other five sides. For example, the side of one of the reference objects that is visible to the optical sensors can change as a result of the movement of the movable machine part. Due to the fact that radiation transmitters are arranged on several sides of the corresponding reference object according to this embodiment, the reference object remains visible for the optical sensors by changing the activation of the now visible radiation transmitter of the reference object. As a result, the position of the reference object can continue to be advantageously determined despite the movement of the movable machine part. As a result, the accuracy of the position determination of the movable machine part can also be advantageously increased. According to another embodiment, it is conceivable that the reference object is configured in the shape of a pyramid or some other shape with a plurality of sides.

According to a further aspect of the present disclosure, a measuring device for determining the position of a movable machine part of a machine is specified. A plurality of optical sensors are arranged on the machine, each of which has a fixed sensor field of view in relation to their arrangement on the machine and a plurality of reference objects are arranged on the movable machine part and each of the reference objects is fixed at a fixed reference position on the machine part , wherein in each case the sensor fields of view of the optical sensors are aligned in the direction of the reference objects and wherein the measuring device has a computing unit which is designed to carry out one of the above-mentioned methods. According to one embodiment, the processing unit has a real-time capable operating system, which means that a constant update rate can be achieved. The computing unit controls the reference objects or the respective radiation transmitters and the optical sensors, and thereby determines which reference objects are activated when and with which optical sensors or with which optical sensors the activated reference object (the activated radiation transmitter) is detected. For this purpose, according to one embodiment a real-time capable fieldbus (e.g. an EtherCAT protocol) can be used.

In accordance with one embodiment, the arithmetic unit has a plurality of distributed sub-units which are set up to communicate with one another and to exchange data, the arithmetic unit being set up to work with a pipeline in order to optimally utilize the available computing capacity. According to this pipeline-based embodiment, several calculation steps and recording processes of the optical sensors can be carried out in parallel. According to one embodiment, a new recording can be evaluated with a sub-unit (sub-computing unit) arranged directly on the optical sensor. During the evaluation, the next recording can already be made with the optical sensor for the next time step. The sensor field of view positions of the individual reference objects in the recordings can accordingly be determined in parallel. The calculation and the optimization of the activation of the individual reference objects and the control of the taking of pictures by the optical sensors can accordingly be carried out using the results of the preceding evaluations. According to one embodiment, the processing unit is a central processing unit that combines the results of the individual sub-processing units and calculates the position and optimizes it, and on which the real-time operating system runs. According to a further embodiment, the device can also only have a single computing unit that executes all tasks. With the pipeline method, all available resources can be used in parallel, so that the "downtimes" of resources are reduced, which means that the entire method can be carried out quickly and efficiently. A potential disadvantage of a dead time of one time step can be compensated for by a higher update rate of the measurement using the pipeline method.

According to one embodiment, the optical sensors are infrared cameras. According to a further embodiment, the optical sensors have a CMOS sensor, a lens and an infrared bandpass filter as infrared cameras. According to a further embodiment, the optical sensors are installed in a waterproof housing so that they can also be used in rain or wet conditions. According to one embodiment, the reference objects or the radiation transmitters are designed to emit infrared radiation that can be detected by the infrared cameras. By means of the infrared radiation and the infrared cameras, the position of the movable machine part can advantageously be determined precisely.

Exemplary embodiments and developments of the method according to the present disclosure are shown in the figures and are explained in more detail on the basis of the following description.

• 1 eine schematische Darstellung einer Maschine mit beweglichen Maschinenteilen und einer Messvorrichtung gemäß einer ersten Ausführungsform,
• 2 eine weitere schematische Darstellung der Maschine mit den beweglichen Maschinenteilen und der Messvorrichtung gemäß der ersten Ausführungsform,
• 3 eine schematische Darstellung der Maschine mit den beweglichen Maschinenteilen und einer Messvorrichtung gemäß einer zweiten Ausführungsform,
• 4 eine schematische Darstellung des virtuellen Modells der Maschine gemäß einer ersten Ausführungsform,
• 5 eine schematische Darstellung von Seitenansichten eines Referenzobjekts gemäß einer ersten Ausführungsform,
• 6 eine schematische Darstellung einer räumlichen Ansicht des Referenzobjekts gemäß der ersten Ausführungsform.

Show it:

• 1 a schematic representation of a machine with moving machine parts and a measuring device according to a first embodiment,
• 2 a further schematic representation of the machine with the moving machine parts and the measuring device according to the first embodiment,
• 3 a schematic representation of the machine with the moving machine parts and a measuring device according to a second embodiment,
• 4 a schematic representation of the virtual model of the machine according to a first embodiment,
• 5 a schematic representation of side views of a reference object according to a first embodiment,
• 6 a schematic representation of a spatial view of the reference object according to the first embodiment.

The 1 shows a schematic representation of a machine 10 as a construction machine 12. The in 1 The construction machine 12 shown is a drilling rig with which drilling can be carried out. In this regard, the construction machine 12 has a plurality of movable machine parts 20 . The moving machine parts 20 are movable in relation to the construction machine 12 . Arranged on the movable machine parts 20 are support arms 22 on which, according to this embodiment, a drilling device for carrying out the bores is arranged in each case. A measuring device 100 for determining the position of the movable machine parts 20 is additionally arranged on the construction machine 12 . Measuring device 100 has a computing unit 110 , a sensor arrangement 120 and a plurality of reference objects 130 . According to this embodiment, the sensor arrangement 120 is arranged on a region of the construction machine 12 which cannot be moved in relation to the construction machine 12 itself.

Accordingly, the sensor arrangement 120 is fixedly arranged on the construction machine 12 . According to this embodiment, the sensor arrangement 120 has tion form several optical sensors 122, whose sensor field of view 126 (not in 1 shown) is aligned in the direction of the reference objects 130 or in the direction of the moving machine parts 20. The optical sensors 122 are arranged relatively far apart from each other, so that the entirety of all sensor fields of view 126 cover the entire range of motion of the moving machine parts 20 as completely as possible. In addition, as a result, the individual sensor fields of view 126 have a larger angle to one another, as a result of which the position of the movable machine parts 20 can be advantageously determined. According to this embodiment, some of the optical sensors are arranged on a mounting bracket 140; a direct arrangement on the machine 10 itself is also conceivable.

According to this embodiment, the reference objects 130 are arranged at a plurality of points on the movable machine parts 20 . In the present case, the reference objects 130 are also arranged on the support arms 22 of the moving machine parts 20 . The reference objects are arranged on the moving machine parts 20 in such a way that they are advantageously visible in the respective sensor fields of view 126 of the optical sensors 122 . Due to a movement of the moving machine parts 20, it is conceivable that initially visible reference objects 130 are no longer visible and initially invisible reference objects 130 become visible. However, the reference objects 130 per movable machine part 20 are arranged in combination with the optical sensors 122 in such a way that, if possible, for every possible position of the movable machine parts 20, the number of reference objects 130 required for precise position determination is visible to the sensor arrangement 120.

According to this embodiment, the arithmetic unit 110 has a plurality of arithmetic sub-units 112 which can carry out parts of the calculation for determining the position of the movable machine parts 20 . The arithmetic unit 110 additionally has a central arithmetic unit, which combines the results provided by the arithmetic sub-units 112 for determining the position of the moving machine parts 20 .

The computing unit 110 is accordingly designed to carry out the method according to the present disclosure, as a result of which the position of the moving machine parts 20 is determined.

The 1 also shows position sensors 150, one of the position sensors 150 being arranged on one of the moving machine parts 20 and the other of the position sensors 150 being arranged on the machine 10 on the driver's cab in a fixed manner. According to one embodiment, the position sensors 150 are an acceleration sensor and/or a gyroscope. The position sensors 150 can also be installed within one of the reference objects 130, which simplifies the wiring. A vertical direction, a direction to the gravitational vector, can be determined by means of the position sensors 150 . Accordingly, the pitch and roll angles of the movable machine parts 20 can be determined directly by means of the position sensors 150 and inserted into the position determination of the movable machine part 20 .

The one in the 2 The schematic representation of the machine 10 shown differs from the schematic representation of the machine 10 according to FIG 1 in that the sensor field of view 126 of the optical sensors 122 is shown schematically in each case. In the 2 Accordingly, it can be seen that the entire possible range of movement of the moving machine parts 20 is covered by the sensor fields of view 126 of the optical sensors 122. The optical sensors 122 are additionally arranged in such a way that there is as little overlapping of the reference objects 130 by other moving machine parts 20 (eg by hoses) in the sensor fields of view 126 as possible.

The one in the 3 The schematic representation of the machine 10 shown differs from the schematic representation of the machine 10 according to FIG 1 in that the sensor arrangement 120 of the measuring device 100 according to this embodiment additionally has an optical sensor 122 which is arranged on one of the moving machine parts 20 . The variable sensor field of view 126 of such an optical sensor 122 varies with the movement of the movable machine part 20. However, this movement / position of this movable machine part 20 can be determined by means of other optical sensors 122 and the reference objects 130, which are arranged on the movable machine part 20, so that it is also possible to determine the position of the reference objects 130 that were recorded with the movable optical sensor 122 . Accordingly, according to this embodiment, a larger range of movement of the movable machine parts 20 can be covered overall and the position determination can also be carried out more robustly and more precisely.

The 4 shows schematically a virtual model of the machine 10'. The virtual model of the machine 10' completely maps the real machine 10. In particular, the virtual model has the same components with the same degrees of freedom, so that all 10 possible from the real machine Movements can also be mapped using the virtual model of the machine 10'. In addition, the virtual model of the machine 10' also has the virtual optical sensors 122' with their virtual sensor fields of view 126', the virtual optical sensors 122' being arranged and aligned at the same positions on the virtual model as the optical sensors 122 on the real one Machine 10. In addition, the virtual model of the machine 10' also has virtual reference objects 130', which are also arranged at the same position as the real reference objects 130. By means of the recordings of the optical sensors 122, in which the respective reference objects 130 can be assigned then the position and orientation of the movable machine part 20 are determined with the greatest probability by means of a minimization function. The virtual model of the machine 10′ can accordingly be aligned and adjusted accordingly, as a result of which the actual position of the movable machine part 20 can in turn be inferred or output.

The 5 and 6 show a reference object 130 according to a first embodiment. The illustrated reference object 130 is cube-shaped and accordingly has six reference object sides 132 . In the 5 and 6 only the reference object housing 131 is shown without further components. The reference object 130 shown has openings for light-emitting diodes 134 . According to this embodiment, light-emitting diodes 134 can be inserted in the reference object housing 131 on four sides 132 of the reference object. A connection opening 136 is provided on one of the reference object sides 132, by means of which the reference object 130 can be wired. The reference object 130 shown is arranged on the moving machine parts 20 . The light-emitting diodes 134 integrated in the reference object 130 are activated, thereby emitting infrared radiation and the optical sensors 122 record this infrared radiation by means of recordings. These recordings are then evaluated to determine the sensor field of view position of the respective reference objects 130 in accordance with the present disclosure.

A further embodiment for the reference object 130 is an LED in a metal housing with a thread on the back (for simple and space-saving attachment to one of the moving machine parts 20). A pane of glass is used for sealing, which may also have a color filter for infrared light. The color filter can be used for all embodiments. The LED can be sunk flush in the movable machine part 20 by means of a blind hole, whereby protruding add-on elements can be avoided and damage to the reference objects 130 in the event of collisions of the movable machine parts 20 can be prevented. If, according to a further embodiment, LEDs are required in several directions, corresponding attachment adapters can be built.

A further embodiment provides an additional diffuser layer on the glass pane. As a result, the point light source of the LED is expanded and its field of view position can be better measured by the optical sensors. This is particularly relevant when the optical sensors are viewed from the LEDs at an angle, making the measurement more robust and accurate.

Claims (10)

Method for determining the position of a movable machine part (20) of a machine (10), a plurality of optical sensors (122) being arranged on the machine (10), each of which has a fixed sensor field of view (126) in relation to its arrangement on the machine (10), and a plurality of reference objects (130) are arranged on the movable machine part (20) and each individual reference object (130) is fixed at a fixed reference position on the machine part (20), the respective sensor fields of view (126) of the optical sensors (122) are aligned in the direction of the reference objects (130), the method having the following steps: - providing a virtual model of the machine (10'), the virtual optical sensors (122') with virtual sensor fields of view (126') and virtual reference objects (130'), wherein the orientation of the virtual optical sensors (122') corresponds to that of the optical sensors (122) and the positioning of the virtual reference objects (130') corresponds to that of the reference objects (130); - In an initialization step (t0): - Determining an initial sensor field of view position of the individual reference objects (130) in the respective sensor fields of view (126) of the respective optical sensors (122) and adapting the virtual model of the machine (10 ') to the initial sensor field of view positions of the individual reference objects (130); - in a first time interval (t1) following the initialization step: - activating the reference objects (130) and taking pictures by the optical sensors (122); - determining the new sensor field of view position of the individual activated reference objects (130) in the individual recordings of the optical sensors (122); - Adjusting the virtual model of the machine (10 ') to the new sensor field of view positions of the individual reference objects (130) by a mini mization function is running; - Determining the position of the movable machine part (20) by reading the virtual position of the virtual machine part (20') from the virtual model, wherein the determination of the initial sensor field of view position of the individual reference objects (130) has the following steps: - Activating one of the reference objects ( 130) and taking pictures by all optical sensors (122) while the reference object (130) is activated; - Evaluating the respective recordings of the optical sensors (122) in which the reference object (130) is activated, whereby the initial sensor field of view position of the reference object (130) is determined; - Repeating the aforementioned method steps with all further reference objects (130), whereby all initial sensor field of view positions of the reference objects (130) are determined. procedure according to claim 1 , In addition to the recordings in which the respective reference object (130) is activated, a further black recording is recorded with each of the optical sensors (122) and is compared with the recordings in which the respective reference object (130) is activated, whereby environmental influences are filtered out become. Method according to one of the preceding claims, wherein a linear system of equations is first solved by means of the minimization function in order to determine an optimal starting point for a subsequent numerical optimization, with the numerical optimization then determining the global minimum or the global maximum starting from the starting point, whereby the virtual model of the machine (10') is adapted using the new sensor field of view positions of the individual reference objects (130). Method according to one of the preceding claims, wherein at least one position sensor (150) is arranged on the movable machine part (20), the position sensor (150) being an acceleration sensor and/or a gyroscope, with the position sensor (150) determining direction data for the gravitational vector and these directional data are also used to determine the position of the movable machine part (20). procedure according to claim 4 At least one position sensor (150) is additionally arranged on the machine (10), by means of which reference data are determined which are used to compensate for the position of the machine (10) when determining the position of the movable machine part (20). Method according to one of Claims 4 or 5 , wherein the direction data and/or the reference data of the position sensors (150) are provided to the minimization function, whereby the adaptation of the virtual model of the machine (10 ') is carried out. Method according to one of the preceding claims, wherein when the reference objects (130) are activated in the subsequent time intervals (t1, tn), a selection of the available reference objects (130) is made, with the selection being based on the position of the position known from the previous position determination Reference objects (130) and/or speed of the reference objects (130) and/or acceleration of the reference objects (130) and/or a geometric structure of the movable machine part (20) and/or a geometric structure of the machine (10) is taken into account. Measuring device (100) for determining the position of a movable machine part (20) of a machine (10), a plurality of optical sensors (122) being arranged on the machine (100), each of which has a fixed sensor field of view (126) in relation to its arrangement on the machine (10) and a plurality of reference objects (130) are arranged on the movable machine part (20) and each one of the reference objects (130) is fixed at a fixed reference position on the machine part (20), the respective sensor fields of view ( 126) of the optical sensors (122) are aligned in the direction of the reference objects (130), and wherein the measuring device (100) additionally has a computing unit (110) which is designed to carry out a method according to one of the preceding claims. Measuring device according to claim 8 , wherein the processing unit (110) has a plurality of distributed sub-units (112) which are set up to communicate with one another and to exchange data, the processing unit (110) and optical sensors (122) being set up to work with a pipeline in order to optimal use of the available computing capacity. Measuring device (100) according to one of Claims 8 or 9 , wherein the optical sensors (122) are infrared cameras and each of the reference objects (130) has a plurality of sides (132) and a radiation transmitter (134) is arranged on at least two of the sides (132) and when the reference object (130) is activated under it can be distinguished which of the radiation transmitters (134) is activated for each reference object (130).

Images