EP2193331A1

EP2193331A1 - Method and sensor arrangement for measuring optical features

Info

Publication number: EP2193331A1
Application number: EP08804728A
Authority: EP
Inventors: Wendelin Feiten; Cäsar KLIMOWICZ
Original assignee: Siemens AG
Current assignee: Siemens AG
Priority date: 2007-09-27
Filing date: 2008-09-25
Publication date: 2010-06-09
Also published as: DE102007046288B4; DE102007046288A1; WO2009043788A1

Abstract

The remission values of a laser scanner are used to detect features that can only be optically detected, such as lane markings (13) for lanes on the ground beneath a loading crane facility (10). This makes it possible for a sensor arrangement (11) to be installed on a loading crane with little effort and little interference of the ongoing operation, because the calibration of the arrangement is simplified and the effort for a manual measurement of the lanes relative to the sensor coordinate system is dispensed with.

Description

description

Method and sensor arrangement for measuring optical characteristics

A sensor assembly, for example mounted on a crane, is used to measure (or estimate) the position and attitude of moving objects, such as the crane itself or a cargo, e.g. a container. Other uses include the measurement of the position and position of a vehicle or a movable component of the crane itself into consideration.

The crane may be, for example, a loading crane. Loading cranes are used on cargo handling sites, warehouses, in assembly halls and shipyards as well as in track construction. In a loading crane for motor vehicles, the floor is inclined relative to the loading crane, so that water can drain. Furthermore, tracks for trucks are marked on the ground under the loading crane.

An execution of a loading crane is a gantry crane. This spans a loading and working area like a portal. As a rule, its side walls run with wheels on two parallel rails. On the crane bridge, the horizontal part of the gantry crane, a trolley moves with a hoist.

Alternatively, a rail slewing crane can be mounted on the crane bridge.

Furthermore, a loading crane, a gantry crane, a gantry crane, a gantry crane and a gantry crane are also suitable.

Moving parts of a crane are z. For example, the trolley or the spreader - a hoist with which containers can be grabbed.

In the context of a loading crane, sensor array measurements serve as a basis to signal truck drivers where to stop them. Furthermore, due to such measurements, the crane itself can be controlled.

The sensor arrangement may for example be composed of one or more of the following elements: a

3D laser scanner, a tiltable 2D laser scanner or a video camera. The elements of the sensor arrangement are usually mounted in such a way in the structure of the crane that - in the case of a gantry crane - several tracks for trucks or railroad railroad tracks are covered.

In order for the sensor assembly to be used in the manner described, it is necessary to calibrate it. This means that the following coordinate systems must be related to each other:

The sensor coordinate system of one or more laser scanners that are installed in the sensor arrangement,

the camera coordinate system of one or more cameras, provided that they are installed in the frame of the sensor arrangement,

the crane coordinate system of the crane relating to e.g. Trolley and spreader are located,

- The ground coordinate system, with respect to which, if necessary, tracks for trucks or railroad railroad tracks are given, which are loaded and unloaded.

Calibration is achieved in the prior art, for example, by placing a calibrating body specially prepared and prepared for this purpose on the ground in the area of the crane and measuring it manually with respect to the crane or the crane coordinate system by a surveying engineer. In addition, lanes can be measured manually in relation to the crane or to the calibration body. The sensor arrangement subsequently detects the calibration element, from which coordinate transformations between the sensor coordinate system and the other coordinate systems can be obtained. The disadvantage here is that the ongoing operation of the crane for calibration longer time must be interrupted.

In the context of the automation of the loading crane system, a specific calibration of the sensor arrangement for the respective workflows is desirable. Frequently structuring features are found in the surroundings of the loading crane system, which can only be detected optically. An example of this is ground markings painted on the ground indicating the location of the lanes. The inclusion of an optical sensor (such as a video camera) in the sensor arrangement, however, has the disadvantage that then the camera coordinate system of the optical sensor must be matched with the sensor coordinate system of the laser scanner.

This is made more difficult in the applications described in large crane systems by their large dimensions and non-standard visual characteristics. For example, the lane markings may be different. For this reason, it is necessary in the prior art to introduce further calibration bodies which are precisely defined in terms of their optical and geometrical features into the system in order to carry out the calibration on the basis of these. A manual measurement of the lane markings here means additional effort.

It is therefore the task of specifying a method and a sensor arrangement for measuring optical features, which simplify the construction of the sensor arrangement or the measurement of the optical features.

This object is achieved by the method and the sensor arrangement for measuring optical features according to the independent patent claims. Preferred developments emerge from the dependent claims.

In the method for measuring optical features with a sensor arrangement, this comprises a laser scanner. The laser Scanner determines remission values. Subsequently, the optical features are extracted from the remission values.

The sensor arrangement for measuring optical features comprises a laser scanner, which is set up to determine remission values. Furthermore, the sensor arrangement comprises a computing unit, which is set up to extract the optical features from the remission values.

Today's laser scanners provide, in addition to distance values, also measurements of the reflected-back energy, depending on the operating mode, to a certain extent a gray value. The gray value is called the remission value. The method and the sensor arrangement make it possible, in addition to the distance values, which are determined by the sensor arrangement, to also use the remission values, which are determined by the laser scanner contained in the sensor arrangement. This makes it possible to dispense with the installation of a video camera. A further advantage is that the extracted optical features are present directly in the sensor coordinate system of the sensor arrangement.

The method and the sensor arrangement use the remission values of the laser scanner to produce a complete gray value image of a section of an environment. Such a grayscale image is in principle comparable to the image of a video camera. Furthermore, optical features are recognizable in the gray scale image. Therefore, the extraction of the optical features from the remission values can be done with classical methods of image processing.

If the optical features are track markers, they are determined directly in the sensor coordinate system of the sensor arrangement. This eliminates manual surveys of the lanes as well as a determination of a coordinate transformation between a ground coordinate system and the sensor coordinate system. Rather, the coordinate transformations between the coordinate systems used are automatically determined without manual measurement. This is more accurate and reduces the effort of installing and connecting the sensor assembly.

By extracting the optical features from the remission values, there is the advantage that separate calibration bodies for measuring the optical features can be dispensed with. This eliminates the hassle of creating, interim storage, placing and clearing the Kalibrierkorper. This saves costs. The connection of the sensor arrangement thus requires less effort and causes less disturbance of the current operation.

In a development of the method, the sensor arrangement is mounted on an object. Distance measurement data from the laser scanner identifies ground measurement data for parts of a floor underneath the object as well as object measurement data for parts of the object. The object measurement data are used to determine a ground coordinate system from the ground measurement data. A coordinate transformation between a sensor coordinate system of the sensor arrangement and the ground coordinate system is calculated, by means of which the sensor arrangement is calibrated.

Analogously, the sensor arrangement is set up in an embodiment for mounting on an object. Furthermore, it has an arithmetic unit, which is set up for the identification of soil measurement data for parts of a soil under the object as well as object measurement data for parts of the object in distance measurement data of the laser scanner. The arithmetic unit is further configured to determine a ground coordinate system from the ground measurement data using the object measurement data, to calculate a coordinate transformation between a sensor coordinate system of the sensor array and the ground coordinate system, and to calibrate the sensor array based on the coordinate transformation. The further developments offer the advantage that the complete calibration of the sensor arrangement with respect to its environment can be carried out without a separate calibration body.

According to a development, the object is a crane, in particular a loading crane, gantry crane, bridge crane, semi-portal crane, gantry crane or portal crane, or any movable or static object on which the sensor arrangement can be mounted. A gantry crane offers the advantage that its pronounced symmetry properties can be used.

According to a development, the sensor arrangement comprises one or more pivotable 2D laser scanners. The use of a tiltable 2D laser scanner offers the advantage of its large field of view that in addition to vehicles in the working area of the crane, large parts of the supporting structure of the crane itself can be detected.

According to a further development, the parts of the object are side walls of a gantry crane. This offers the advantage that the orientation of these side walls can be used to determine the object coordinate system.

In the following, exemplary embodiments of the invention will be explained in more detail with reference to figures. Show it:

1 crane with a sensor arrangement and a cargo under the crane,

FIG. 2 shows distance measurement data of the sensor arrangement, FIG. 3 shows a crane coordinate system, a sensor coordinate system and a ground coordinate system,

Figure 4 is a flowchart for calibration.

Figure 1 shows a crane 10. On the crane 10, a sensor assembly 11 is mounted, which consists of two elements in the case shown in Figure 1. Also shown is a cargo 12, such as a container on a truck, which is detected by the sensor assembly 11. Also visible in Figure 1 are wheels 14, with which the crane 10 can be moved on rails. A floor 15 under the crane 10 is inclined, so that water can flow away. On the floor 15 lane markers 13 are attached, which mark tracks for vehicles.

FIG. 2 shows measurement data of the sensor arrangement 11, in this case distance measurement data of a laser scanner. In the measurement data, ground measurement data 21 of parts of the floor 15 under the crane 10 and crane measurement data 22 of parts of the crane 10 can be identified. This allows a geometric measurement of the crane and its working space.

The measurement data can, for example, by pivoting a

2D laser scanner over the parts of the crane 10 and the parts of the bottom 15 are obtained.

The crane measurement data 22 are here rectangles in 3D out-segmented side walls of the crane 10. The ground measurement data 21 are accordingly out-segmented ground points. Of these, only a subset may need to be used to achieve sufficient accuracy; This saves computing time and storage requirements.

FIG. 3 again shows the crane 10, its wheels 14, the sensor arrangement 11 and the floor 15 and the lane markings 13. A ground coordinate system 16, a crane coordinate system 17 and a sensor coordinate system 18 of the sensor arrangement 11 are additionally shown.

FIG. 4 shows a flowchart for calibrating the sensor arrangement 11. In a first step 1, the sensor arrangement 11 acquires measurement data of its surroundings as shown in FIG. These are distance measurement data. In a second step 2, the ground measurement data 21 shown in FIG. 2 and the crane measurement data 22 are identified in the measurement data. This can be computerized using Planned drawings are made, which are accurate enough for the mounting position of the sensor assembly 11 and its elements and other elements of the crane 10. The measurement data are given here as 3D measurement points, which are all initially present in the sensor coordinate system 18 of the sensor arrangement 11.

In a third step 3, the crane measurement data 22 are used to determine a ground coordinate system 16 from the ground measurement data 21. In a fourth step 4, a coordinate transformation between a sensor coordinate system 18 of the sensor arrangement and the ground coordinate system 16 is calculated, by means of which the sensor arrangement 11 is calibrated.

The method for calibrating the sensor arrangement 11 uses symmetries, such as flat symmetries or translation symmetries in three-dimensional space. Such symmetries, the crane 10 on - especially in the design as a gantry crane. In this case, the symmetries are extracted from the crane measurement data 22 and possibly the ground measurement data 21. This is based on the knowledge that a surface of 3D measurement points represented by the crane measurement data 22 has a normal vector which can be used as the y direction vector of the crane coordinate system 17. Furthermore, it is possible to make use of the fact that a surface of 3D measurement points represented by the ground measurement data 21 has a normal vector which can be selected as the z direction vector of the ground coordinate system 16. Surprisingly, the x-direction vector of the ground coordinate system 16 can be selected to be identical to the x-direction vector of the crane coordinate system 17. This x-direction vector is both parallel to the ground 15 and parallel to a surface represented by the crane measurement data 22. This becomes clear with the example of the gantry crane. As it travels on rails, they run parallel to the ground as well as parallel to the inner walls of the gantry crane. The x-direction vector of both the ground coordinate system 16 and the crane coordinate system 17 can thus be selected parallel to the rails. Thus is the x-direction vector is perpendicular both to the normal vector of the area of SD measurement points represented by the ground measurement data 21 and to the normal vector of the area of 3D measurement points represented by the crane measurement data 22. On the basis of this, these coordinate systems can be successively developed.

In the following, an algorithm suitable for this purpose is given by way of example. Any other algorithms are possible. In the alternative, the algorithm calculates the crane coordinate system 17, which is used for the derivation of the ground coordinate system 16.

In the scenario of a gantry crane in a harbor, 22 3D measurement points are identified as crane measurement data that belong to the sidewall (the so-called "sill bar"), on the one hand 3D measurement points belonging to the seaward side wall and, on the other hand, 3D measurement points that belong to the landside side wall. For gantry cranes on the mainland, any other characterization of the two sides of the gantry crane can be chosen.

A major axis transformation (described in: Jonathon Shlens: "A Tutorial on Principal Component Analysis", available on the Internet on July 2, 2007 at http://www.snl.salk.edu/~shlens/pub/notes/pca.pdf) ) on the 3D measurement points of the seaward side wall of the crane 10 and on the 3D measurement points of the land side wall of the crane 10 each allows the formation of a normal vector in the center of gravity of the respective 3D measurement points. From these two normal vectors, the y-direction vector of the crane coordinate system 17 is now formed by averaging. In this case, it is chosen as a convention that the y direction vector faces the sea side.

Likewise by main axis transformation, a normal vector is generated on the 3D measurement points in the ground measurement data 21 formed and selected as the z-direction vector of the ground coordinate system 16.

Subsequently, the x direction vector of both the ground coordinate system 16 and the crane coordinate system 17 is obtained from the cross product of the y direction vector of the crane coordinate system 17 and the z direction vector of the ground coordinate system 16.

In the next step, the z direction vector of the crane coordinate system is calculated from the cross product of the x direction vector and the y direction vector of the crane coordinate system 17. Accordingly, the y-direction vector of the ground coordinate system 16 results from the cross product of the z-direction vector and the x-direction vector of the ground coordinate system 16.

As the origin of the crane coordinate system 17, the center between the base points of the normal vectors formed from the crane measurement data 22 is first selected.

Subsequently, the 3D measurement points in the crane measurement data 22, which belong to the seaward side wall and the side wall on the landside, are transformed into the crane coordinate system 17. In the course of this, their expansion in the x-direction is also determined as minimum and maximum. Thereafter, the crane coordinate system 17 is shifted in its x-direction so that its origin lies in the middle between the determined minimum and maximum. Thus, the crane coordinate system 17 is completely determined.

As the origin of the ground coordinate system 16, the center of gravity of the 3D measurement points in the ground measurement data 21 is first determined. Subsequently, the z-axis of the crane coordinate system 17 is cut with the plane formed by the 3D measurement points in the ground measurement data 21. The intersection chosen as the origin of the ground coordinate system 16. Now lanes under the crane 10 can be manually measured between tracks on which the wheels 14 of the crane 10 run. The position of the center between the tracks relative to the sensor arrangement 11 results from the previously determined coordinate systems.

On the basis of the determined coordinate systems, a coordinate transformation to the sensor coordinate system 18 of the sensor arrangement 11 is calculated in each case. With this, the sensor assembly 11 is calibrated.

Extraction of optical features from remission values of the laser scanner:

For the extraction of optical features from remission values of the laser scanner, only remission values for 3D measurement points are further processed, which were previously identified as ground measurement data 21. These 3D measurement points are projected into the plane of the floor 15 determined by them.

An evaluation of their remission values now yields a two-dimensional gray value image of the ground 15. In this gray value image, optical features such as the lane markings 13 are determined by classical image processing methods.

In a variant, the coordinate values of the 3D measurement points, which are initially present as floating-point numbers, are mapped onto a suitable grid which, for example, is selected axis-parallel to the x-direction vector and y-direction vector of the ground coordinate system 16. This provides a rectangular discretized pixel image, the value at each pixel being formed by appropriate filtering of the remission values associated with that pixel. For this purpose, suitable filters are, for example, mean, median or similar filters.

Thus, eliminates the need for manual measurement of lanes for trucks under a loading crane. The lanes are automatically determined from the simultaneously determined Determined 3D distance readings and remission values of the laser scanner. The process becomes very robust and accurate. A disturbance of the ongoing operation of the loading crane system when connecting the sensor assembly 11 is largely avoided.

Alternatively, the optical features can only be projected onto the previously determined plane of the base 15 after their extraction from the remission values, in order to locate these three-dimensionally in the sensor coordinate system 18 of the sensor arrangement 11.

The same applies to any other areas of application of the sensor arrangement 11. The described exemplary embodiments can be implemented analogously to other scenarios.

Claims

claims

1. A method for measuring optical features with a sensor arrangement (11), - in which the sensor arrangement (11) comprises a laser scanner, which determines remission values at which the optical features are extracted from the remission values.

2. The method of claim 1, wherein the optical features track markers (13).

3. The method according to claim 2, wherein on the basis of the lane markings (13) lanes are determined.

4. The method according to claim 1,

in which the sensor arrangement (11) is mounted on an object,

in which ground measurement data (21) for parts of a floor (15) under the object and object measurement data for parts of the object are identified in distance measurement data of the laser scanner,

in which the object measurement data are used to determine a soil coordinate system (16) from the soil measurement data (21),

- In which a coordinate transformation between a sensor coordinate system (18) of the sensor arrangement (11) and the ground coordinate system (16) is calculated, with which the sensor arrangement (11) is calibrated.

5. The method according to claim 4,

in which a y-direction vector of an object coordinate system is determined on the basis of the object measurement data,

in which a z-direction vector of the ground coordinate system (16) is determined on the basis of the ground measurement data (21),

- In which an x-direction vector of both the ground coordinate system (16) and the object coordinate system based on a cross product of the y-direction vector of the object coordinate system and the z-direction vector of the ground coordinate system (16) is calculated,

in which a z-direction vector of the object coordinate system is calculated from a cross product of the x-direction vector and the y-direction vector of the object coordinate system,

in which a y-direction vector of the ground coordinate system

(16) is calculated from a cross product of the z-direction vector and the x-direction vector of the ground coordinate system (16).

6. The method of claim 5, wherein the y-direction vector of the object coordinate system is formed by main axis transformation or principal component analysis by a normal vector at the center of gravity of 3D measurement points in the object measurement data.

7. The method of claim 5 or 6, wherein the z-direction vector of the ground coordinate system (16) by Hauptachsen- transformation or principal component analysis by a normal vector in the center of gravity of 3D measurement points in the soil measurement data (21) is formed.

8. The method according to claim 5, in which 3D measurement points in the ground measurement data assigned to the remission values are mapped to a grid which is parallel to the x direction vector and the y direction vector of the ground coordinate system. 16), in which the remission values of the corresponding 3D measuring points are filtered for each raster element, in which the optical features are extracted from the filtered remission values using methods of image processing.

9. sensor arrangement (11) for measuring optical characteristics, with a laser scanner, set up for determining remission values, with a computing unit, set up to extract the optical features from the remission values.

10. Sensor arrangement (11) according to claim 9, - arranged for mounting on an object, with a computing unit, arranged for the identification of ground measurement data (21) for parts of a floor (15) under the object and object measurement data for parts of the object in distance measurement data of the laser scanner,

- set up to determine a ground coordinate system (16) from the ground measurement data (21) using the object measurement data, set up to calculate a coordinate transformation between a sensor coordinate system (18) of the sensor arrangement (11) and the floor Coordinate system (16) arranged to calibrate the sensor array (11) based on the coordinate transformation.

11. Sensor arrangement (11) according to claim 10, wherein the object is a crane (10), in particular a loading crane, gantry crane, bridge crane, half gantry crane, gantry crane or portal crane.

12. Sensor arrangement (11) according to one of claims 9 to 11, wherein the laser scanner is a pivotable 2D laser scanner.