DE102022203011A1 - Adjustment of a detection of a rail track based on registration of a rail model with a camera image - Google Patents

Abstract

A method for determining an adjusted parameterized 3D rail model (M') is described. The method comprises acquiring image data (BD) from a surrounding area (UB) of a rail vehicle (1), which includes a rail track (ST), by an image recording unit (BA) arranged on the rail vehicle (1). An activation map (AK) is determined based on the captured image data (BD). Finally, the adjusted parameterized 3D rail model (M'), which reproduces the pose (P) of the rail track (ST) relative to the image recording unit (BA), is based on a comparison of a parameterized 3D rail model (M) with the determined activation map (AK ) determined. An adjustment device (40) is also described. A rail vehicle (1) is also described.

Description

The invention relates to a method for determining an adjusted, parameterized 3D rail model. In the method, image data from an area surrounding a rail vehicle is recorded by an image recording unit arranged on the rail vehicle. The surrounding area includes a rail track. The invention further relates to an adjustment device. The adjustment device comprises an input interface for receiving image data from an area surrounding a rail vehicle, which includes a rail track, from an image recording unit arranged on the rail vehicle. Furthermore, the invention relates to a rail vehicle.

Automated and driverless train systems are needed to detect objects along a railway line and to distinguish whether these objects are insignificant infrastructure assets next to the track or dangerous obstacles within the gauge. Additionally, located trackside assets can be integrated into a continuous mapping approach to retrieve updated maps of rail infrastructure. The use of cameras for such object detection and recognition tasks is widespread. Recently, various machine learning based applications have gained popularity with their compelling performance.

There are basically three different ways to recognize a rail track. A first possibility is to detect the rail track in the image sensor used for object recognition. This keeps the system independent of calibration (with respect to an auxiliary sensor) and prevents additional system-related errors from occurring. In image-based approaches, a rail path is detected using a variety of edge filtering methods or machine learning methods (e.g. panoptic segmentation with AI-based methods). Post-processing approaches are applied to improve the visualization and localization of the track path or even to determine the distance of objects on the track in relation to the camera. There are vanishing point or Hough transform methods that are limited to straight rails. Other methods use nonlinear functions to approximate curved rail shapes. The known track width is used to infer the distance of an object within the gauge if track detection is interrupted. In general, the methods described above do not determine the location of the tracks or the exact dimensions of the tracks. Line-like features are assumed to represent the tracks and the results are not checked for plausibility using a generative track model.

A second possibility involves adding an auxiliary camera or range sensor (lidar) to extract the tracks. Lidar-based methods filter the ground plane and the remaining lidar points are used to fit rail models. For example, so-called Markov chain Monte Carlo methods (abbreviated with the acronym MCMC) fit a rail profile model into the point cloud generated by the lidar and the successive matches are used to obtain the curved path of the track with a third-order interpolation .

A third option could be to expand with an inertial navigation system, also known as INS (INS is an acronym for “inertial navigation system”). The INS provides the pose of a sensor system within an accurate map. This means that the route of the track in front of the train can be deduced directly. However, the use of additional sensors is more error-prone and requires a sophisticated a priori calibration of the sensor system. An accurate map is a crucial requirement for INS-based procedures. Map creation is expensive and laborious and the map can quickly become outdated.

The task is therefore to provide reliable detection of a rail path with as little effort as possible in terms of the arrangement for detecting the environment and the computing effort.

This task is solved by a method for determining an adjusted parameterized 3D rail model according to claim 1, an adjustment device according to claim 12 and a rail vehicle according to claim 13.

In the method according to the invention for determining an adjusted, parameterized 3D rail model, an activation map is acquired on the basis of the image data mentioned at the beginning, acquired by an image recording unit of a rail vehicle from a rail track in the area surrounding the rail vehicle. In this context, image data should be understood as two-dimensional image data of the environment of the rail vehicle, which depicts the three-dimensional environment.

A parameterized 3D rail model is a rail model that indicates the pose of one or more rail strands relative to the image capture unit of the rail vehicle. The term pose is understood to mean the orientation and the relative position of the rail track and in particular a section of such a rail track relative to the image recording unit of the rail vehicle. The parameterized 3D rail model includes a set of parameters that enables a basic shape of a 3D rail model to be adapted to a real course of a rail track.

The environment of the rail vehicle includes in particular the areas which lie in the gauge profile of the rail vehicle and around it in the direction of travel and are therefore of particular interest for the journey of the rail vehicle.

An activation map includes a feature map in which features extracted from image data are displayed. In fact, the term activation card comes from AI (AI is an acronym for artificial intelligence). There, the output of a network is often referred to as activation, based on the neurons that activate specific outputs. Therefore, the output of a network can be called an activation card.

In connection with the invention, the above-mentioned features preferably include edge data or outline data of objects present in the image data. AI-based segmentation methods, also known as panoptic segmentation methods, can also be used to generate the activation map. A pixel with a predetermined apparent brightness, also referred to as intensity, preferably encodes a probability that a rail track runs at this point. Such a pixel is also referred to below as an activation point.

Furthermore, the adjusted parameterized 3D rail model, which reflects the pose of the rail track relative to the image recording unit, is determined by comparing a parameterized 3D rail model with the determined activation map. The parameterized 3D rail model is figuratively adapted to the activation map based on the comparison. For this purpose, the parameterized 3D rail model is first merged into a two-dimensional representation of the parameterized 3D rail model with the activation map. During the adjustment, both a model for the imaging behavior of the image recording unit and its pose relative to the image area as well as the parameterized 3D rail model are adjusted so that they can be brought into line with the activation map. This means that the structures shown in the activation map form a kind of default to which the two-dimensional representation of the parameterized 3D rail model is adapted.

Advantageously, only a single image recording unit is required to determine the pose of a rail track. By using a generalized 3D rail geometry model as a parameterized 3D rail model, the approach according to the invention can be generalized so that transition curves, such as clothoids, and also switches or several parallel rail tracks can also be detected or included.

Determining the adjusted parameters of the rail model and the rail pose makes it possible to semantically check the detection result of the rail track. If the parameters or rail pose deviate significantly from expected results, it is probably not rails.

The method according to the invention also copes with sparse segmentation results or ambiguous edge detection. It is robust against missing edge features and noisy activation maps.

With the method according to the invention, a rail model is adjusted with which the pose of a rail track can be precisely determined relative to a camera. Since a camera is usually mounted on a car body of a rail vehicle and the bogie of the rail vehicle and its car body are not firmly connected to each other, there is no constant extrinsic calibration of the camera with the rails. Therefore, the extrinsic camera orientation with respect to the rails needs to be updated. The result of the adjustment according to the invention can therefore also be used for other applications.

Such applications involve determining the orientation of the car body to the track using a high-precision map and GNSS (an acronym for global navigation satellite system) and INS (INS stands for “inertial navigation system”). Additional information from the camera could be helpful here.

The adjustment device according to the invention has, in addition to the input interface mentioned at the beginning, for receiving image data from a surrounding area of a rail vehicle also a determination unit for determining an activation card based on the captured image data.

Furthermore, the adjustment device according to the invention comprises an adjustment unit for determining an adjusted, parameterized 3D rail model, which reproduces the pose of the rail track relative to the image recording unit, by comparing the parameterized 3D rail model with the determined activation map.

The adjustment device according to the invention shares the advantages of the method according to the invention for determining an adjusted 3D rail model.

The rail vehicle according to the invention has an image recording unit for acquiring image data from a surrounding area of the rail vehicle, which includes a rail track.

Part of the rail vehicle according to the invention is also an adjustment device according to the invention for determining an adjusted, parameterized 3D rail model.

The rail vehicle according to the invention also has a control unit for controlling a journey of the rail vehicle based on the adjusted, parameterized 3D rail model. The control unit preferably includes the adjustment device according to the invention. The adjustment device can also be designed separately from the control unit and interact with it. The rail vehicle according to the invention shares the advantages of the adjustment device according to the invention.

A majority of the aforementioned components of the adjustment device can be implemented entirely or partially in the form of software modules in a processor of a corresponding computing system, for example by a control unit of a rail vehicle. A largely software-based implementation has the advantage that previously used computing systems can also be easily retrofitted by a software update in order to work in the manner according to the invention. In this respect, the task is also solved by a corresponding computer program product with a computer program which can be loaded directly into a computing system, with program sections to carry out the steps of the method according to the invention, at least the steps that can be carried out by a computer, in particular the steps for determining an activation card based on the captured image data and to determine the adjusted parameterized 3D rail model when the program is executed in the computing system. In addition to the computer program, such a computer program product may optionally contain additional components, such as: B. documentation and/or additional components, including hardware components, such as hardware keys (dongles, etc.) for using the software.

For transport to the computing system or to the control unit and/or for storage on or in the computing system or the control unit, a computer-readable medium, e.g. a memory stick, a hard drive or another transportable or permanently installed data carrier, on which the data that can be read by a computing system can be used and executable program sections of the computer program are stored. For this purpose, the computing system can, for example, have one or more cooperating microprocessors or the like.

The dependent claims and the following description each contain particularly advantageous refinements and developments of the invention. In particular, the claims of one claim category can also be developed analogously to the dependent claims of another claim category and their descriptive parts. In addition, within the scope of the invention, the various features of different exemplary embodiments and claims can also be combined to form new exemplary embodiments.

• i) das Projizieren eines durch Kandidaten-Parameterwerte parametrisierten 3D-Schienenmodells auf die Aktivierungskarte auf Basis einer Kandidaten-Projektionsvorschrift, und
• ii) das Ermitteln von angepassten Parameterwerten des parametrisierten 3D-Schienenmodells und/oder einer angepassten Projektionsvorschrift auf Basis des Vergleichs des projizierten parametrisierten Schienenmodells mit der Aktivierungskarte.

In a variant of the method according to the invention for determining an adjusted parameterized 3D rail model, determining the adjusted parameterized 3D rail model includes:

• i) projecting a 3D rail model parameterized by candidate parameter values onto the activation map based on a candidate projection rule, and
• ii) determining adapted parameter values of the parameterized 3D rail model and/or an adapted projection rule based on the comparison of the projected parameterized rail model with the activation map.

The projection achieves the fusion of the two-dimensional representation of the 3D rail model parameterized with candidate parameter values with the activation map. The candidate parameter values represent predetermined initial parameter values with which the 3D rail model is parameterized. The candidate parameter values preferably include the current parameter values of the 3D rail model or the Parameter values of the 3D rail model that were determined during the last adjustment and have since been used to detect the pose of a detected rail track. The adjusted parameter values and the adjusted projection rule can be used as adjusted parameter values and as an adjusted projection rule.

Preferably, determining an adjusted parameterized 3D rail model includes carrying out an iterative approximation method, wherein steps i) and ii) are repeated and, upon repetition, the adjusted parameter values determined in step ii) are used in step i) as candidate parameter values and/ or the adapted projection rule determined in step ii) is used in step i) as a candidate projection rule. Advantageously, the parameterized 3D rail model and its projection onto the image data can be adjusted step by step, with the precision of the adjusted parameter values and the adjusted projection rule being increased with each iteration.

• eine Abweichung des projizierten Schienenmodells von der Aktivierungskarte,
• eine Korrelation zwischen dem projizierten parametrisierten Schienenmodell mit der Aktivierungskarte.

The comparison preferably includes determining at least one of the following variables:

• a deviation of the projected rail model from the activation map,
• a correlation between the projected parameterized rail model with the activation map.

A deviation of the projected parameterized rail model from the activation map should be understood as a difference between activation values of activation points of the parameterized rail model projected onto the activation map and activation values of activation points of the activation map, which were image-based, i.e. generated on the basis of the captured image data. The difference between the individual activation points should be understood to mean in particular a difference between the activation values, preferably the intensity values, i.e. the apparent brightness, of the individual points of the projected, parameterized rail model and the points or activation points present at the same position of the image-based activation map. To put it bluntly, it is determined whether the rail model fits the structures identified as rails based on the image or covers them on the activation map.

The activation values of the individual points of the rail model are preferably set so that they are always 100% (or 1), while the activation values of the image-based activation points of the activation map, i.e. the activation points determined by segmentation or an edge filter, are less than the value 1 or maximum be the value 1. The individual points of the rail model can be viewed as unique and therefore assigned the value 1, while for the “rail points” of the activation map that are determined based on images, for example by an AI-based segmentation model, there is only a certain probability <= 1 that they are also part of the rails are. The difference is minimized by adjusting the rail parameters. This means that a parameter set is sought for the 3D rail model in which the sum of the differences between the activation values of the image-based activation points and the activation values of the model-based activation points present at the same position is minimal. The determined deviation can advantageously be used to adapt the 3D rail model to the actual course of the rails. The parameters or model parameters that are gradually adjusted can be included in the next iteration step of the adjustment process in order to reduce the deviation and thus gradually improve the result of the adjustment.

Alternatively, instead of the above-described minimization of the difference between the image-based activation points and the model-based activation points, the sum of the activation values of the image-based activation points and the activation values of the model-based activation points can also be maximized.

In particular, such a deviation should include a so-called “matching loss”. A loss function is a special function in mathematical statistics and part of a statistical decision problem. It assigns to each decision, in the form of a point estimate, a range estimate or a test, the damage caused by a decision that deviates from the true parameter.

The correlation represents a connection between the course of the projected parameterized rail model and structures in the activation map.

Preferably, the comparison includes determining a minimum of the deviation between the projected parameterized rail model and the activation map when they are figuratively superimposed. Ie, the parameter values of the parameterized rail model and/or the projection rule are determined in such a way that when they are used, the deviation of the parameterized rail model from Structures of the image-based activation map is minimal. An optimal adjustment result is advantageously achieved by determining the minimum.

The determined deviation preferably includes a deviation between apparent brightness values of the projected, parameterized rail model and the image-based activation points of the activation map at the same position in the activation map. The apparent brightness values indicate a probability that a rail track runs at a position in the activation map. The adaptation of the projected, parameterized rail model is advantageously carried out in such a way that it is positioned in the activation map where the probability of localizing a rail track is highest.

The deviation preferably has the sum of the squares of the differences between activation values of activation points of the projected parameterized rail model and values of activation points of the activation map at the same location. The intensity values of the projected rail model refer to intensity values of sampling points during a “virtual” scanning of the environment of a rail vehicle by a scanning sensor, such as a lidar system. The values of the activation points preferably also relate to intensity values or density values of the image-based activation points. The intensity values or density values indicate a probability derived from the object detection that a predefined object, in this case a rail track, runs at a position. When determining the deviation, the errors or deviations between the projected, parameterized rail model and activations of the activation map are advantageously added up, so that the deviation takes into account the entire course of the rail model in the activation map.

Particularly preferably, a combined activation map is generated on the basis of the projected, parameterized rail model and the image-based activation map by projecting the rail model onto the image-based activation map. The comparison is then made between activation values of model-based activation points and activation values of image-based activation points generated at the same location on the combined activation map.

An adjusted parameterized 3D rail model is preferably determined using a local nonlinear optimization method for calculating the parameters of the adjusted parameterized 3D rail model and the parameters of an exact projection of the adjusted parameterized 3D rail model onto the captured image data. With the help of such generally iterative approximation methods, a more precise parameterized 3D rail model and a more precise projection are determined in each step, which then forms a basis for the parameterized 3D rail model and the projection in the next step. Examples of optimization methods in this category are the downhill simplex algorithm, the Gauss-Newton method or the Levenberg-Marquardt algorithm.

The projection parameters include, among other things, the rotation, the translation or the focal length and the optical center, and therefore have a different scaling. For efficient and robust optimization, it is necessary to align the magnitudes of these parameters.

• - 3D-Punkte, welche entlang eines Schienenkopfes des Schienenstrangs mit einer vorbestimmten Abtastdichte angeordnet sind,
• - eine vorbestimmte parametrisierbare Grundform.

The parameterized 3D rail model preferably has at least one of the following features:

• - 3D points which are arranged along a rail head of the rail track with a predetermined sampling density,
• - a predetermined parameterizable basic form.

In other words, the parameterized 3D rail model generates 3D points along the rail head. These 3D points can then be projected onto the activation map for adjustment and, after being projected onto the activation map, matched to the activation points arranged therein.

If the 3D rail model includes a predetermined parameterizable basic shape, the adaptation of the 3D rail model to a real rail path is made possible by a suitable choice of parameter values.

• - eine Klothoide,
• - eine Verzweigung,
• - parallele Geraden.

The basic shape preferably comprises the shape of one of the following objects:

• - a clothoid,
• - a branch,
• - parallel straight lines.

A clothoid, also called an Euler spiral, can model a rail transition curve. A sampling density depends, for example, on the distance and the pose of a rail track relative to the image recording unit. In principle, you can choose the sampling density so that it corresponds to the distance becomes larger so that the perspective image is taken into account.

A basic shape of a branch can be used to model different types of branching, such as an intersection or a switch.

Parallel lines can be used to model straight rails.

The activation map is preferably determined by segmenting the image data. In other words, segmented image data is determined based on the captured image data and the activation map is determined based on the segmented image data. Segmentation involves dividing the image data into sub-areas, which are assigned to a classification. Segmentation may include panoptic segmentation. A panoptic segmentation classification can, for example, distinguish pixels of the image data that belong to people or objects in the foreground of the vehicle's surroundings from pixels that mark the background. Segmentation is preferably carried out using an AI-based segmentation method, with each point in the image data being assigned a probability value that this point is part of a rail track. This probability value can be used as an activation value in the comparison mentioned above.

The segmentation preferably includes edge detection in the image data. Edge detection can be achieved, for example, using a Sobel filter.

The activation map is preferably determined by smoothing the segmented image data. Smoothing, also known as blurring, is understood to mean a broadening of the distribution of the activation points over the activation map, preferably based on the application of a statistical function. The smoothing is preferably carried out by applying a Gaussian kernel to the segmented image data. Advantageously, such a more continuous distribution of activation points can be used more effectively for an approximation method, since the approximation can take place step by step from the periphery of such a distribution towards the center. In other words, the continuous distribution gives the approximation method an orientation in which direction the rail model can be adapted to the image-based activation points. If the distribution of image-based activation points is only discrete or line-like and limited to a fraction of the total area, problems can arise when calculating approximate values.

• - eine Bildkamera,
• - eine optische Bildkamera,
• - eine Infrarotkamera.

The image recording unit preferably comprises one of the following types of sensors:

• - an image camera,
• - an optical image camera,
• - an infrared camera.

The image recording unit preferably comprises an image recording unit stabilized by intertial sensors, preferably a MEMS-based image recording unit. MEMS-based image capture units include inertial sensors, i.e. acceleration and rotation sensors as well as micromechanical systems or components for stabilizing an image capture unit. They are used for situational awareness. If microelectronic components are used to control the pose of the image recording unit, a constant change in the pose of the image recording unit relative to the rail vehicle and also to the rail track results due to a constant compensating movement of the stabilizers. The adjustment according to the invention can advantageously compensate for the dynamic changes in the pose of the image recording unit.

• 1 ein Flussdiagramm, welches ein Verfahren zum Ermitteln eines adjustierten 3D-Schienenmodells gemäß einem Ausführungsbeispiel der Erfindung veranschaulicht,
• 2 ein Flussdiagramm, welches den in 1 veranschaulichten Schritt 1.II zum Ermitteln einer Aktivierungskarte im Detail veranschaulicht,
• 3 ein Flussdiagramm, welches den in 1 gezeigten Schritt 1.IV zum Ermitteln eines adjustierten parametrisierten 3D-Schienenmodells im Detail veranschaulicht,
• 4 eine schematische Darstellung einer Justage-Einrichtung gemäß einem Ausführungsbeispiel der Erfindung,
• 5 eine schematische Darstellung der Justage-Einheit der in 4 veranschaulichten Justage-Einrichtung gemäß einem Ausführungsbeispiel der Erfindung,
• 6 eine symbolhafte Darstellung des in 1 bis 3 veranschaulichten Verfahrens,
• 7 eine schematische Darstellung eines Schienenfahrzeugs gemäß einem Ausführungsbeispiel der Erfindung,
• 8 eine schematische Darstellung einer durch Glättung erzeugten bildbasierten Aktivierungskarte.

The invention is explained in more detail below with reference to the attached figures using exemplary embodiments. Show it:

• 1 a flowchart illustrating a method for determining an adjusted 3D rail model according to an exemplary embodiment of the invention,
• 2 a flowchart showing the in 1 illustrated step 1.II for determining an activation map illustrated in detail,
• 3 a flowchart showing the in 1 Step 1.IV shown for determining an adjusted parameterized 3D rail model is illustrated in detail,
• 4 a schematic representation of an adjustment device according to an exemplary embodiment of the invention,
• 5 a schematic representation of the adjustment unit in 4 illustrated adjustment device according to an embodiment of the invention,
• 6 a symbolic representation of the in 1 until 3 illustrated method,
• 7 a schematic representation of a rail vehicle according to an embodiment of the invention,
• 8th a schematic representation of an image-based activation map generated by smoothing.

In 1 is a flowchart 100 shown, which illustrates a method for determining an adjusted parameterized 3D rail model M 'according to an exemplary embodiment of the invention.

In step 1.I, image data BD from an surrounding area UB of a rail vehicle 1 (see 7 ) recorded. The image data BD includes a rail track ST or several rail tracks ST (see 6 ), which is or are located in the field of vision of an image recording unit BA of the rail vehicle 1.

In step 1.II, an activation card AK is determined based on the captured image data BD. Details on determining the activation card are provided in 2 illustrated.

In step 1.III, a parameterized 3D rail model M is determined with a parameter set C ₀ to be adjusted and a projection rule T ^cr0 to be adjusted. The mentioned data to be adjusted can, for example, come from a control unit 3 of the rail vehicle 1 (see 7 ), which uses this data to process the image data BD from the image recording unit BA in order to determine a pose P of the rails lying in front of the rail vehicle 1.

In step 1.IV, an optimization step takes place in which a deviation L of the parameterized rail model M2D projected into the activation map AK based on the projection rule T ^cr0 is determined from the activation map AK and is minimized using an iteration process. As a result, an adjusted parameter set C and an adjusted projection rule T ^cr are determined.

In step 1.V, the adjusted parameter set C for an adjusted parameterized 3D rail model M 'and the adjusted projection rule T ^cr are used for the subsequent determination of the pose P of the rail track ST running in front of the rail vehicle 1.

In 2 is a flowchart showing the in 1 illustrated step 1.II for determining an activation card AK illustrated in detail.

In step 1.IIa, the captured image data BD is used to generate segmented image data SBD. The segmented image data SBD is generated using an edge filter that detects edges based on differences in brightness and thus makes object outlines visible. Furthermore, the segmentation can be done based on an AI-based classification model.

In step 1.IIb, an activation map AK is generated, with the segmented image data SBD being smoothed or blurred to a certain extent by using a Gaussian kernel GK, so that they have a certain continuity with continuous transitions in brightness and are therefore better suited for use Optimization method are suitable.

In 3 A flowchart is shown which illustrates the in 1 Step 1.IV shown for determining an adjusted parameterized 3D rail model M 'is illustrated in detail. For this purpose, an iterative approximation procedure is used for optimization. A variant for determining the above-mentioned deviation L, and thus an error function, includes the method of determining the least squares.

Generally speaking, the step of determining an adjusted parameterized 3D rail model M' is using a local nonlinear optimization method to calculate the parameters C of the adjusted parameterized 3D rail model M' and the parameters of an exact projection T ^CR of the adjusted parameterized 3D rail model M 'connected to the captured image data BD. With the help of such generally iterative approximation methods, a more precise parameterized 3D rail model and a more precise projection are determined in each step, which then form a basis for the parameterized 3D rail model and the projection in the next step. Examples of optimization methods in this category are the downhill simplex algorithm, the Gauss-Newton method or the Levenberg-Marquardt algorithm.

Some of these methods require the 1st or 2nd derivative of the loss function with respect to the parameters being sought. In this case, the Jacobi matrix determined can be used as a quality criterion for the determined parameters. The projection parameters include, among other things, the rotation, the translation or the focal length and the optical center, and therefore have a different scaling. For efficient and robust optimization, it is necessary to align the magnitudes of these parameters.

In step 1.IVa, the parameterized 3D rail model M determined in step 1.III is projected onto the captured image data BD with the parameter set C ₀ to be adjusted and the projection rule T ^cr0 to be adjusted. A two-dimensional representation M2D of the parameterized 3D rail model M, also referred to as a projected parameterized rail model, which is inserted into the captured two-dimensional image data BD, is generated.

In step 1.IVb, a deviation L between apparent brightness values of the projected parameterized rail model M2D and activation points of the activation map AK is determined. This deviation is determined based on the sum of the squares of the differences between intensity values of the projected parameterized rail model M2D and activation points of the activation map AK.

In step 1.IVc, adapted parameter values C ^k of the parameterized rail model M and an adapted projection matrix T ^crk are determined based on the determined deviation using the approximation methods mentioned in connection with step 1.IV. For example, as part of the Gauss-Newton method, a Jakobi matrix J can be determined based on the determined deviation L. Based on the Jakobi matrix J, adjusted parameter values C ^k and an adjusted projection matrix T ^crk can then be determined. Alternatively, direct derivations of the deviation L with respect to the mentioned parameters C ^k of the parameterized rail model and the parameters characterizing the projection, in particular parameters of the projection matrix T ^crk , can be formed and these values can be used to determine approximate parameter values of the parameterized 3D rail model M and the adapted ones projection can be used.

In step 1.IVd it is checked whether the quality Q of the determined variables C ^k , T ^crk reaches a quality criterion. The quality criterion can, for example, include a predetermined threshold value for the determined deviation L. If the threshold value is undershot, the quality criterion Q is achieved. In addition, the Jakobi matrix J or its entries can be used for a quality assessment.

If the quality criterion Q is achieved, what in 3 is symbolized with “y”, then step 1.IVe is proceeded to, and the determined adjusted parameter set C _k and the determined adjusted projection matrix T ^crk are used as an adjusted parameter set C for an adjusted parameterized 3D rail model M' and as an adjusted projection matrix T ^cr issued.

If the quality criterion Q has not yet been achieved, what happens in 3 is marked with “n”, then one returns to step 1.IVa and the adjusted parameter set C _k and the determined adjusted projection matrix T ^crk are used to project the rail model M onto the activation map AK and the iteration is continued until the quality criterion Q is fulfilled in step 1.IVd.

In 4 an adjustment device 40 is shown, which has an input interface 41 for receiving image data BD from a surrounding area UB of a rail vehicle 1 (see 7 ). The image data BD includes one or more rail tracks ST (see 6 ), which is provided by an image recording unit BA (in.) arranged on the rail vehicle 1 4 not shown).

The adjustment device 40 also includes a determination unit 42 for determining an activation card AK based on the captured image data BD.

Part of the adjustment device 40 is also an adjustment unit 43 for determining an adjusted, parameterized 3D rail model M '. The adjusted parameterized 3D rail model M 'represents the pose P of the rail track ST relative to the image recording unit BA. Adjusted parameter values C are assigned to the adjusted parameterized 3D rail model M'. The adjusted, parameterized 3D rail model M' is determined through an iterative process, which is also referred to as "matching", whereby a parameterized 3D rail model M is compared with the determined activation map AK. For this comparison, a projection rule T ^cr, which is also automatically adjusted in this process, is required in order to project the compared rail model M onto the activation map AK.

In 5 is a schematic representation of the in 4 illustrated adjustment unit 43 of the adjustment device 40 is shown. The adjustment unit 43 includes a model determination unit 43a for receiving initial parameter values as candidate parameter values C ₀ and an initial candidate projection matrix T ^cr0 , which, for example, from a control unit 3 of the rail vehicle 1 (see 7 ) to determine a pose P of a rail track ST (see 6 ) be used.

The thus specified parameterized 3D rail model M and the activation map AK are transmitted to a fusion unit 43b, which generates the specified parameterized 3D rail model M and the activation map AK by projecting the specified parameterized 3D rail nenstrand model M on the activation card AK to form a combined activation card KAK.

The combined activation map KAK is transmitted to a deviation determination unit 43c, which determines a deviation L between apparent brightness values of the projected parameterized rail model M2D and the activation map AK on the basis of the combined activation map KAK.

Based on the determined deviation L, an adaptation unit 43d determines adapted parameters C ^k and an adapted projection matrix T ^crk .

A quality unit 43e determines whether the variables C _k , T ^crk determined by the adaptation unit 43d reach a quality criterion Q.

If the quality criterion Q is achieved, the determined adjusted parameter set C _k and the determined adjusted projection matrix T ^crk are output as adjusted parameters C for an adjusted parameterized 3D rail model M 'and as an adjusted projection matrix T ^cr .

If the quality criterion Q is not yet reached, the adapted parameter set C _k and the determined adapted projection matrix T ^crk are transmitted to the fusion unit 43b and used there to project the 3D rail model M parameterized with the adapted parameter set C _k onto the activation card AK.

In 6 is a symbolic representation of the in 1 until 3 already illustrated method shown. Individual steps are shown in a sequence of symbolic image representations, with the sequence forming a chain that runs from the top left of the image to the right and then continues at the bottom left of the image and also runs from left to right in the bottom row.

At the top left there is one similar to step 1.I (see 1 ) Corresponding acquisition of image data BD from a visual area lying in front of a rail vehicle 1 in the direction of travel with rail tracks ST is shown. The next step is one similar to step 1.IIa (see 2 ) Corresponding determination of segmented image data SBD is illustrated based on edge detection in the captured image data BD. Furthermore, the segmented image data SBD is “blurred”, which corresponds to step 1.IIb (see 2 ) corresponds and the activation card shown at the top right results in AK.

At the bottom left, a step in generating a parameterized rail model M based on candidate parameters C _k is symbolized. This procedure corresponds to step 1.III (see 1 ).

To the right, an image projection based on a candidate projection matrix T ^rck is symbolized. This step corresponds to step 1.IVa (see 3 ).

The bottom row below shows a combined activation map KAK, in which both the activation map AK based on the captured image data BD and the brightly displayed parameterized rail model M2D projected onto the activation map AK and crossing with a rail track ST are shown.

Finally, as a final symbolic representation, the step of minimizing a deviation L of the projected parameterized rail model M2D from the activation map AK is shown. This step corresponds to step 1.IV (see 1 ). Based on the determined deviation L, adjusted parameters C _k are used, which are then used again to determine an adjusted parameterized 3D rail model M, and an adjusted projection matrix T ^crk , which is used to re-project the parameterized 3D rail model adjusted by the adjusted parameters C _k . Rail model M is used on the activation card AK, which is symbolized by corresponding arrows pointing back.

In 7 is a schematic representation of a rail vehicle 1 illustrated according to an embodiment of the invention. The rail vehicle 1 moves in the direction of the arrow over a rail track ST.

The rail vehicle 1 includes an image recording unit BA for acquiring image data BD from a surrounding area UB of the rail vehicle 1. The surrounding area UB includes a rail track ST lying in front of the rail vehicle 1. Part of the rail vehicle 1 is also an adjustment device 40, as shown in 4 and 5 is illustrated in detail to determine an adjusted parameterized 3D rail model M'. The adjustment device 40 transmits the adjusted, parameterized 3D rail model M 'with the adjusted parameter set C and the adjusted projection matrix T ^cr to a control unit 3, which, in order to control a journey of the rail vehicle 1, uses a pose of the rail track ST lying in front of the rail vehicle 1 on a basis of the adjusted parameterized 3D rail model M'. Parameter values C ⁰ , T ^cr0 of a 3D rail model M to be adjusted, which is currently being used by the control unit 3, can also be transmitted to the adjustment device 40 to make an adjustment of these parameter values C ₀ , T ^cr0 .

In 8th a schematic representation of an image-based activation map AK generated by smoothing is shown. For simplicity, the activation map AK is only shown as a projection onto the x-axis. Of course, an activation map AK is usually two-dimensional, that is, it has xy coordinates for the individual activation points. In the activation map AK, discrete activation values AW are represented by rectangular bars at the positions where rail tracks ST are present. In addition, a parameterized rail model M2D projected onto the activation map AK is shown at the positions x ₀ and x ₀ 'with activation values AW' symbolized by dashed rectangular bars (AW = 1, AW ' = 1). If an iterative determination of suitable model parameters now begins, for example, with the projected parameterized rail model M2D at the positions x ₀ and x ₀ ' between the rail strands ST, the derivation results in the sum of the squares of the differences between the activation values AW of the rail strands ST and the activation values AW' of the projected parameterized rail model M2D has the value 0.

That is, if the projected parameterized rail model M2D is shifted slightly in the x direction, the squares of the differences between the activation values AW of the rail strands ST and the activation values AW' of the projected parameterized model M2D do not change. For example, the value of the sum of the squares of the differences between the activation values AW of the rail tracks ST and the activation values AW' of the projected parameterized model M2D remains at the value 4.

However, this would be a derivation-based approximation, for example a Newton method with the calculation rule x _n+1 = x _n - f (x _n ) /f' (x _n ) (for example n = 0, 1, 2, ..., f (x ) equal to the sum of the squares of the differences between the activation values AW of the rail tracks ST and the activation values AW of the projected parameterized model M2D) not applicable, because x ₁ would then have a negative infinite value. The Newton method should only be chosen here for the sake of simple illustration, for example as a representative of approximation methods for functions with multi-dimensional arguments or multi-dimensional functions, such as the Gauss-Newton method or other approximation methods mentioned above in the text.

If the discrete distribution of the activation values AW of the rail track ST is now converted into a continuous distribution ST', which in 8th is represented by curved solid lines, transformed, the derivation of the squares of the differences between the activation values AW of the rail strands ST and the activation values AW 'of the projected parameterized rail model M2D results in a value different from the value 0, so that the iteration is continued can, since X ₁ takes on a finite value.

Finally, it should be pointed out once again that the methods and devices described above are merely preferred exemplary embodiments of the invention and that the invention can be varied by a person skilled in the art without departing from the scope of the invention, insofar as it is specified by the claims. For the sake of completeness, it should also be noted that the use of the indefinite articles “a” or “an” does not exclude the fact that the characteristics in question can be present multiple times. Likewise, the term “unit” does not exclude the fact that it consists of several components, which may also be spatially distributed.

Claims (15)

Method for determining an adjusted parameterized 3D rail model (M'), comprising the steps: - Acquiring image data (BD) from a surrounding area (UB) of a rail vehicle (1), which comprises a rail track (ST), by an image recording unit (BA) arranged on the rail vehicle (1), - Determining an activation map (AK) based on the captured image data (BD), - Determining the adjusted parameterized 3D rail model (M'), which reproduces the pose (P) of the rail track (ST) relative to the image recording unit (BA), based on a comparison of a parameterized 3D rail model (M) with the determined activation map (AK ). Procedure according to Claim 1 , wherein determining the adjusted parameterized 3D rail model (M') comprises the following steps: i) Projecting the 3D rail model (M) parameterized by candidate parameters (C ₀ ) onto the activation map (AK) based on a candidate projection rule (T ^cr0 ), ii) determining adapted parameter values (C _k ) of the parameterized 3D rail model (M) and/or an adapted projection rule (T ^crk ) based on the comparison of the projected parameterized rail model (M2D) with the activation map (AK) . Procedure according to Claim 2 , wherein determining an adjusted parameterized 3D rail model (M') includes carrying out an iterative approximation process, wherein steps i) and ii) are repeated and, upon repetition, the adjusted parameter values (C _k ) determined in step ii) in step i ) are used as candidate parameter values (C ₀ ) and/or the adapted projection rule (T ^crk ) determined in step ii) is used in step i) as a candidate projection rule (T ^cr0 ). Procedure according to Claim 2 or 3 , wherein the comparison includes determining at least one of the following variables: - a deviation (L) of the projected parameterized rail model (M2D) from the activation map (AK), - a correlation (K) between the projected parameterized rail model (M2D) with the activation map (AK) . Procedure according to Claim 4 , wherein the comparison comprises a determination of a deviation (L) and the comparison comprises a determination of a minimum of the deviation (L) between the projected parameterized rail model (M2D) and the activation map (AK). Procedure according to one of the Claims 4 or 5 , wherein the comparison comprises a determination of a deviation (L) and the determined deviation (L) comprises a deviation between activation values (AW') of the projected parameterized rail model (M2D) and activation values (AW) of the activation map (AK). Procedure according to one of the Claims 4 until 6 , where the comparison comprises a determination of a deviation (L) and the deviation (L) is the sum of the squares of the differences between activation values (AW') of activation points of the projected parameterized rail model (M2D) and activation values (AW) of activation points of the activation map (AK ) having. Method according to one of the preceding claims, wherein the parameterized 3D rail model (M) has at least one of the following features: - 3D points which are arranged along a rail head of the rail track (ST) with a predetermined scanning density, - a predetermined parameterizable basic shape, which preferably comprises the shape of one of the following objects: - a clothoid, - a branch, - parallel straight lines. Method according to one of the preceding claims, wherein segmented image data (SBD) is determined based on the image data (BD) and the activation map (AK) is determined based on the segmented image data (SBD). Procedure according to Claim 9 , whereby the activation map (AK) is determined by smoothing the segmented image data (SBD). Procedure according to Claim 10 , where the smoothing is done by applying a Gaussian kernel (GK) to the segmented image data (SBD). Adjustment device (40), comprising: - an input interface (41) to - Receiving image data (BD) from a surrounding area (UB) of a rail vehicle (1), which comprises a rail track (ST), from an image recording unit (BA) arranged on the rail vehicle (1), - a determination unit (42) for determining an activation card (AK) based on the captured image data (BD), - an adjustment unit (43) for determining an adjusted parameterized 3D rail model (M'), which reproduces the pose (P) of the rail track (ST) relative to the image recording unit (BA), by comparing a parameterized 3D rail model (M ) with the determined activation card (AK). Rail vehicle (1), comprising - an image recording unit (BA) for acquiring image data (BD) from a surrounding area (UB) of the rail vehicle (1), which comprises a rail track (ST), - an adjustment device (40). Claim 12 for determining an adjusted parameterized 3D rail model (M'), - a control unit (3) for controlling a journey of the rail vehicle (2) based on the adjusted parameterized 3D rail model (M'). Computer program product, comprising instructions which, when the program is executed by a computer, cause it to carry out the steps of the method according to one of the Claims 1 until 11 to determine an adjusted parameterized 3D rail model (M'). Computer-readable storage medium comprising instructions which, when executed by a computer, cause it to follow the steps of the method Claim 1 until 11 to determine an adjusted parameterized 3D rail model (M').

Images