EP4119417A1 - Monitoring method and system - Google Patents

Abstract

The present invention relates to a method and a system (50) for monitoring a vehicle environment (10) when a vehicle (1) is traveling on a route (2), and to a vehicle (1). A scene in the vehicle's surroundings is detected by sensors and corresponding sensor data is generated. A virtual scene representation of the recorded scene is determined on the basis of the sensor data and quality-assured route information stored in a database (52).

Description

The present invention relates to a method and a system for monitoring a vehicle environment when a vehicle is traveling on a route, and a vehicle, in particular a rail vehicle.

Modern vehicles, in particular rail vehicles, are usually equipped with a sensor system that is used to monitor the area around the vehicle. With the help of the sensor system, other road users or lanes can be detected, for example, and the information obtained can be used by assistance systems for - if necessary supporting - control of the vehicle. It is a particular challenge to process the amount of data generated when monitoring the vehicle's surroundings. For example, it can be difficult to analyze complex driving situations in which a large number of static objects and/or other road users interact with one another and to determine a response by the assistance system that is appropriate to the situation.

It is an object of the present invention to improve, in particular to simplify, an analysis of driving situations.

This object is achieved by a method and a system for monitoring the area around the vehicle when a vehicle is traveling on a route, and by a vehicle according to the independent claims.

Preferred embodiments are subject matter of the dependent claims and the following description.

A first aspect of the invention relates to a method, in particular a computer-implemented method, for monitoring a Vehicle environment when driving a vehicle, in particular a rail vehicle, on a particular predetermined route. The method has the following steps: (i) sensory detection of a scene in the vehicle environment and generation of corresponding sensor data; (ii) Generating a virtual scene representation of the captured scene based on the sensor data and quality-assured route information stored in a database.

A scene within the meaning of the invention is preferably to be understood as a driving situation of the vehicle. In other words, a scene can represent a "snapshot" of the vehicle's surroundings at a particular point in time. A scene is thus expediently characterized by objects from the area surrounding the vehicle and their arrangement relative to one another and/or to the vehicle at a specific point in time.

A virtual scene representation within the meaning of the invention is preferably understood to mean a digital image of a (real) scene, for example from the vehicle environment. Using a virtual representation of a scene, an assistance system can analyze the vehicle environment and control the vehicle or at least individual vehicle functions based on the analysis and/or at least give the vehicle driver instructions on how to control the vehicle.

Route information within the meaning of the invention is preferably to be understood as additional information about possible scenes on the route. The route information can be, for example, additional information about objects that can be detected by sensors or a specific area in the scene that is currently being detected. The route information can be at least partially route-specific. For example, it is conceivable for the route information to identify special objects, object constellations, route sections and/or the like that occur along the route and/or their effects on one another and/or on the vehicle.

A database within the meaning of the invention is preferably understood to mean a map, in particular a digital map, which can contain route-specific information, for example. In this case, the information contained in the database, in particular route-specific information, is expediently linked to geographical information. In particular, it is preferred if the database also contains generic information about the vehicle environment, for example about road users possibly occurring in the vehicle environment. The database expediently contains, for example, in addition to route information about the course of streets or paths, additional information about the surroundings of the streets, for example about so-called landmarks. With an extremely high spatial resolution, the landmarks can characterize, for example, traffic signs, traffic lights, street lamps, buildings and/or other prominent objects in the vicinity of streets or paths.

Quality-assured information within the meaning of the invention is preferably understood to mean information that meets at least one specified security requirement. Quality-assured information can, for example, be correct with a specified probability or have a small error, so that a system using the information only has a low probability of behavior that is dangerous to health or life. An (assistance) system operated on the basis of the quality-assured information preferably fulfills a so-called safety requirement level (SIL) in order to be able to detect any risk to people, the environment or processes from the system with a specified reliability - for example to less than one risk every eleven years. to be able to reduce.

The quality-assured information can, for example, characterize properties, also referred to below as characteristics, of objects. The quality assurance of the information expediently includes, among other things, that at least one Combination of properties or characteristics is specific to each of the objects. In particular, it can be provided for quality-assured information that the unambiguous description of an object is secured by at least one feature, in particular a combination of features, with regard to specificity in relation to other objects.

One aspect of the invention is based on the approach of using external information to adapt, in particular to simplify and/or supplement, a virtual scene representation that is based on sensory detection of a vehicle environment and characterizes a real scene from the vehicle environment. External information here means that the information is not provided by the vehicle or its sensors, but rather is taken from a database, for example. A virtual scene representation adapted in this way can then be analyzed, for example, and the control of at least one sub-function of the vehicle can be used as a basis. The determination of the scene, taking into account the information from the database, enables not only a more reliable, but possibly also a faster scene analysis.

Accessing external information, also referred to as route information, stored in the database is particularly advantageous since this allows prior knowledge about the vehicle environment along the route to be incorporated into the determination of the virtual scene representation—in particular into its adaptation. This exploits the fact that rail vehicles in particular are repeatedly traveling on the same route and objects that do not impair driving safety, for example, are previously known or can at least be determined. If, for example, the virtual scene representation is to be used as a basis for a risk analysis to assess driving safety, these objects do not have to be included in the scene representation at all, or they can be marked there as irrelevant for driving safety. Through this technical support can also a vehicle driver who has not yet been trained for this route can drive a new route with a comparable level of safety as a trained driver, which means that the flexibility of rail vehicle operations with regard to personnel planning can be significantly increased.

To determine the virtual scene representation, for example, a scene in the area surrounding the vehicle, such as the track bed in front of a train or a platform when the train enters a station, can be detected by sensors and the position of the vehicle on its route can be determined. Route information corresponding to the position can then be loaded from a digital map. The virtual scene representation can be generated on the basis of the sensor data generated during the sensory detection of the vehicle surroundings and the route information. In particular, it is conceivable that the virtual scene representation determined on the basis of the sensor data is adapted using the route information, i. H. is further modified.

The route information is expediently quality-assured, i. H. in particular accurate, complete, up-to-date and available.

This means that the semantics of the information should be well defined so that, for example, an algorithm based on the information can correctly interpret it. For example, if the information contains object features to characterize an object, the value range of the features should be defined exactly and completely together with the possible characteristics and the semantics of each characteristic. In addition, the required parameters should be defined for each use of the information, for example in connection with a safety-related function. Such parameters can be, for example, the quality features specified for identifying an object or determining the position of an object. The parameters should be the correct situation for each trip show on the route. The information should also be available for use while driving with the quality required, for example, in a safety architecture.

The use of quality-assured route information not only allows a particularly reliable scene adaptation, but also, if necessary, a check of the vehicle's sensory functions by comparing the route information with information obtained from the sensor data. For example, properties derived from the sensor data of an object detected in the vehicle environment can be compared with so-called feature vectors stored in the digital map, which characterize objects in the vehicle environment.

The feature vectors preferably have specific features of the detected objects as entries. The length of the vectors, i. H. the number of features they describe corresponds appropriately to a measure of reliability. A large amount of information stored in the database allows objects from the vehicle's surroundings that are detected using the sensor data to be validated with a high level of reliability.

In particular, an object recognized on the basis of the sensor data can be identified in the database with a high level of certainty by a combination of features of a feature vector, i. H. associated with an object recorded in the database. As a result, the presence of the object can be verified in the sensor data.

The feature vectors are expediently overdetermined. This means that a subset of specific features contained in the feature vectors is already sufficient to clearly identify the object. As a result, the robustness when comparing the sensor data with the corresponding feature vectors can be increased.

The length and properties of the feature vectors are preferably defined or predetermined by a risk analysis and security architecture. In particular, the features in the feature vector should be sufficiently specific so that a coincidental match between objects from the vehicle environment and objects recorded in the database can be ruled out with a sufficiently high degree of certainty.

The route information for scene adaptation is expediently also taken from such feature vectors. The feature vectors can, for example, contain information about how great the influence of the respective object is on the driving safety of the vehicle. Correspondingly, certain objects in the scene representation can be given greater consideration when the vehicle is controlled, while other objects can be neglected.

In the following, preferred embodiments of the invention and their developments are described, which, insofar as this is not expressly excluded, can be combined with one another as desired and with the aspects of the invention described below.

In a preferred embodiment, when the virtual scene representation is generated, a degree of correspondence is determined for the correspondence between environmental information derived from the sensor data and the route information. The degree of agreement can also be understood as a degree of confidence which characterizes the reliability of the detection of the objects in the vicinity of the vehicle, in particular the extent of discrepancies between the environmental information and the route information. In other words, the degree of agreement can be used to specify a specificity with which an object is recognized on the basis of the sensor data. The reliability of object recognition can be assessed particularly easily on the basis of the degree of agreement.

The degree of agreement can, for example, assume a high value if the sensor data not only characterizes objects that are also characterized by the route information, but also if the characterization by the respective information essentially matches. The route information can not only contain information about the position of an object, but also about other quality-assured features, the traceability and specificity of which can be proven to an expert. The route information can, for example, also contain information about the shape, structure, extent, texture, color and/or the like of an object. The greater the proportion of this information that matches corresponding information from the sensor data, the greater the determined degree of correspondence can be. Accordingly, the ascertained measure of correspondence can assume a small value if the information contained in the sensor data about a position and shape of a detected object essentially corresponds to the corresponding route information, but the information about the extent and texture does not.

Expediently, a check is made as to whether the degree of correspondence reaches or exceeds a predetermined correspondence threshold value. The agreement threshold value can be used as an indicator as to whether a number of properties of a recognized object that is specified as sufficient or certain agree with the route information. If this is the case, reliable detection of the objects can be assumed. From this, conclusions can be drawn, for example, about the reliability of the position determination of the vehicle, the reliable and precise functioning of the sensors for detecting and determining the object positions and, if necessary, the correct calibration of several sensors to one another. If this is not the case, it can be assumed that the object was not recognized correctly.

In this case, the agreement threshold value preferably depends on the security requirements or can be dependent be elected from. If the matching threshold value is chosen to be large and the matching measure reaches or exceeds it, a high security level can in principle be associated with the object recognition. On the other hand, if the matching threshold value is chosen to be smaller and the matching level is reached or exceeded, only a low security level can be associated with the object recognition.

Where appropriate, different route information may be valid for different circumstances in which the vehicle is traveling on the route. Therefore, in a further preferred embodiment, the route information is provided as a function of at least one travel parameter, ie read from the database, for example. A travel parameter can be, for example, the position, the speed and/or the direction of travel of the vehicle. Alternatively or additionally, a trip parameter can also be the weather, the time, the date and/or the like. This enables a differentiated assessment of the recorded scene.

By taking into account route information dependent on travel parameters, the impairment of driving safety by an object can be taken into account in the larger context. For example, people on a platform when a train is passing through the station can be assessed differently from a safety point of view than people who are on a possibly structurally separate path next to the railway line. While the latter do not have to be included in the virtual scene representation, considering the people on the platform can be of the utmost importance for driving safety.

However, if a person is on or near the open track without a protective barrier, particularly at the same distance from the track as the person is in the station, then they are considered more relevant to the safety of the journey, as that person is not actually in that position should stop. If this person is on the free route without a barrier in the physical vicinity of a hospital where people with psychiatric illnesses are treated, the assessment of the short distance of the person to the road can be of even greater importance. Accordingly, the assessment of the fact that a person is in the vicinity of the track can depend very much on the context in the scene. The information for the safety-related evaluation of the position of the person relative to the track is preferably stored in the database in a correspondingly quality-assured manner. The safety-related evaluation can thus be stored for the entire route when planning the journey, which means that this evaluation does not have to be determined by an algorithm during the journey.

In a further preferred embodiment, the ascertained virtual scene representation is used as the basis for an assessment of driving safety. For example, an analysis of the virtual scene representation can be carried out and a risk assessment can be carried out on the basis of the analysis. Driving safety is preferably to be understood as safety both with regard to a risk to the vehicle from its environment, for example from objects in the vehicle environment, and with regard to a risk to the vehicle environment, ie for example objects in the vehicle environment, from the vehicle . On the basis of the virtual representation of the scene, the vehicle can be safely steered through its surroundings, in particular along the specified route.

In a further preferred embodiment, for objects identified in the recorded scene, the route information is used to check whether a predetermined risk criterion is met. The objects are preferably taken into account depending on a result of the check when determining the virtual scene representation, for example marked in the scene representation by a "flag", ie a status indicator, or not included at all. The objects can preferably be taken into account as a function of a result of the check/marking in an assessment of driving safety based on the scene representation. For example, it is conceivable that objects are removed from the scene representation during the evaluation depending on their marking. This can prevent the (computing) effort from increasing in an analysis of the scene representation, in particular with regard to driving safety, due to the unnecessary consideration of objects that are per se dangerous, for example stationary objects such as buildings, vegetation and/or the like.

The route information expediently contains a risk assessment—with regard to the journey of the vehicle—of individual objects in the recorded scene. For example, individual objects along the route, possibly even entire types of objects, can be rated as dangerous or not dangerous and/or have a danger score. This type information is expediently also quality-assured. This allows a particularly simple check as to whether the specified risk criterion is met or a particularly simple consideration of the route information.

In a further preferred embodiment, the route information contains type information specific to an object type, ie information that characterizes an object type. All objects of an object type are preferably taken into account when determining the virtual scene representation according to the type information, e.g. B. marked. All objects of an object type are particularly preferably removed from the virtual scene representation on the basis of the type information, for example in order to facilitate the scene analysis. In this context, type information can be understood in particular as generalized information on an object type, for example on a vehicle or a specific vehicle class, a pedestrian, the vegetation, buildings, signal systems and/or the like. The type information can, in particular, be a generally valid risk assessment of a specific object type, ie a risk assessment valid for all objects of this type, for example in the form of a risk score. The type information can contain, for example, the information that a pedestrian can potentially be particularly endangered by the vehicle. Alternatively or additionally, the type information can contain the information that there is generally no danger from birds or an oncoming rail vehicle (on a different track). With the help of the type information, account can be taken of the fact that certain objects from a detected scene do not have to be taken into account a priori with regard to driving safety.

In another preferred embodiment, the route information includes scene information specific to an area in the captured scene. In other words, the route information contains information that characterizes an area in the scene, preferably with regard to its effect on driving safety. The determination or a (later) adjustment of the virtual scene representation is then expediently carried out on the basis of the scene information. The scene information can be used to better secure the journey, even in potentially dangerous sections of the route.

For example, the scene information can identify areas in which objects are never to be regarded as dangerous. For example, areas can be marked that are separated from the route of the vehicle, for example from the track bed, by a barrier, such as a structural measure. For objects recognized in the recorded scene, the scene information can be used to check whether they can be removed from the virtual scene representation or whether they should be recorded at all. Conversely, the scene information can also be used to highlight areas in and/or from which there is a particularly great danger. For example, in a scene analysis, a area characterized by the scene information, such as a platform, should be given particular attention.

In a further preferred embodiment, the route information contains interaction information on the interaction between at least two objects in the recorded scene. For example, the route information can contain information about a barrier in the vehicle environment, which restricts the freedom of movement of other objects in the vehicle environment. For example, the information can be stored in the database that a footpath running parallel to the route is separated from the route by a fence, for example a track bed, which prevents pedestrians from entering the route. It is also conceivable that the position of a tree standing at a level crossing is stored in the database, which makes it difficult to detect people on the level crossing due to the shadow it casts. This makes it possible to take into account the impact of certain objects on their surroundings in a differentiated manner, possibly even as early as the sensory capture of the scene.

The determination, in particular the (subsequent) adjustment, of the virtual scene representation is preferably undertaken taking into account the interaction information. In the case of the aforementioned examples, people on the footpath could be removed from the virtual scene representation or marked as not dangerous, or the scene representation in the shade of the tree could be recorded and/or processed with other parameters or sensors.

In a further preferred embodiment, a safety area in the detected scene is determined on the basis of the route information, in particular the interaction information, in which the risk of driving through an object from the scene is at least reduced. A security area can also be understood as a protection zone in which objects cannot impair driving safety. A safety area can be formed, for example, by a barrier, such as a fence, a wall, a barrier, a safety line on a platform and/or the like. The safety area is expediently located on a side of the barrier that faces away from the route. Objects detected in the scene that are in the safety area can be removed from the scene representation or marked as non-dangerous accordingly. Alternatively, at least their risk assessment can be adjusted, for example a risk score assigned to them can be reduced at least temporarily.

In a further preferred embodiment, the effect of the safety area on a trajectory of an object from the scene is determined. This effect on the trajectory is expediently taken into account when determining, in particular the (later) adaptation, of the virtual scene representation. The effect of the security area on an object preferably depends on the type information corresponding to the object, i. H. from the object type. For example, bollards can slow down a cyclist's journey, while allowing a pedestrian to continue walking essentially unhindered, but stopping a car. The effects of a barrier on moving objects can thus be taken into account in a differentiated manner.

The effect on the trajectory can also depend on the type information corresponding to the barrier. For example, depending on the type of barrier, scene matching can be based on a strong constraint on the movement of objects, such as a fence or barrier, or a weak constraint, such as a safety line on a train platform. Information on the strength of the effect is expediently contained in the interaction information.

In a further preferred embodiment, the route information is validated while the vehicle is traveling on the route. During the journey, it is preferably checked in particular whether an object in the detected scene has the expected effects on other objects in the scene, for example according to the interaction information. It can thus be checked, for example, whether a barrier or a safety area defined by it is intact, i.e. whether it actually restricts the movement of other objects, in particular to the extent expected. With such a validation of the route information, the journey of the vehicle, but in particular the future journeys of other vehicles on the same route, can be secured even better. Alternatively or additionally, it is also conceivable that appropriate safety measures are initiated if the validation fails, for example the route information is corrected and/or an emergency stop is initiated.

In a further preferred embodiment, the route information is validated using a comparison of the temporal development of the recorded scene with a prediction based on the route information. For example, a determined ("measured") trajectory of a cyclist detected in the area of a barrier can be compared with a trajectory of the cyclist predicted, for example, using the interaction information. If there is a match, it can be assumed that the route information is correct.

In a further preferred embodiment, (i) additional object information is received specifically for an object from the detected scene and (ii) the virtual scene representation is determined, in particular adapted, on the basis of the additional object information. Such additional object information can be sent, for example, by objects from the scene, such as other road users, signal systems, personnel working on the route and/or the like. The consideration of the additional object information is particularly advantageous because you can place a particularly high level of trust in it. In other words, the scene representation can be adjusted based on specially secured information.

It is preferred that the determined virtual scene representation is transmitted to other road users, for example an oncoming train. Information about the reliability of the determination of the virtual scene representation is preferably also transmitted during the transmission. This can facilitate the scene analysis of the other road user.

In a further preferred embodiment, the additional object information contains status information that characterizes a status of an object in the recorded scene. The current risk to the vehicle's travel and/or the object posed by the vehicle can preferably be assessed on the basis of the status information. Such additional information can, for example, relate to the opening status of a barrier: if the barrier is closed, there is a low risk, but if the barrier is open, there is a high risk. The additional information can also include that personnel working on the route have noticed and confirmed the approach of the vehicle. However, the status information can also characterize a dynamic condition, for example the acceleration of a road user: if the road user brakes before a level crossing, there is a low risk, but if the road user accelerates or maintains his speed, there is a high risk. The information about dynamic objects can also come from a trackside system which monitors a certain area and transmits the information about dynamic objects in a quality-assured manner.

In a further preferred embodiment, the route information contains lighting information. The lighting information can, for example, information about the shadows cast by objects in the captured scene, glare effects, reflections, brightness gradients and/or the like, which are to be expected, for example, when capturing the scene by sensors. Lighting-related effects are expediently taken into account when determining the virtual scene representation, possibly even as early as when the scene is detected by sensors. For example, shadowed areas in the scene representation can be illuminated or objects can be identified as reflections of another object and removed from the scene representation or not included at all. In this way, even seemingly complex decision-making situations can be reliably resolved or at least simplified.

A second aspect of the invention relates to a system for monitoring a vehicle environment when a vehicle, in particular a rail vehicle, is traveling on a route, in particular a predetermined one. The system has a sensor device that is set up to capture a scene and generate corresponding sensor data, and a database in which quality-assured route information is stored. In addition, a data processing device is provided, which is set up to generate a virtual scene representation based on the sensor data and the route information.

Such a system is preferably set up to carry out the method according to the first aspect of the invention. It can be used particularly advantageously in a rail vehicle to ensure safe travel of the rail vehicle on a predetermined route, i. H. along rails, and/or to avoid or at least reduce the risk to objects in the vicinity of the vehicle or, more generally, in the vicinity of the route.

A vehicle, in particular a rail vehicle, according to a third aspect of the invention has a system according to the second aspect of the invention.

The description of preferred embodiments of the invention given so far contains numerous features, some of which are summarized in the individual dependent claims. However, these features can expediently also be considered individually and combined to form further meaningful combinations. In particular, these features can each be combined individually and in any suitable combination with the method according to the first aspect of the invention, the system according to the second aspect of the invention and the vehicle according to the third aspect of the invention. Thus, method features are also to be seen formulated as a property of the corresponding device unit and vice versa.

The properties, features and advantages of this invention described above, and the manner in which they are achieved, will become clearer and more clearly understood in connection with the following description of the exemplary embodiments, which are explained in more detail in connection with the drawings. The exemplary embodiments serve to explain the invention and do not restrict the invention to the combination of features specified therein, not even in relation to functional features. In addition, suitable features of each exemplary embodiment can also be explicitly considered in isolation, removed from one exemplary embodiment, introduced into another exemplary embodiment to supplement it and combined with any of the claims.

FIG 1

ein Beispiel einer Szene aus dem Umfeld eines Fahrzeugs;

FIG 2

ein Beispiel einer Szeneninformation spezifisch für einen Bereich einer Szene;

FIG 3

ein Beispiel eines Systems zum Überwachen eines Fahrzeugumfelds; und

FIG 4

ein Beispiel für eine Nutzung von in einer Datenbank gespeicherten, qualitätsgesicherten Routenin-formation.

Show it:

FIG 1

an example of a scene from the surroundings of a vehicle;

FIG 2

an example of scene information specific to a portion of a scene;

3

an example of a system for monitoring a vehicle environment; and

FIG 4

an example of the use of quality-assured route information stored in a database.

FIG 1 shows an example of a scene from the environment 10 of a vehicle 1. The vehicle 1 is designed here, for example, as a rail vehicle that travels on a route 2 specified by a rail track. The scene is characterized by objects 11 from the environment 10 of the vehicle 1 - or, to put it more generally: from the environment of the route 2 . Presently, in the environment 10 of the vehicle 1, purely by way of example, there is a tree 11a, a wall 11b, a footpath 11c, birds 11d, a signal system 11e, a bridge 11f and people 11g.

The vehicle 1 is equipped with a sensor device 51 with which it can detect its surroundings 10, in particular the current scene, also referred to as the driving situation. The information about the vehicle environment 10 obtained during the detection, for example about the objects 11 constituting the scene, can be used as a basis for controlling the vehicle 1 or at least individual vehicle functions. Such information can be used in particular by assistance systems in order to move vehicle 1 safely along route 2, i. H. without endangering the vehicle 1 or objects 11 in the environment 10 of the vehicle.

The information obtained during the acquisition can also be understood as a virtual scene representation since it characterizes the real scene. Such a virtual scene representation can be derived from sensor data generated by the sensor device 51, for example.

Since the analysis of such a virtual scene representation, for example with regard to control measures to be taken, is generally very time-consuming and complex, quality-assured route information stored in a database is expediently used to determine, and in particular adapt, the scene representation. As a result, account can be taken of the fact that usually not all objects 11 in the surroundings 10 of the vehicle 1 (can) impair the driving safety of the vehicle 1 .

For example, stationary objects such as the tree 11a, the wall 11b, the footpath 11c, or the bridge 11f do not pose any danger to the rail-bound vehicle 1. In a scene analysis, they therefore need to be given little or no attention. In other words, these stationary objects can, for example, be removed from the scene representation or at least be ignored in a risk assessment based on the virtual scene representation.

On the other hand, dynamic objects such as the birds 11d or the people 11g potentially represent a danger to the vehicle 1 - or can be endangered by the vehicle 1 - since they can move into the path of the vehicle 1 . Increased attention can therefore be paid to these dynamic objects in the scene analysis.

Likewise, objects 11 provided specifically for the operation of the vehicle 1, such as the signal system 11e, are also important for driving safety. Objects 11 that can be designated as operating objects should also be taken into account in the scene analysis.

Route information on the stationary objects 11 that are located in the vehicle surroundings 10 each time a vehicle 1 travels on route 2 or are detected by sensor device 51 each time vehicle 1 travels along route 2 can be recorded directly in the database. In the virtual scene representation can be easily checked using the route information, whether an object 11 detected in the environment 10 is stationary or dynamic - for example by checking whether its position relative to the vehicle 1 or to the route 2 is characterized by the route information. The linking of a stationary object 11 with a fixed position can be understood as a risk criterion, on the basis of which stationary and/or non-stationary objects 11 are correspondingly marked in the scene representation (i.e. provided with a corresponding "flag") or even completely removed from the scene representation scene representation can be removed. This marking can be used when evaluating driving safety—and consequently also when generating responsive control signals for controlling vehicle 1—whether and/or which objects 11 are to be taken into account or can remain unconsidered.

It is conceivable that selected types of dynamic objects 11d, 11g in the virtual scene representation are also marked as harmless or are removed from the virtual scene representation. For example, it can be assumed that the birds 11d cannot impair the driving safety of the vehicle 1. The route information can therefore contain corresponding type information that is specific to an object type. This type information thus represents generic information with which an analysis of the virtual scene representation can also be improved, in particular simplified, with regard to dynamic objects 11d, 11g.

But even potentially dangerous objects 11 such as people 11g can be excluded from a risk assessment under certain circumstances, such as in connection with FIG 2 is explained in more detail.

FIG 2 12 shows an example of scene information specific to each area 12a, 12b of a scene. The scene is characterized here by objects 11h, 11i in the environment 10 of a vehicle 1 on a route 2. In the present case, the objects are two fences 11h and two barriers 11i, which are positioned along a rail track forming the route 2 and are therefore intended to ensure the driving safety of the vehicle 1—which is a rail-bound vehicle 1 in the present case.

Both the fences 11h and the barriers 11i are stationary objects that restrict the freedom of movement of dynamic objects. Such dynamic objects, for example pedestrians, cyclists or even cars, which are located in one of the areas 12a, 12b, accordingly cannot get into the travel path of the vehicle 1. The areas 12a, 12b can therefore also be referred to as security areas 13 or protection zones. The stationary objects 11h, 11i, by which the areas 12a, 12b or safety areas 13 are defined, can accordingly also be referred to as barriers.

A route information with which a virtual image of the in FIG 2 shown scene can be adjusted, preferably contains information regarding such barriers or areas 12a, 12b. However, it is at least expedient if such information for restricting the freedom of movement of other, in particular dynamic, objects can be derived from the route information. For example, the route information can contain interaction information which characterizes the effects of a (stationary) object such as the fences 11h and/or the barriers 11i, ie in particular a barrier, on other objects. Accordingly, dynamic objects can be disregarded when assessing the driving safety of the vehicle 1 on the basis of a virtual scene representation if they are located in one of the areas 12a, 12b or safety areas 13.

It is also conceivable that objects from the (real) scene provide additional object information that supplements the route information, with which the virtual scene representation can be determined or adapted in an even more differentiated manner. In the present example, the barriers 11i could, for example, use a communication device 14a to provide object information that characterizes the opening state. The vehicle 1 can receive this object information with a corresponding communication device 14b. Depending on this object information, a decision can then be made as to whether a dynamic object in the areas 12b assigned to the barriers 11i is currently moving into the route, i. H. on route 2, is blocked or not.

FIG 3 shows an example of a system 50 for monitoring a vehicle environment 10 when a vehicle, in particular a rail vehicle, is traveling on a route. The system 50 has a sensor device 51 , a database 52 and a data processing device 53 .

In this case, sensor device 51 is set up to capture a scene from surroundings 10 of the vehicle. In this case, the sensor device 51 can, for example, detect objects 11 from the vehicle surroundings 10 . The sensor device 51 expediently generates corresponding sensor data, which can be processed by the data processing device 53, for example. The sensor device 51 expediently has an optical sensor device, for example one or more cameras, a radar device, a lidar device and/or the like.

The data processing device 53 is set up to determine a virtual scene representation of the scene detected by sensors on the basis of the sensor data. The data processing device 53 is expediently set up to recognize the objects 11 in the surroundings 10 of the vehicle as such on the basis of the sensor data. The data processing device 53 can preferably recognize the Characterize objects 11, for example, derive properties such as size, shape, texture, color, position relative to other objects and/or the vehicle, temperature and/or the like of the detected objects 11 from the sensor data. In other words, the data processing device 53 is preferably set up to determine a so-called virtual feature vector with a large number of entries for each of the objects 11 in the recorded scene, which characterizes the respective object 11 in the scene representation in detail.

The database 52 contains quality-assured route information. Such route information preferably characterizes the route on which the vehicle is driving. The route information can contain, for example, information about stationary objects that appear in the vicinity of the vehicle or are detected by sensor device 51 while the vehicle is moving along the route. The database 52 expediently contains a detailed feature vector for each object 11, by means of which the respective object 11 can be clearly identified, for example, and/or a risk to the vehicle's travel can be assessed.

Alternatively or additionally, the route information can also contain information about dynamic objects that could possibly appear in the area surrounding the vehicle, ie generic information about other road users, living beings and/or the like.

The data processing device 53 is preferably set up to adapt, for example to simplify and/or supplement, the virtual scene representation on the basis of the route information. For example, based on the route information in the surroundings 10, the data processing device 53 can highlight objects 11 recognized in the virtual scene representation as important for driving safety or mark them as insignificant.

It is also conceivable that the data processing device 53 receives additional object information from objects 11 from the real scene, which supplement the route information. Such object information can contain, for example, information about a state of the object 11, such as the open state of a barrier, the acceleration of another road user and/or the like. With this additional object information, the data processing device 53 can carry out a particularly reliable and detailed adaptation of the virtual scene representation.

The system 50 can be installed in the vehicle whose surroundings 10 are to be monitored. Alternatively, however, it is also possible for at least part of the system 50 to be arranged outside of the vehicle. In particular, the system 50 may be configured distributed between the vehicle and a land side. For example, the database 52 can be a central database in which the route information is stored on a server and can be called up, for example, via a wireless communication link. Likewise, it is also conceivable that the data processing device 53 is a central or at least external data processing device to which the vehicle can be connected via a wireless communication link.

FIG 4 shows an example of the use of quality-assured route information stored in a database 52 . In this case, an object 11e in the vehicle environment 10 is detected using a sensor device 51 of a vehicle 1 on a route 2 . Using an algorithm for object recognition, in particular an artificial intelligence, features of the object 11e—in the present case, purely by way of example, a signaling system—can be derived from the sensor data generated in the process. If necessary, background knowledge about the vehicle environment 10 can also flow into the derivation. For example, some features can be detected by image processing such as edge detection, mathematical morphology, Fourier, Hough and/or Gabor transform, wavelet transform and/or the like can be determined. Alternatively or additionally, features can be extracted from neural networks or recognized by digital text recognition (OCR). 2.5D or 3D shapes of the object 11e, possibly also of substructures and/or information about joints of the object 11e, material types, size/expansion and/or estimated weight obtained from a movement analysis can be analyzed for feature determination. It is also conceivable to determine histograms of such variables, for example substructures of the shape of the object, which may have been standardized, as a feature. The characteristics, e.g. B. size V'1, shape V'2, color V'3, position V'4 relative to route 2 and / or the like, together form a feature vector V'. The feature vector V' thus characterizes the detected object 11e.

In the present example, the route information also contains a feature vector V. The feature vector V indicates that the object 11e is located at a specific point on route 2 and is characterized by features V1, V2, V3, V4, . A comparison W of the feature vectors V′ and V can thus be used to prove, for example, that the object 11e was recognized (exactly) at the expected location. In particular, the assured quality of the route information can be transferred to the result of the object recognition or associated with the result of the object recognition. The information obtained by recognizing the object 11e can be used in further processing with the quality determined in the security verification.

Evidence can also be provided at low cost that hardware and software are working. In particular, the presence of errors, which could lead to the system malfunctioning, can be ruled out at least for a predetermined period of time. If an error budget associated with a tolerated risk is provided, an error accumulation in another part of the system can also be compensated for if necessary.

In the comparison W, it can be checked in particular whether an object type T or subtypes T1, . . . The object type T and the subtypes T1, .

An algorithm for carrying out the comparison W should be designed in accordance with the requirements of the intended security requirements and be based, for example, on mathematical and statistical theories such as probability theory, in particular conditional probabilities. Background knowledge about the vehicle environment 10 can also be included in the comparison W, for example the probability distribution of the feature values in the recorded scene, the dependency or independence of the features, the probability that features cannot be determined due to the current vehicle position, coverage by a other object, the weather or other environmental influences, technical failures in data processing, and/or the like.

Although the invention has been illustrated and described in detail by the preferred embodiments, the invention is not limited by the disclosed examples and other variations can be derived therefrom by those skilled in the art without departing from the scope of the invention.

Claims (15)

Method for monitoring a vehicle environment (10) when driving a vehicle (1), in particular a rail vehicle, on a route (2), with the steps: - Sensory detection of a scene in the vehicle environment (10) and generating corresponding sensor data; and - generating a virtual scene representation of the captured scene based on the sensor data and Quality-assured route information stored in a database (52). Method according to Claim 1, in which the route information is provided as a function of at least one travel parameter. Method according to one of Claims 1 or 2, in which the determined virtual scene representation is used as the basis for an assessment of driving safety. Method according to one of the preceding claims, wherein for objects (11, 11a-11i) identified in the detected scene, the route information is used to check whether a predetermined risk criterion is met, and the objects (11, 11a-11i) depending on a result of the Check are taken into account when determining the virtual scene representation. Method according to one of the preceding claims, wherein the route information contains type information specific to an object type and all objects (11, 11a-11i) of an object type are taken into account when determining the virtual scene representation according to the type information. A method according to any one of the preceding claims, wherein the route information includes scene information specific to an area (12a, 12b) in the detected scene and the adaptation of the virtual scene representation is carried out on the basis of the scene information. Method according to one of the preceding claims, wherein the route information contains interaction information on the interaction between at least two objects (11, 11a-11i) in the detected scene and the virtual scene representation is adapted taking into account the interaction information. Method according to one of the preceding claims, wherein a safety area (13) in the detected scene is determined on the basis of the route information, in which the risk of driving through an object (11, 11a-11i) from the scene is at least reduced. Method according to Claim 8, in which the effect of the safety area (13) on a trajectory of an object (11, 11a-11i) is determined from the scene and taken into account when adapting the virtual scene representation. Method according to one of the preceding claims, in which the route information is validated while the vehicle (1) is traveling on the route (2). 11. The method of claim 10, wherein the route information is validated based on a comparison of the temporal evolution of the captured scene with a prediction based on the route information. A method according to any one of the preceding claims, comprising: - receiving additional object information specific to an object (11, 11a-11i) from the captured scene; and - determining the virtual scene representation based on the additional object information. Method according to one of the preceding claims, wherein the route information contains lighting information and lighting-related effects are taken into account when adapting the virtual scene representation. System (50) for monitoring a vehicle environment (10) when driving a vehicle (1), in particular a rail vehicle, on a route (2). - a sensor device (51), which is set up to capture a scene and generate corresponding sensor data, - A database (52) in which quality-assured route information is stored, and - A data processing device (53) which is set up to generate a virtual scene representation on the basis of the sensor data and the route information. Vehicle (1), in particular rail vehicle, with a system (50) according to claim 14.

Images