EP3857437A1 - Procédé et dispositif d'analyse d'un flux de données de capteur et procédé de guidage d'un véhicule - Google Patents

Procédé et dispositif d'analyse d'un flux de données de capteur et procédé de guidage d'un véhicule

Info

Publication number
EP3857437A1
EP3857437A1 EP19780138.4A EP19780138A EP3857437A1 EP 3857437 A1 EP3857437 A1 EP 3857437A1 EP 19780138 A EP19780138 A EP 19780138A EP 3857437 A1 EP3857437 A1 EP 3857437A1
Authority
EP
European Patent Office
Prior art keywords
sensor data
data stream
section
template
similarity
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP19780138.4A
Other languages
German (de)
English (en)
Inventor
Pavlo TKACHENKO
Jinwei ZHOU
Luigi Del Re
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
AVL List GmbH
Original Assignee
AVL List GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by AVL List GmbH filed Critical AVL List GmbH
Publication of EP3857437A1 publication Critical patent/EP3857437A1/fr
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
    • G08G1/0125—Traffic data processing
    • G08G1/0129—Traffic data processing for creating historical data or processing based on historical data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements

Definitions

  • the present invention relates to a method and a device for analyzing a sensor data stream that characterizes a vehicle environment with regard to the presence of traffic scenarios, and a method for driving a vehicle.
  • ADAS advanced driver assistance systems
  • the support ranges from the mere display of possibly relevant information (e.g. issuing a warning by a lane change assistant) to partially autonomous interventions (e.g. regulation of the torque applied to the wheel axles by an anti-lock braking system) to fully or at least partially autonomous interventions in the control of the vehicle (eg adaptive cruise control using adaptive cruise control, ACC).
  • partially autonomous interventions e.g. regulation of the torque applied to the wheel axles by an anti-lock braking system
  • fully or at least partially autonomous interventions in the control of the vehicle eg adaptive cruise control using adaptive cruise control, ACC.
  • driver assistance systems The basis for such driver assistance systems is usually formed by sensor data, for example provided signals from ultrasound sensors, radar sensors or cameras, which can be used to determine the current driving situation and in response to this the function of the respective driver assistance system can be carried out. Particularly in the case of driver assistance systems which (autonomously) intervene in the control of the vehicle, the current driving situation must be able to be identified with the greatest reliability using the sensor data.
  • a traffic scenario in which a neighboring vehicle in front of the ego vehicle, which is equipped with the driver assistance system, can cut into the same lane by recognizing that a sensor-detected transverse distance perpendicular to the direction of travel to the neighboring vehicle decreases and finally, at least essentially, assumes the value 0 if the neighboring vehicle is immediately in front of the ego vehicle.
  • the driver assistance system to be tested can be fed with sensor data which characterize the already known traffic scenario.
  • sensor data which characterize the already known traffic scenario.
  • a large number of sensor data which may also characterize slight variations in the traffic scenario, are generally required.
  • WO 2017/210222 A1 discloses the automatic generation of simulation scenarios for validating a driver assistance system.
  • a large number of such simulation scenarios can be generated in particular by varying recorded scenarios, the variations being based on a data stream which is generated by isolating differences between similar recorded scenarios.
  • This object is achieved by a method and a device for analyzing a sensor data stream that characterizes a vehicle environment in relation to the presence of traffic scenarios and by a method for operating a driver assistance system according to the independent claims.
  • a first aspect of the invention relates to a method for analyzing a sensor data stream, which characterizes a vehicle environment, in relation to the existence of traffic scenarios, comprising the following working steps: (i) determining a similarity measure which defines the degree of agreement between indicates a section of the sensor data stream and at least one template stored in a database, by mapping the section of the sensor data stream onto the at least one template, preferably by means of dynamic time normalization, the template characterizing a known traffic scenario; and (ii) assigning the known traffic scenario to the section of the sensor data stream if the similarity measure fulfills a predefined similarity criterion.
  • the method is carried out using a computer.
  • a sensor data stream in the sense of the invention is, in particular, a chronological sequence of sensor data, in particular corresponding signals, which characterize a vehicle environment at a particular point in time.
  • a sensor data stream can in particular continuously supply information relating to the vehicle environment.
  • a sensor data stream can be provided or generated, for example, by a sensor device which preferably has one or more, possibly different, sensors for detecting the vehicle environment.
  • a sensor data stream can also be generated artificially, for example by means of a simulation.
  • the sensor data stream is preferably made up of preprocessed, in particular processed, e.g. merged, sensor data is formed and contains, for example, information regarding the relative distances between road users or other objects, in particular taking into account road curvatures.
  • a section of a sensor data stream in the sense of the invention is, in particular, a temporal section of the sensor data stream.
  • a section of a sensor data stream can be, for example, a section of the sensor data stream.
  • a section of a sensor data stream can contain sensor data that were or are provided in a time window that may be predetermined.
  • a section can in particular contain a sequence of values, in particular a sequence of values.
  • a template in the sense of the invention is in particular a sequence of values and can in particular contain a sequence of values.
  • a template can in particular represent a driving maneuver of at least one vehicle.
  • a template is preferably a, in particular generic, section of a sensor data stream which is characteristic of a specific, in particular known, traffic scenario.
  • Assigning a section of a sensor data stream to a known traffic scenario in the sense of the invention is, in particular, classifying the label, in particular the section, of the sensor data stream.
  • the section of the sensor data stream is appropriately identified during the assignment, for example by setting a marker or a value, which are each characteristic of the known traffic scenario.
  • Mapping a section of a sensor data stream to a template in the sense of the invention is, in particular, an adaptation, in particular in transforming, of the section and / or the template in such a way that the section, in particular a temporal course of the section, with the Template, in particular with a time course of the template, at least essentially, in particular as closely as possible.
  • the section and / or the template can be compressed and / or stretched during imaging, so that, for example, a form of the time sequence of values contained in the section corresponds at least substantially with a form of the time sequence of values contained in the template.
  • Dynamic time warping (DTW) in the sense of the invention is, in particular, a method, in particular an algorithm, for order of value sequences, e.g. To map time sequences like a section of a sensor data stream, possibly with different lengths.
  • a matrix is preferably generated which contains, as matrix elements, a distance, for example Euclidean distance, a Manhattan distance or a Mahalanobis distance, between individual elements of the value sequences.
  • a distance for example Euclidean distance, a Manhattan distance or a Mahalanobis distance, between individual elements of the value sequences.
  • the minimum costs for the different assignments of the individual elements of the value sequences mapped by the distances can then be determined in order to find the most precise mapping of the value sequences onto one another.
  • a measure of similarity in the sense of the invention is in particular a value that shows the similarity between two value sequences, e.g. Time sequences characterized as a section of a sensor data stream.
  • the measure of similarity is preferably based on a distance, for example a Euclidean distance, a Manhattan distance or a Mahala nobis distance.
  • the degree of similarity can be proportional to, in particular the same, the distance.
  • the similarity measure can also be inverse to the distance.
  • the degree of similarity preferably corresponds to a distance that is determined by an optimization function.
  • the similarity measure can characterize a, possibly abstract, distance between a section of a sensor data stream and a template.
  • a traffic scenario in the sense of the invention is preferably a temporal development of elements of scenes within a sequence of scenes, which begins with a start scene. In contrast to scenes, scenarios cover a certain period of time.
  • a scene preferably describes a snapshot of the surroundings, which includes all spatially stationary elements and dynamic elements.
  • the invention is based in particular on the approach of using a sensor data stream that e.g. by a sensor device or a simulator that detects a vehicle environment, compare the extracted section with a template that characterizes a known traffic scenario, for example a spatial constellation of road users and / or their dynamic development, in particular at least one driving maneuver. Depending on a result of the comparison, the section of the sensor data stream can then be classified. E.g. the section can be classified as belonging to the known traffic scenario characterized by the template.
  • information relating to the known traffic scenario can be output or provided, for example transmitted to a driver assistance system via an interface.
  • the comparison of the section of the sensor data stream with the template is preferably carried out by dynamic time normalization of the section and the template, i.e. by mapping the section onto the template.
  • a measure of similarity in particular a distance between the section and the template, can provide an indication of how well the section and the template fit together, e.g. how strongly a vehicle environment characterized by the section of the sensor data flow differs from the known traffic scenario. Does the similarity measure meet a similarity criterion, i.e. if the differences between the section and the template are not too great, the known traffic scenario characterized by the template is assigned to the section of the sensor data stream. In other words, an unknown traffic scenario characterized by the section of the sensor data stream is identified in this case with the known traffic scenario.
  • dynamic time minimization makes it possible to identify the unknown traffic scenario, which is characterized by the sensor data stream, in particular the section, particularly quickly and reliably.
  • the invention allows, on the basis of the dynamic time normalization, which is preferably carried out using an algorithm which saves computing time, for example that of Rakthanmanon et. at. in “Searching and mining trillions of time series sub-sequences under dynamic time warping”, Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 12, 262-270 (2018) only one parameter, in particular the similarity measure, is to be determined and used to reliably identify the traffic scenario.
  • the use of dynamic time standardization allows the known traffic scenario to be reliably assigned to a section of the sensor data stream that characterizes a variation of the traffic scenario, in particular by mapping it onto a template that is generic for the known traffic scenario. This can prevent the section of the sensor data stream from being classified differently in these cases. This enables efficient processing of the sensor data stream.
  • the invention makes it possible to further improve the analysis of a sensor data stream with regard to the presence of traffic scenarios, in particular to recognize existing traffic scenarios more reliably and / or in a simple manner by analyzing the sensor data stream.
  • the section of the sensor data stream is stored in the database when it is assigned to the known traffic scenario.
  • the section is preferably classified accordingly, ie marked as belonging to the known traffic scenario.
  • the section can in particular be assigned to a cluster of sections already stored in the database, which have been assigned to the known traffic scenario, or form part of such a cluster.
  • the section of the sensor data stored in this way can current in the future (further) processing of the data stored in the database, for example in the future adaptation of the at least one template.
  • the method also has the following work step: storing the section of the sensor data stream as a further template in the database if the similarity measure does not meet the predefined similarity criterion.
  • the further template can preferably be used in the following in order to be able to assign further, in particular future, sections of the sensor data stream to the traffic scenario characterized by the further template, in particular previously unknown.
  • the database can be expanded.
  • new traffic scenarios e.g. new driving maneuvers can be added.
  • learning of new traffic scenarios and a corresponding dynamic expansion of the database can be realized in this way.
  • a rare traffic scenario occurs, for example a vehicle executes an unusual maneuver that has not yet been or has not been recorded by a template in the database. Then the section of the sensor data stream and this rare and hitherto unknown traffic scenario can be characterized, which does not match any of several templates stored in the database, at least within the framework specified by the similarity criterion, i.e. in particular not precise enough, in particular the dynamic time normalization, on which one of the templates can be mapped, saved as a new template in the database and thus, for example, form a new traffic scenario class.
  • the similarity criterion i.e. in particular not precise enough, in particular the dynamic time normalization
  • the method further comprises the following steps: (i) checking whether at least one template is stored in the database; and (ii) adding the portion of the sensor data stream as a template to the database depending on the result of the test.
  • the section of the sensor data stream is stored in the database if no template has been or has not yet been stored in the database.
  • the database can advantageously be initialized or set up. In particular, this can eliminate the need for an expensive preparation or population of the database.
  • the method of Furthermore, the following work step: Adapting the template if the similarity measure simultaneously fulfills the predefined similarity criterion and a predefined adaptation criterion.
  • the template can be adapted if the section of the sensor data stream is assigned to the known traffic scenario, which is characterized by the template, but there are certain differences between the section and the template.
  • sections of the sensor data stream can be assigned to the correct traffic scenario in a particularly reliable manner.
  • the template can be adapted, in particular by recalculation, if, based on a distance between the section of the sensor data stream and the template determined by the dynamic time normalization, the section can be assigned to a cluster of sections corresponding to the known traffic scenario, but not in the vicinity of the The focus of the cluster is.
  • all sections of the sensor data stream which are associated with the known traffic scenario, in particular previously stored in the database are averaged by means of dynamic time standardization.
  • the averaging is preferably carried out with e.g. from Petitjean et. al. in “A global averaging method for dynamic time warping, with applications to clustering”, Pattern regocnition, 44 (3), 678-693 (201 1) known center of gravity averaging methods of dynamic standardized time sequences (engl dynamic time warping barycenter averaging). This avoids iterative pairwise averaging, which advantageously reduces the computational effort.
  • an averaging can be used in any order of the sensor data contained in the stored sections of the sensor data stream, and the, in particular temporal, length of the template adapted in this way is advantageously not increased.
  • the method thus preferably forms a real-time version of the so-called k-means algorithm (k-means clustering), the templates stored in the database forming focal points of various clusters, each of which corresponds to a known traffic scenario.
  • a new member ie a further section of the sensor data stream, can be added to a cluster if the similarity criterion is met.
  • the adaptation criterion is also fulfilled at the same time, for example the mapping of the section of the sensor data stream onto the template 1
  • the focus of the corresponding cluster, ie the template is adjusted, in particular recalculated, in particular by averaging all members of the cluster.
  • the section of the sensor data stream If the similarity criterion is not already met, ie if the section of the sensor data stream cannot be meaningfully mapped to one of the templates, in particular by dynamic time standardization, the section forms a new cluster.
  • the method By designing the method as a k-means algorithm, sections of the sensor data stream can be assigned to the correct traffic scenario particularly reliably, in particular with high probability, or the database can be dynamically adapted or expanded.
  • the predefined similarity criterion is or is met if the similarity measure falls below a predefined similarity threshold value. It is preferably checked whether a distance between the section of the sensor data stream and the template determined by dynamic time normalization is smaller than the predetermined similarity threshold. This makes it possible to make a reliable and unambiguous decision, in particular when saving computing power, whether the section of the sensor data stream is to be assigned to the known traffic scenario, which is characterized by the template.
  • the predefined adaptation criterion is or is met if the similarity measure exceeds an adaptation threshold value which is dependent on the similarity criterion, in particular on the similarity threshold value.
  • the template can be adapted in particular if the degree of correspondence between the section of the sensor data stream and the template is not too high.
  • an adaptation of the template is preferably only carried out if the portion of the sensor data stream currently mapped onto the template deviates from the template at least to a certain extent and thus an adaptation of the Stencil is also attached. In particular, unnecessary adaptation of the templates can be avoided in this way.
  • the dependency of the adaptation criterion, in particular the adaptation threshold value is preferably characterized by a function, in particular a mathematical function.
  • the value of this function can indicate the adaptation criterion, in particular the adaptation threshold, if at least the similarity criterion, in particular the similarity threshold, is selected as the input variable of the function.
  • the function can be set up to weight the similarity criterion, in particular the similarity threshold, e.g. by multiplying by a factor less than one.
  • the adaptation criterion can correspond to a weighting of the similarity criterion.
  • the method also has the following work step: presetting the similarity criterion, in particular the similarity threshold value, on the basis of a comparison of several, in particular all, similarity measures with one another, which are used in the mapping, in particular by dynamic time normalization, at least of the known scenario assigned sections of the sensor data stream, in particular stored in the database.
  • a similarity measure characterizing the lowest degree of correspondence between the portion of the sensor data stream and the template e.g. a maximum distance determined by the dynamic time normalization between the section of the sensor data stream and the template is determined and the similarity criterion, in particular the similarity threshold value, is formed therefrom.
  • the similarity criterion, in particular the similarity threshold value can be dynamically adapted, for example, to the quality of the sensor data contained in the sensor data stream.
  • all members of a cluster corresponding to a known traffic scenario can be mapped onto the template by dynamic time standardization.
  • the maximum distance is selected, for example filtered out, from the distances determined between the members, ie sections of the sensor data stream and the template, and used as a similarity criterion, in particular a similarity threshold value.
  • several similarity measures are determined by mapping the section of the sensor data stream onto one of several templates stored in the database, each of which characterizes a different known traffic scenario, and the section of the sensor data stream is based on the known traffic scenario a comparison of the several determined similarity measures with each other.
  • an unknown traffic scenario which is characterized by the section of the sensor data stream, is compared with several already known traffic scenarios and is preferably identified on the basis of this comparison.
  • a similarity measure that characterizes the highest degree of correspondence between the section of the sensor data stream and a template e.g. a minimum distance between the section of the sensor data stream and the template determined during the dynamic time normalization of the section of the sensor data stream is determined and the section is assigned to the known traffic scenario that is characterized by the corresponding template.
  • the traffic scenario which is currently characterized, in particular mapped, by the section of the sensor data stream, can be identified reliably and quickly.
  • the sensor data stream characterizes at least one, in particular transverse, i.e. perpendicular to the direction of travel, distance between two road users in the traffic scenario. If, for example, two vehicles drive alongside one another in two adjacent lanes, the distance between the vehicles does not change. If one of the vehicles performs a driving maneuver, for example by accelerating or overtaking or by swiveling into the other lane, the distance between the vehicles changes. This change can be characteristic of the driving maneuver or the traffic scenario. The, in particular temporal, course of the distance during this driving maneuver can thus form a template for the traffic scenario stored in the database. A section of the sensor data stream can thus be assigned to the correct traffic scenario with a high degree of probability.
  • the sensor data stream is provided by a sensor device of a vehicle during the operation of the vehicle.
  • the sensor device preferably has a plurality of sensors, for example at least one camera, at least one radar sensor, at least one song sensor, at least one ultrasound sensor and / or the like, which generate corresponding sensor data, preferably at least essentially continuously, wherein the sensor data stream is preferably formed by a fusion of the sensor data. Based on the assignment of the section of the sensor data stream to the known traffic scenario, information relating to the traffic scenario identified in this way can then be output in situ, ie essentially in real time, and made available, for example, to a driver assistance system.
  • the database can also be expanded in situ, ie in real time.
  • the method further comprises the following work step: selecting the section from the sensor data stream, a start of the section of the sensor data stream and / or an end of the section of the sensor data stream being selected such that the start of the Section of the sensor data stream and the end of the section of the sensor data stream are spaced apart from one another by a predetermined period of time.
  • the section of the sensor data stream selected in this way preferably forms a time window within which sensor data contained in the sensor data stream, for example provided by a sensor device of a vehicle or a simulation, is taken into account, for example when identifying a present traffic scenario.
  • the duration i.e. by choosing the length of the time window, a particularly reliable assignment of the section of the sensor data stream to the known traffic scenario can be achieved.
  • the end of the section is preferably formed from sensor data currently generated by the sensor device or the simulation. This enables, for example, a reliable identification of the current traffic situation.
  • at least the selection of the section of the sensor data stream, the determination of the similarity measure and the assignment of the section of the sensor data stream are carried out repeatedly.
  • the section of the sensor data stream is repeatedly selected, the beginning and / or the end of the section of the sensor data stream is or are selected such that the section of the sensor data stream overlaps at most only half with a previously selected section of the sensor data stream.
  • a further section of the sensor data stream is only mapped onto at least one template when half of the predetermined time period has elapsed. This makes it possible to ensure that two sections of the sensor data stream which are successively mapped onto a plurality of templates differ sufficiently from one another in order to be able to be assigned to different known traffic scenarios.
  • a second aspect of the invention relates to a device for analyzing a sensor data stream, which characterizes a vehicle environment, with respect to the presence of traffic scenarios.
  • the device preferably has a processing module which is set up to map a similarity measure, which indicates the degree of correspondence between a section of the sensor data stream and at least one template stored in a database, to the section of the sensor data stream to determine at least one template, preferably by means of dynamic time standardization, the template characterizing a known traffic scenario.
  • the device preferably has an assignment module which is set up to assign the known traffic scenario to the section of the sensor data stream if the similarity measure fulfills a predefined similarity criterion.
  • the device preferably also has a sensor device which is set up to detect a vehicle environment and to provide the sensor data stream.
  • the sensor device can have one or more sensors, for example cameras, ultrasound sensors, radar sensors, lidar sensors and / or the like, in order to be able to reliably, preferably redundantly, detect relevant variables, such as distances between road users, for characterizing an existing traffic scenario.
  • the device preferably also has a database which is set up to store at least one template, preferably a plurality of templates, each of which characterizes or characterizes a known traffic scenario.
  • a third aspect of the invention relates to a method for driving a vehicle based on a sensor data stream by means of a driver assistance system, the sensor data stream being analyzed by means of the method according to the first aspect of the invention.
  • the section of the sensor data stream which preferably characterizes a current traffic scenario
  • the known traffic scenario which is characterized by the template stored in the database
  • an output signal is generated which is the known one or traffic scenario identified using the section of the sensor data stream, in particular a known maneuver, is characterized and made available to the driver assistance system.
  • the driver assistance system can thus react reliably to the existing traffic scenario.
  • FIG. 1 shows a preferred embodiment of a device according to the invention
  • FIG. 3 shows a representation to explain an assignment of a known traffic scenario, which is characterized by a template, to a section of a sensor data stream;
  • Fig. 4 is an illustration for explaining an adaptation of a template.
  • 1 shows a preferred embodiment of a device 1 according to the invention for analyzing a sensor data stream D, which characterizes a vehicle environment, with a processing module 2, an assignment module 3 and a database 4.
  • the device 1 is preferably with a sensor device 5, which is used to detect a vehicle environment and provision of a corresponding sensor data stream D is set up and connected to a driver assistance system 6.
  • the driver assistance system 6 can be controlled, for example, on the basis of an output signal A, which is generated by the device 1 on the basis of the analysis of the sensor data stream D.
  • the sensor device 5 and / or the driver assistance system 6 is or are part of the device 1.
  • the processing module 2 and / or the assignment module 3 are preferably software, e.g. as program code, and can be executed by means of a data processing unit 7.
  • the processing module 2 is preferably set up to determine a similarity measure which characterizes the degree of agreement between a section of the sensor data stream D and at least one template S stored in the database 4.
  • the processing module 2 can provide the section of the sensor data stream D, for example by extracting sensor data contained in the sensor data stream D during a predetermined time window, and for example by applying a method for dynamic time normalization (DTW) to the at least one map a template S.
  • DTW dynamic time normalization
  • the similarity measure is preferably obtained as a result of this mapping.
  • the processing module 2 can in particular be set up to distort the section extracted from the sensor data stream S dynamically, ie in particular non-linearly, for example to compress and / or stretch it at least in sections.
  • the sensor data contained in the section can be assigned, for example, corresponding template data corresponding to the template.
  • the processing module 2 preferably determines the assignment on the basis of an optimization function, in particular in such a way that the section of the sensor data stream D is particularly precise, ie with minimal deviations, on the template S is mapped.
  • a remaining distance, in particular in the form of a difference, between the section of the sensor data stream D and the template S preferably forms the similarity measure.
  • the distance can in particular characterize the differences between the sensor data contained in the section and the template data assigned to them.
  • the differences between sensor and template data for a pair of sensor data and template data are determined and added up in order to obtain the distance.
  • the templates S stored in the database 4 preferably characterize a known traffic scenario.
  • the template data corresponding to a template can correspond, for example, to values, in particular the time profile, of a parameter that at least partially describes the traffic scenario.
  • a parameter can be, for example, the distance between two road users.
  • the assignment module 3 can accordingly be set up to assign a known traffic scenario to the section of the sensor data stream D on the basis of the templates S stored in the database 4 and the similarity measure determined by the processing module 2, in particular if the similarity measure fulfills a predetermined similarity criterion, e.g. is smaller than a predefined similarity threshold.
  • the assignment module 3 is preferably set up to check whether the determined degree of similarity fulfills the predetermined similarity criterion.
  • the processing module 2 can determine whether the distance obtained in the dynamic time normalization of the section of the sensor data stream D between the section of the sensor data stream D and the template S falls below a predetermined similarity threshold, i.e. whether the section and the template S are so similar to each other that the determined distance between them is smaller than the similarity threshold.
  • the assignment module 3 can assign the section of the sensor data stream D to a known traffic scenario, which is characterized by the template S.
  • the output signal A is preferably generated by the assignment module 3, preferably also as a function of the result of the test, and characterizes the known traffic scenario.
  • the output signal A can contain, for example, information relating to the known traffic scenario.
  • the database 4 can be expanded with a further template S, which is formed in particular by the section of the sensor data stream D that is currently being considered.
  • the section can be stored as a further template S in the database 4.
  • FIG. 2 shows a preferred exemplary embodiment of a method 100 according to the invention for analyzing a sensor data stream that characterizes a vehicle environment with respect to the presence of traffic scenarios.
  • the sensor data stream is made available, for example by detecting a vehicle environment with a sensor device of a vehicle or by simulation.
  • the sensor data stream is or is preferably formed from a time sequence of sensor data, it being possible for the sensor data to contain values of a parameter which describes a traffic scenario.
  • the sensor data stream can depict the time course of distances between road users.
  • a section is selected from the sensor data stream, for example by extracting the sensor data from the sensor data stream within a predetermined time window.
  • a beginning of the section and / or an end of the section can be selected, the beginning and the end of the section preferably being spaced apart from one another by a predetermined period of time.
  • the end of the section is preferably formed from the sensor data stream from the last provided sensor data.
  • the section of the sensor data stream can be mapped to the at least one template stored in the database in a further method step S5, a similarity measure preferably being determined.
  • the similarity measure preferably characterizes a degree of agreement between the section of the sensor data stream and the template.
  • the mapping of the section onto the template can be carried out on the basis of a dynamic time standardization, the similarity measure preferably being formed by a distance between the section and the template obtained by the dynamic time standardization.
  • the distance can be a measure of the difference between the section imaged on the template and the template, in particular between sensor data contained in the section and template data contained in the template.
  • the distance or the similarity measure is accordingly preferably small if the section and the template are very similar to one another, or large if the section and the template are dissimilar.
  • the section is preferably mapped onto each of the templates and a similarity measure is determined in each case in this way.
  • a further method step S6 it is preferably checked whether the similarity measure or the similarity measures ascertained in method step S5 meets or fulfill a predefined similarity criterion, for example whether the distance obtained on the basis of the dynamic time normalization is smaller than a predefined similarity threshold value .
  • the section of the sensor data stream can be stored in method step S4 as a further template in the database, in particular in addition to templates already stored in the database.
  • the section of the sensor data stream thus represents another, previously unknown, traffic scenario. If, on the other hand, at least one similarity measure meets the similarity criterion, for example the distance obtained on the basis of the dynamic time normalization is smaller than the predetermined similarity threshold value, a known traffic scenario can be assigned to the section of the sensor data stream in a further method step S7.
  • the section is assigned a known traffic scenario on the basis of a result of the comparison. For example, when comparing, that of the similarity measures can be determined which characterizes the highest degree of agreement between the section and the template corresponding to the similarity measure. In particular, the smallest of the distances obtained in the dynamic time normalization can be selected and the traffic scenario characterized by the corresponding template can be assigned to the section.
  • a further method step S8 it can be checked whether the similarity measure, optionally selected from several similarity measures determined in method step S5 as described above, also fulfills a predefined adaptation criterion in addition to the similarity criterion, for example whether the distance obtained in the dynamic time normalization is greater as an adjustment threshold.
  • the template corresponding to the similarity measure can be adapted in a further method step S9, taking into account the section of the sensor data stream.
  • the template can be corrected by averaging several sections of the sensor data stream, to which the corresponding known traffic scenario was assigned in method step S7.
  • FIG. 3 shows an illustration to explain an assignment of a known traffic scenario, which is characterized by a template S, to a section B of a sensor data stream.
  • the traffic scenario is, for example, a single maneuver, a maneuver out of maneuver, driving in a row in one lane or driving between two lanes.
  • Different sections B correspond, for example, to and within different driving maneuvers Driving maneuvers also with different versions or variations of the respective driving maneuver.
  • the position of the sections B of the sensor data stream shown in FIG. 3 depends on a transverse start distance dX start for each of the sections, ie the component of the distance between two vehicles perpendicular to the direction of travel at the start of the driving maneuver, and a transverse end distance dX end , ie the component of the distance between two vehicles perpendicular to the direction of travel at the end of the driving maneuver.
  • sections B are shown in FIG. 4, which correspond to a maneuver in which two vehicles first drive in two adjacent lanes and one of the vehicles reevers in front of the other vehicle.
  • FIG. 4 correspond to a maneuver in which two vehicles first drive in two adjacent lanes and one of the vehicles reevers in front of the other vehicle.
  • the sections B in the selected representation form cluster C, the templates S in the example shown each forming the center of gravity of a cluster C from a plurality of sections B of the sensor data stream.
  • a cluster C is preferably given all those sections B of the sensor data stream whose temporal profiles (shown in FIG. 4, for example) have a certain similarity to one another.
  • this similarity is illustrated by the spatial position, ie the similar start and end distances dX start , dX end , of the sections B, wherein similar sections B of the sensor data stream are each within a range of one another.
  • all sections B which correspond to different versions of the single maneuver, are in a range around dX start ⁇ 4 and dX end ⁇ 0, while all sections B, which correspond to different versions of a subsequent maneuvers, with one vehicle following another in the same lane, are in a range around dX start ⁇ 0 and dX end ⁇ 0.
  • a cluster C is preferably assigned all those sections B of the sensor data stream for which a determined similarity measure, which characterizes the degree of agreement between the respective section B and the template S of the cluster C, fulfills a similarity criterion.
  • the degree of similarity is preferably determined by mapping the respective section B onto the respective template S using dynamic time normalization.
  • the section B is preferably so dynamic when mapping to the respective template S by the dynamic time normalization, i.e. at least in sections, compressed and / or stretched, that it at least essentially corresponds to the template S or follows its course.
  • the temporal course of a parameter such as the transverse distance between two vehicles, which at least partially characterizes the still unknown traffic scenario to be identified and is mapped by section B of the sensor data stream, can be determined by the dynamic time normalization at the time mapped by template S for the known traffic scenario characteristic course of this parameter can be adjusted.
  • the similarity measure preferably specifies the deviations between the time profile shown by section B and the time profile shown by template S (for examples of the time profile of section B, reference is made to FIG. 4).
  • one of the templates S or one of the clusters C and thus one of the known traffic scenarios can be assigned to the further section B. That of the known traffic scenarios is preferably assigned to the further section B, for the template S of which the highest degree of correspondence with the section B has been determined.
  • the degree of similarity which indicates the degree of correspondence, can correspond, for example, to a distance between the section B of the sensor data stream and the respective template S that was determined as part of the dynamic time normalization. Fulfilling the similarity criterion can include, for example: the distance falls below a similarity threshold. Alternatively or additionally, the fulfillment of the similarity criterion can include: the determined distance is smaller than all other distances determined with respect to the other templates S.
  • a section B is preferably assigned to a cluster C or the driving maneuver or traffic scenario corresponding thereto if the distance between section B and template S corresponding to cluster C is sufficiently small and in particular smaller than the distances between section B and all of them other templates S is.
  • the similarity measure fulfills a given adaptation criterion. If this is the case, the corresponding template S, which characterizes the known traffic scenario assigned to section B, is preferably adapted. This is also described below in connection with FIG. 4.
  • the adaptation criterion is preferably met if the distance between the section B and the template S determined by the dynamic time standardization is greater than a predetermined adaptation threshold value.
  • the adaptation criterion is met if a section B is assigned to a cluster C, but not close to the center of gravity of the cluster C, i.e. the template S, lies.
  • the differences between the section B and the template S can be regarded as sufficient that an adaptation of the template S, i.e. a recalculation of the center of gravity of cluster C leads to a change that has to be taken into account, which in particular can have an influence on future comparisons with further sections B of the sensor data stream.
  • FIG. 4 shows an illustration for explaining an adaptation of a template S.
  • the time profile of a transverse distance dX between two vehicles ie the distance between the vehicles perpendicular to their direction of travel, is shown.
  • the transverse distance dX can be used as a parameter that at least partially describes a traffic scenario, in particular a one-maneuver.
  • two vehicles are located on adjacent lanes, so that the transverse distance between them is, for example, approximately 4 m.
  • the two vehicles are in the same lane, so that the transverse distance between them is essentially 0 m.
  • section B shows a multiplicity of sections B of a sensor data stream which contains a chronological sequence of sensor data which characterize the transverse distance dX in the same traffic scenario in each case, in this case a one-maneuver.
  • section B is shorter or longer. Regardless of this, however, the forms of the temporal course are at least similar.
  • This similarity can be used to assign a known traffic scenario, for example the single maneuver, to a section B of the sensor data stream, with section B using to determine a similarity measure between section B and a template S which characterizes the known traffic scenario dynamic time normalization is shown on template S. This is explained in detail in connection with FIGS. 2 and 3.
  • the template S is preferably determined on the basis of a plurality of sections B of the sensor data stream, for example previously stored in a database.
  • the sections B previously stored in the database which correspond to the known traffic scenario, here the one-maneuver, can e.g. during test drives with a vehicle and recorded accordingly, e.g. manually, from the sensor data stream.
  • sections B can also be sections extracted in situ from the sensor data stream and stored in the database.
  • averaging is preferably carried out on the basis of dynamic time normalization.
  • a center of gravity averaging method is dynamically standardized time sequences (Engl dynamic time warping barycenter averaging) applied, which for example in Petitjean et. al. in “A global averaging method for dynamic time warping, with applications to clustering”, Pattern regocnition, 44 (3), 678-693 (2011).
  • the averaging of the sections B of the sensor data stream shown in FIG. 4 results in the template S also shown in FIG. 4, which is generic in relation to the traffic scenario characterized by it, in this case the one-maneuver.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Traffic Control Systems (AREA)
  • Control Of Driving Devices And Active Controlling Of Vehicle (AREA)

Abstract

La présente invention concerne un procédé et un dispositif d'analyse d'un flux de données de capteur qui caractérise l'environnement d'un véhicule, au sujet de la présence de scénarios de trafic, ainsi qu'un procédé de guidage d'un véhicule. Une mesure de similitude qui indique le degré de concordance entre une partie du flux de données de capteur et au moins un modèle stocké dans une base de données est déterminée par reproduction de la partie du flux de données de capteur sur le ou les modèles, de préférence au moyen d'une normalisation temporelle dynamique Selon l'invention, le modèle caractérise un scénario de trafic connu. Le scénario de trafic connu est associé à la partie du flux de données de capteur lorsque la mesure de similitude satisfait à un critère de similitude prédéterminé.
EP19780138.4A 2018-09-24 2019-09-24 Procédé et dispositif d'analyse d'un flux de données de capteur et procédé de guidage d'un véhicule Pending EP3857437A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
ATA50818/2018A AT521724A1 (de) 2018-09-24 2018-09-24 Verfahren und Vorrichtung zur Analyse eines Sensordatenstroms sowie Verfahren zum Führen eines Fahrzeugs
PCT/AT2019/060315 WO2020061603A1 (fr) 2018-09-24 2019-09-24 Procédé et dispositif d'analyse d'un flux de données de capteur et procédé de guidage d'un véhicule

Publications (1)

Publication Number Publication Date
EP3857437A1 true EP3857437A1 (fr) 2021-08-04

Family

ID=68109071

Family Applications (1)

Application Number Title Priority Date Filing Date
EP19780138.4A Pending EP3857437A1 (fr) 2018-09-24 2019-09-24 Procédé et dispositif d'analyse d'un flux de données de capteur et procédé de guidage d'un véhicule

Country Status (7)

Country Link
US (1) US12067866B2 (fr)
EP (1) EP3857437A1 (fr)
JP (1) JP2022502750A (fr)
KR (1) KR20210063409A (fr)
CN (1) CN112771535A (fr)
AT (1) AT521724A1 (fr)
WO (1) WO2020061603A1 (fr)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11886195B2 (en) * 2020-11-30 2024-01-30 Woven By Toyota, U.S., Inc. Active learning and validation system for vehicles
CN112805724B (zh) * 2021-01-08 2022-05-13 华为技术有限公司 车辆驾驶场景识别方法及装置
GB2618341B (en) * 2022-05-03 2024-09-04 Oxa Autonomy Ltd Controlling an autonomous vehicle
EP4300316A1 (fr) * 2022-07-01 2024-01-03 dSPACE GmbH Procédé mis en uvre par ordinateur et système de génération d'une bibliothèque de scénarios

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160362118A1 (en) * 2011-08-31 2016-12-15 Pulsar Informatics, Inc. Driver performance metric
CN104508719B (zh) * 2012-07-17 2018-02-23 日产自动车株式会社 驾驶辅助系统以及驾驶辅助方法
KR101473957B1 (ko) * 2013-01-28 2014-12-18 군산대학교산학협력단 운전 패턴 인식에 기반하여 보험료를 결정하는 장치 및 방법
CN105389984A (zh) * 2015-11-16 2016-03-09 北京智视信息科技有限公司 一种基于移动终端传感信息融合的驾驶行为识别方法
DE102016205152A1 (de) * 2016-03-29 2017-10-05 Avl List Gmbh Fahrerassistenzsystem zum Unterstützen eines Fahrers beim Führen eines Fahrzeugs
US11137763B2 (en) 2016-05-30 2021-10-05 Faraday & Future Inc. Generating and fusing traffic scenarios for automated driving systems
EP3619649A4 (fr) * 2017-05-05 2021-03-17 Arimo, LLC Analyse de données de séquences à l'aide de réseaux neuronaux
CN107358678B (zh) * 2017-06-07 2020-11-27 北京航空航天大学 一种驾驶事件检测和识别方法
US10401862B2 (en) * 2017-10-31 2019-09-03 Waymo Llc Semantic object clustering for autonomous vehicle decision making
JP2019095878A (ja) * 2017-11-20 2019-06-20 株式会社デンソー 車両用検索装置
ES2736901A1 (es) * 2018-06-29 2020-01-08 Geotab Inc Caracterización de una colisión de vehículo

Also Published As

Publication number Publication date
US12067866B2 (en) 2024-08-20
KR20210063409A (ko) 2021-06-01
JP2022502750A (ja) 2022-01-11
AT521724A1 (de) 2020-04-15
CN112771535A (zh) 2021-05-07
WO2020061603A1 (fr) 2020-04-02
US20220230536A1 (en) 2022-07-21

Similar Documents

Publication Publication Date Title
WO2020061603A1 (fr) Procédé et dispositif d'analyse d'un flux de données de capteur et procédé de guidage d'un véhicule
DE102017205093A1 (de) Verfahren und System zur Vorhersage von Sensorsignalen eines Fahrzeugs
EP3393875B1 (fr) Procédé de détection améliorée d'objets au moyen d'un système d'aide à la conduite
DE102019126195A1 (de) Verfahren zur effizienten, simulativen Applikation automatisierter Fahrfunktionen
DE102019208233A1 (de) Verfahren und Vorrichtung zum automatischen Ausführen einer Steuerfunktion eines Fahrzeugs
DE102021200889A1 (de) Verfahren zum Schätzen eines Fahrzeugparameters für den Betrieb eines Fahrzeugs
DE102017128082A1 (de) Meta-Architektur-Design für ein CNN-Netzwerk
DE102017208692A1 (de) Verfahren zum Bereitstellen von Trainingsdaten für eine Funktionsprüfung einer Erkennungseinrichtung sowie Datenbanksystem
DE102019129904A1 (de) Automatische Fahrkompetenzanalyse
DE102018122115A1 (de) Verfahren zur Umgebungserfassung eines Fahrzeugs
DE102019218127B4 (de) Verfahren und Vorrichtung zum optimalen Bereitstellen von KI-Systemen
DE102021133977A1 (de) Verfahren und System zur Klassifikation von Szenarien eines virtuellen Tests sowie Trainingsverfahren
DE102021107675A1 (de) System und Verfahren zum Implementieren von intelligenten Fahrzeugfunktionen
DE102021202334A1 (de) Filterung von betriebsszenarien im betrieb eines fahrzeugs
WO2021122339A1 (fr) Procédé et dispositif permettant de fabriquer un réseau neuronal plus robuste par rapport à des perturbations antagonistes
DE102019119084A1 (de) Bestimmen eines Signalstatus einer Lichtsignalanlage
DE102019130484A1 (de) Verfahren und Vorrichtung zum Anlernen eines Ensembles von neuronalen Netzen
DE102019219927A1 (de) Verfahren und Vorrichtung zum Erkennen einer Entfremdung einer Sensordatendomäne von einer Referenzdatendomäne
DE102019004842A1 (de) Verfahren zum Betreiben eines wenigstens teilweise automatisierten Fahrzeugs
DE112021007341T5 (de) Steuervorrichtung und Steuerverfahren
WO2022199916A1 (fr) Procédé d'évaluation de logiciel pour un dispositif de commande d'un véhicule
DE102022131849A1 (de) Verfahren zum Bereitstellen einer Fahrseiteninformation
DE102022211839A1 (de) Bestimmung einer Höheninformation für ein Objekt im Umfeld eines Fahrzeugs
DE102021131868A1 (de) Zuverlässigkeitsüberprüfung einer Objektklassifizierung
DE102022116054A1 (de) Verfahren und Vorrichtung zur Erkennung und/oder zur Nachverfolgung von Objekten

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20210426

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20230206

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230503