EP2771873B1 - Grid-based environmental model for a vehicle - Google Patents
Grid-based environmental model for a vehicle Download PDFInfo
- Publication number
- EP2771873B1 EP2771873B1 EP12787353.7A EP12787353A EP2771873B1 EP 2771873 B1 EP2771873 B1 EP 2771873B1 EP 12787353 A EP12787353 A EP 12787353A EP 2771873 B1 EP2771873 B1 EP 2771873B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- grid
- vehicle
- analyzing
- data
- control unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000007613 environmental effect Effects 0.000 title claims description 8
- 238000000034 method Methods 0.000 claims description 32
- 238000007906 compression Methods 0.000 claims description 20
- 230000006835 compression Effects 0.000 claims description 20
- 230000005540 biological transmission Effects 0.000 claims description 17
- 238000001514 detection method Methods 0.000 claims description 5
- 230000002123 temporal effect Effects 0.000 claims description 5
- 238000013144 data compression Methods 0.000 claims description 2
- 238000011156 evaluation Methods 0.000 description 18
- 230000006870 function Effects 0.000 description 9
- 230000015572 biosynthetic process Effects 0.000 description 4
- 238000005259 measurement Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 238000005034 decoration Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
- G08G1/161—Decentralised systems, e.g. inter-vehicle communication
- G08G1/163—Decentralised systems, e.g. inter-vehicle communication involving continuous checking
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
Definitions
- the invention is in the technical field of environmental detection with sensors in a vehicle.
- the sensor data are processed in a suitable manner in order to provide an environment model for further applications, in particular driver assistance functions.
- Prior art is e.g. a grid-based environment model.
- the environment of a vehicle is subdivided into cells, and for each cell a feature or several characteristics for the environment description is stored.
- object-based methods for environment modeling which provide the position data of detected objects, belong to the state of the art.
- Grid-based methods are that the whole environment of the vehicle is described.
- the information from sensor data for free and object-occupied and unknown areas is made available instead of only for areas occupied with objects (object-based methods) in the environment model.
- the explicit modeling of free areas is becoming more important. Since many newer assistance functions, such as an evasion assistant, information about a space that is usable as a maneuver room for the vehicle need.
- Grid-based methods for describing the environment usually require a larger amount of data than object-based methods, and thus an application in the vehicle requires larger memory resources and transmission bandwidths.
- the WO 2007/028 932 A1 discloses a method for supporting a vehicle controller using a grid based environment model.
- the DE 10 2007 012 458 A1 discloses a method for object formation in environment modeling.
- the DE 10 2010 006 828 A1 discloses a method for creating an environment model for a vehicle.
- the DE 10 2007 013 023 A1 discloses a grid-based environment model.
- a method for a sensor system for environment detection for a motor vehicle wherein a grid-based environmental model is calculated.
- a grid-based environment model is based on dividing the environment of a vehicle into cells and storing for each cell a characteristic descriptive of the environment. Storing sensor raw data or storing a classification for each cell as a probability value, eg storing the probability that a cell is occupied or not occupied, requires a high storage capacity, which also requires a bus system with high bandwidth in a transmission from or to a control unit.
- each grid cell is assigned at least one discrete value (class).
- the discrete value or the class is a measure of whether an object is at the position that is represented by the grid cell and whether this object is traversable or with what probability the object is traversable.
- the discretization or renegotiation of a class takes place by the evaluation of environment data which contain statements about detected environment objects, and at least one threshold value.
- the number of thresholds can be defined as desired and influences the number of possible classes. With a threshold value a maximum of two discrete values or classes, with two threshold values a maximum of three discrete values or classes can be limited.
- a lossless compression method is applied to the discrete values of a grid, in particular prior to transmission via a data transmission system in the vehicle. This includes grid values that were compressed before being transmitted.
- compression is achieved by decorrelation of the temporal dependency, i. achieved by a difference of temporally successive discrete values of the grid cells, a compression of the data.
- difference values are transmitted via a data transmission system in the vehicle.
- a lossy compression method is applied to the discrete values of a grid, in particular prior to transmission via a data transmission system in the vehicle.
- the discrete values of the grid which represent areas further from the vehicle, are more compressed.
- the discrete values of the grid are compressed more strongly.
- the back of a construction site wall, which is arranged facing away from the vehicle more compressed than the front of the site wall, which is arranged facing the vehicle.
- a particularly computationally efficient embodiment of the invention provides that the assignment of the discrete value or the class on the basis of the environment data is carried out with the aid of an allocation table, wherein the allocation table in the memory of the evaluation is deposited.
- the change of the content of a plurality of grid cells, the Verkdnungstabelle changed.
- the changed assignment rule which assigns a changed value range of the environment data to a discrete value or a class, allows the content of all cells to be changed at once without recalculating and overwriting the discrete values in each grid cell.
- the environment model is transmitted based on discrete values in the grid via a data transmission system in a vehicle to an evaluation or control unit.
- the data transmission system is preferably a bus system in the vehicle, which connects at least two evaluation or control units.
- an evaluation unit creates the grid-based environment model and a further evaluation or control unit uses the environment model for controlling a driver assistance function.
- the invention claimed here comprises a sensor system for object detection for a vehicle having a first computing and evaluation unit on which a method as described above is stored.
- a second evaluation or control unit and a data transmission system is provided, wherein via the data transmission system, the first is connected to the second evaluation or control unit in a vehicle.
- the first evaluation or control unit for creating an environment model and the second evaluation or control unit for controlling a driver assistance system is provided.
- a grid-based environment model is based on dividing the environment of a vehicle into cells and storing for each cell a characteristic descriptive of the environment. Saving sensor raw data or storing a classification for each cell as a probability, eg the probability that a cell is busy or unoccupied, requires a high storage capacity, which also requires a high bandwidth bus system when transferred to or from a controller. Direct application of a compression method to a calculated grid often does not result in a high compression factor since the probabilities of neighboring cells often differ only minimally. An example is such a grid is in FIG. 1 shown on the left.
- a use of the environment data, in particular in a vehicle for a driver assistance system usually requires a binary decision, is decided on the basis of a threshold for the probability of whether the cell is occupied and thus not overridden or free for a vehicle and thus overridden for a vehicle , Of importance, therefore, are discrete decision classes, e.g. with a numerical value or similar state the states occupied / free for a grid cell.
- the generation of the environment model with the calculation of the binary values of the grid cells or discrete values of the grid cells in the case of more than two decision classes is performed by a first evaluation or control unit and then to a second evaluation. or control unit transferred.
- the second evaluation and control unit of the control of driver assistance functions namely, for example, the output of a brake, steering, Lichtieriungs- or warning signal and the first evaluation and control unit is the evaluation of a sensor system for environment detection.
- the discretization or classification of the values stored in the grid cells large areas with high spatial correlation are present.
- FIG. 1 shown In FIG. 1 left is a grid before the discretization of the grid values and in Fig. 1 on the right a grid with discretized values is shown. In FIG. 1 on the right there are now two states, namely unfilled or filled lattice cells forming contiguous regions.
- compression methods for spatial decoration such as run-length coding or quad trees on the discrete values, a high data compression can be achieved.
- the values of the grid cells are updated at given time intervals. There is usually a high temporal correlation between temporally successive values of the grid cells in binary (discretized) representation, since the probability in a cell changes even with the integration of new measurement data, but the assignment to a discrete class on the basis of the threshold value does not in many cases.
- a further strong compression of the data is achieved by decorrelation of the temporal dependence, ie the difference formation of temporally successive grids.
- FIG. 2 On the left is a grid with discretized values of the measurement cycle n.
- FIG. 2 right is a grid with discretized values of the subsequent measurement cycle n + 1. If one then forms the difference between the corresponding grid cells "Difference Grid (n + 1) - Grid (n)", one finds that only the discretized value of 3 grid cells has changed. These cells are in the FIG. 2 Marked on the right with a bold border.
- a lossy method can be used. This can be used on the one hand for a further reduction of the data rate, but also in order to obtain a constant data rate after the compression, as is usually required for automotive applications. In doing so, areas further from the vehicle are compressed more, since the required accuracy of the environment modeling decreases with the distance to the vehicle (e.g., parking assistance to crossways on highways).
- features facing the vehicle are particularly relevant, so that alternatively or additionally the features facing away from the vehicle can be compressed more strongly (for example, front / back of a construction site wall). For this purpose, in particular, a tree of four, in which further loss or the vehicle remote areas by limiting the depth of the tree, such lossy compression can be achieved.
- the difference data for a recalculation can be transmitted in comparison to the previously valid ones.
- An important point of the invention is thus the application of a threshold value formation before the application of compression methods, because only after the threshold value formation a high spatial and temporal correlation, which enables high compression factors, is present.
- the assignment of the discrete value based on the environment data using an assignment table is carried out, wherein the allocation table is hinelves in the memory of the evaluation. In the allocation table all, so the thresholds are stored, which allows the assignment to a discrete value.
- An assignment table is given here by way of example. Table 1 cell value 0% - 25% 25% - 50% 50% - 75% 75% - 100% class 1 2 3 4
- a problem-adjusted discretization e.g., logarithmic
- a problem-adjusted discretization may be used which uses large discretization steps in less relevant regions of the continuous input values, and thereby more finely resolve the relevant regions for an equal number of discrete classes.
- a further advantage of displaying functions via tables is the easy verifiability of the input and output values, and special cases can also be handled by corresponding entries in the table.
Description
Die Erfindung liegt auf dem technischen Gebiet der Umgebungserfassung mit Sensoren in einem Fahrzeug. Die Sensordaten werden in geeigneter Weise aufbereitet, um ein Umfeldmodell für weitere Applikationen insbesondere Fahrerassistenzfunktionen zur Verfügung zu stellen.The invention is in the technical field of environmental detection with sensors in a vehicle. The sensor data are processed in a suitable manner in order to provide an environment model for further applications, in particular driver assistance functions.
Stand der Technik ist z.B. ein gitterbasiertes Umfeldmodell. Dazu wird das Umfeld eines Fahrzeuges in Zellen zu unterteilen und für jede Zelle wird ein Merkmal oder mehrere Merkmale zur Umfeldbeschreibung gespeichert. Weiterhin gehören objektbasierten Verfahren zur Umfeldmodellierung, die die Positionsdaten von erkannten Objekten zur Verfügung stellen, zum Stand der Technik.Prior art is e.g. a grid-based environment model. For this purpose, the environment of a vehicle is subdivided into cells, and for each cell a feature or several characteristics for the environment description is stored. Furthermore, object-based methods for environment modeling, which provide the position data of detected objects, belong to the state of the art.
Ein Vorteil der gitterbasierte Verfahren ist es, dass das ganze Umfeld des Fahrzeugs beschrieben wird. Es werden die Informationen aus Sensordaten für freie und mit Objekten belegte und unbekannte Bereiche statt nur für mit Objekten belegte Bereiche (objektbasierten Verfahren) im Umfeldmodell zur Verfügung gestellt. Die explizite Modellierung von freien Bereichen gewinnt an Bedeutung. Da viele neuere Assistenzfunktionen, wie z.B. ein Ausweichassistent, Informationen über einen Freiraum, der als Manöverraum für das Fahrzeug nutzbar ist, benötigen. Gitterbasierte Verfahren zur Umfeldbeschreibung benötigen in der Regel eine größere Datenmenge als objektbasierte Verfahren und damit.erfordert eine Applikation im Fahrzeug größere Speicherresourcen und Übertragungsbandbreiten.One advantage of grid-based methods is that the whole environment of the vehicle is described. The information from sensor data for free and object-occupied and unknown areas is made available instead of only for areas occupied with objects (object-based methods) in the environment model. The explicit modeling of free areas is becoming more important. Since many newer assistance functions, such as an evasion assistant, information about a space that is usable as a maneuver room for the vehicle need. Grid-based methods for describing the environment usually require a larger amount of data than object-based methods, and thus an application in the vehicle requires larger memory resources and transmission bandwidths.
Die
Die
Die
Die
Es ist die Aufgabe der hier vorliegenden Erfindung, die Daten eines gitterbasierten Umfeldmodells so zu komprimieren, dass eine Verwendung im Fahrzeug und insbesondere eine Übertragung der Umfeldmodelldaten über übliche Fahrzeugbussysteme zwischen Steuergeräten möglich ist.It is the object of the present invention to compress the data of a grid-based environment model such that use in the vehicle and, in particular, transmission of the surrounding model data via conventional vehicle bus systems between control units is possible.
Die Aufgabe wird durch die Merkmale der unabhängigen Ansprüche gelöst.The object is solved by the features of the independent claims.
Es wird ein Verfahren für ein Sensorsystem zur Umfelderfassung für ein Kraftfahrzeug beansprucht, wobei ein gitterbasiertes Umfeldmodell berechnet wird. Ein gitterbasiertes Umfeldmodell basiert darauf, das Umfeld eines Fahrzeuges in Zellen zu unterteilen und für jede Zelle ein das Umfeld beschreibendes Merkmal zu speichern. Das Speichern von Sensorrohdaten oder das Speichern einer Klassifikation für jede Zelle als Wahrscheinlichkeitswert, z.B. ein Speichern der Wahrscheinlichkeit, dass eine Zelle belegt oder nicht belegt ist, benötigt eine hohe Speicherkapazität, der bei einer Übertragung von oder zu einem Steuergerät zudem ein Bussystem mit hoher Bandbreite erfordert.
Erfindungsgemäß wird jeder Gitterzelle zumindest ein diskreter Wert (Klasse) zugeordnet. Insbesondere ist der diskrete Wert bzw. die Klasse ein Maß dafür, ob sich ein Objekt an der Position befindet, die durch die Gitterzelle repräsentiert wird und ob dieses Objekt überfahrbar ist oder mit welcher Wahrscheinlichkeit das Objekt überfahrbar ist.
Die Diskretisierung bzw. Zurdnung einer Klasse erfolgt durch die Auswertung von Umfelddaten, die Aussagen über erfasste Umgebungsobjekte enthalten, und zumindest einem Schwellwert. Die Anzahl der Schwellwerte kann beliebig definiert werden und beeinflusst die Anzahl der möglichen Klassen. Mit einem Schwellwert können maximal zwei diskrete Werte bzw. Klassen, bei zwei Schwellwerten maximal drei diskrete Werte bzw. Klassen begrenzt werden.A method is claimed for a sensor system for environment detection for a motor vehicle, wherein a grid-based environmental model is calculated. A grid-based environment model is based on dividing the environment of a vehicle into cells and storing for each cell a characteristic descriptive of the environment. Storing sensor raw data or storing a classification for each cell as a probability value, eg storing the probability that a cell is occupied or not occupied, requires a high storage capacity, which also requires a bus system with high bandwidth in a transmission from or to a control unit.
According to the invention, each grid cell is assigned at least one discrete value (class). In particular, the discrete value or the class is a measure of whether an object is at the position that is represented by the grid cell and whether this object is traversable or with what probability the object is traversable.
The discretization or renegotiation of a class takes place by the evaluation of environment data which contain statements about detected environment objects, and at least one threshold value. The number of thresholds can be defined as desired and influences the number of possible classes. With a threshold value a maximum of two discrete values or classes, with two threshold values a maximum of three discrete values or classes can be limited.
Vorzugsweise wird ein verlustfreies Kompressionverfahren auf die diskreten Werte eines Gitters angewendet, insbesondere vor einer Übertragung über ein Datenübertragungssystem im Fahrzeug. Dies umfasst auch Gitterwerte, die vor einer Übertragung komprimiert wurden.Preferably, a lossless compression method is applied to the discrete values of a grid, in particular prior to transmission via a data transmission system in the vehicle. This includes grid values that were compressed before being transmitted.
In einer bevorzugten Ausgestaltung der Erfindung wird eine Kompression durch Dekorrelation der zeitlichen Abhängigkeit, d.h. durch eine Differenzbildung zeitlich aufeinanderfolgender diskreter Werte der Gitterzellen eine Kompression der Daten erreicht. Insbesondere werden nur die Differenzwerte über ein Datenübertragungssystem im Fahrzeug übertragen.In a preferred embodiment of the invention, compression is achieved by decorrelation of the temporal dependency, i. achieved by a difference of temporally successive discrete values of the grid cells, a compression of the data. In particular, only the difference values are transmitted via a data transmission system in the vehicle.
In einer weiteren positiven Ausgestaltung der Erfindung , wird ein verlustbehaftetes Kompressionsverfahren auf die auf die diskreten Werte eines Gitters angewendet, insbesondere vor einer Übertragung über ein Datenübertragungssystem im Fahrzeug.In a further advantageous embodiment of the invention, a lossy compression method is applied to the discrete values of a grid, in particular prior to transmission via a data transmission system in the vehicle.
Insbesondere werden dazu die diskreten Werte des Gitters, die weiter vom Fahrzeug entfernte Bereiche repräsentieren, stärker komprimiert.In particular, to this end, the discrete values of the grid, which represent areas further from the vehicle, are more compressed.
Vorzugsweise werden alternativ oder zusätzlich die diskreten Werte des Gitters, die dem Fahrzeug abgewandte Merkmale repräsentieren, stärker komprimiert. Z.B. wird die Rückseite einer Baustellenwand, die dem Fahrzeug abgewandt angeordnet ist, stärker komprimiert als die Vorderseite der Baustellenwand, die dem Fahrzeug zugewandt angeordnet ist.
Eine besonders recheneffiziente Ausgestaltung der Erfindung sieht vor, dass die Zuordnung des diskreten Werts bzw. der Klasse anhand der Umfelddaten mit Hilfe einer Zuordnungstabelle erfolgt, wobei die Zuordnungstabelle im Speicher der Auswerteeinheit hinterlegt ist. In einer bevorzugten Ausgestaltung der Erfindung wird zur Änderung des Inhalts einer Mehrzahl von Gitterzellen, die Zurdnungstabelle verändert. Durch die geänderte Zuordnungsvorschrift die einem diskreten Wert bzw. eine Klasse einen veränderten Wertebereich der Umfelddaten zuordnen kann der Inhalt aller Zellen auf einmal verändert werden, ohne die diskreten Werte in jeder Gitterzelle neu zu berechnen und zu überschreiben.Preferably, alternatively or additionally, the discrete values of the grid, which represent features facing away from the vehicle, are compressed more strongly. For example, the back of a construction site wall, which is arranged facing away from the vehicle, more compressed than the front of the site wall, which is arranged facing the vehicle.
A particularly computationally efficient embodiment of the invention provides that the assignment of the discrete value or the class on the basis of the environment data is carried out with the aid of an allocation table, wherein the allocation table in the memory of the evaluation is deposited. In a preferred embodiment of the invention, the change of the content of a plurality of grid cells, the Verkdnungstabelle changed. The changed assignment rule, which assigns a changed value range of the environment data to a discrete value or a class, allows the content of all cells to be changed at once without recalculating and overwriting the discrete values in each grid cell.
In einer bevorzugten Ausgestaltung der Erfindung wird das Umfeldmodell basierend auf diskreten Werte im Gitter über ein Datenübertragungssystem in einem Fahrzeug zu einer Auswerte- oder Steuereinheit übertragen. Das Datenübertragungssystem ist vorzugsweise ein Bussystem im Fahrzeug, das zumindest zwei Auswerte- bzw. Steuereinheiten verbindet. Vorzugsweise erstellt eine Auswerteeinheit das gitterbasierte Umfeldmodell und eine weitere Auswerte- bzw. Steuereinheit nutzt das Umfeldmodell zur Steuerung einer Fahrerassistenzfunktion.In a preferred embodiment of the invention, the environment model is transmitted based on discrete values in the grid via a data transmission system in a vehicle to an evaluation or control unit. The data transmission system is preferably a bus system in the vehicle, which connects at least two evaluation or control units. Preferably, an evaluation unit creates the grid-based environment model and a further evaluation or control unit uses the environment model for controlling a driver assistance function.
Die hier beanspruchte Erfindung umfasst ein Sensorsystem zur Objekterfassung für ein Fahrzeug mit einer erste Rechen- und Auswerteeinheit auf der ein Verfahren wie zuvor beschrieben hinterlegt ist.The invention claimed here comprises a sensor system for object detection for a vehicle having a first computing and evaluation unit on which a method as described above is stored.
Insbesondere ist eine zweite Auswerte- oder Steuereinheit und ein Datenübertragungssystem vorgesehen, wobei über das Datenübertragungssystem die erste mit der zweite Auswerte- oder Steuereinheit in einem Fahrzeug verbunden ist.
In einer bevorzugten Ausgestaltung der Erfindung ist die erste Auswerte- oder Steuereinheit zur Erstellung eines Umfeldmodells und die zweite Auswerte- oder Steuereinheit zur Steuerung eines Fahrerassistenzsystems vorgesehen.In particular, a second evaluation or control unit and a data transmission system is provided, wherein via the data transmission system, the first is connected to the second evaluation or control unit in a vehicle.
In a preferred embodiment of the invention, the first evaluation or control unit for creating an environment model and the second evaluation or control unit for controlling a driver assistance system is provided.
Die Erfindung wird im Folgenden anhand von Ausführungsbeispielen und Abbildungen näher erläutert.The invention will be explained in more detail below with reference to exemplary embodiments and illustrations.
Ein gitterbasiertes Umfeldmodell basiert darauf, das Umfeld eines Fahrzeuges in Zellen zu unterteilen und für jede Zelle ein das Umfeld beschreibendes Merkmal zu speichern. Das Speichern von Sensorrohdaten oder das Speichern einer Klassifikation für jede Zelle als Wahrscheinlichkeit, z.B. die Wahrscheinlichkeit, dass eine Zelle belegt oder nicht belegt ist, benötigt eine hohe Speicherkapazität, der bei einer Übertragung von oder zu einem Steuergerät zudem ein Bussystem mit hoher Bandbreite erfordert. Eine direkte Anwendung eines Komprimierungverfahren auf ein berechnetes Gitter führt oft nicht zu einem hohen Kompressionsfaktor, da sich die Wahrscheinlichkeiten benachbarter Zellen oft nur minimal unterscheiden. Beispielhaft ist ein solches Gitter ist in
Eine Verwendung der Umfelddaten insbesondere in einem Fahrzeug für ein Fahrerassistenzsystem fordert in der Regel eine binäre Entscheidung, bei der anhand eines Schwellwertes für die Wahrscheinlichkeit entschieden wird, ob die Zelle belegt und damit für ein Fahrzeug nicht überfahrbar oder frei und damit für ein Fahrzeug überfahrbar ist. Von Bedeutung sind also diskrete Entscheidungsklassen die z.B. mit einem Zahlenwert o.ä. die Zustände belegt/frei für eine Gitterzelle angeben. In einer positiven Ausgestaltung der Erfindung, wird die Erstellung des Umfeldmodells mit der Berechnung der binären Werte der Gitterzellen, bzw. diskreten Werte der Gitterzellen im Fall von mehr als zwei Entscheidungsklassen durch eine erste Auswerte- bzw. Steuereinheit durchgeführt und danach an eine zweite Auswerte- bzw. Steuereinheit übertragen.A use of the environment data, in particular in a vehicle for a driver assistance system usually requires a binary decision, is decided on the basis of a threshold for the probability of whether the cell is occupied and thus not overridden or free for a vehicle and thus overridden for a vehicle , Of importance, therefore, are discrete decision classes, e.g. with a numerical value or similar state the states occupied / free for a grid cell. In a positive embodiment of the invention, the generation of the environment model with the calculation of the binary values of the grid cells or discrete values of the grid cells in the case of more than two decision classes is performed by a first evaluation or control unit and then to a second evaluation. or control unit transferred.
In diesem Ausführungsbeispiel dient die zweite Auswerte- und Steuereinheit der Steuerung von Fahrerassistenzfunktionen, nämlich z.B. der Ausgabe eines Brems-,Lenk-, Lichtsteuerungs- oder Warnsignals und die erste Auswerte- und Steuereinheit ist die Auswerteeinheit eines Sensorsystems zur Umfelderfassung. Nach der Diskretisierung bzw. Klassifikation der in den Gitterzellen hinterlegten Werte liegen große Bereiche mit hoher räumlicher Korrelation vor. Dies ist beispielhaft in
Die Werte der Gitterzellen werden in vorgegeben zeitlichen Abständen aktualisiert. Es besteht zwischen zeitlich aufeinanderfolgenden Werten der Gitterzellen in binärer (diskretisierter) Darstellung i.d.R. eine hohe zeitliche Korrelation, da sich selbst bei Integration neuer Messdaten zwar die Wahrscheinlichkeit in einer Zelle ändert, die Zuordnung zu einer diskreten Klasse anhand des Schwellwertes aber in vielen Fällen nicht. In einer bevorzugten Ausgestaltung der Erfindung wird durch Dekorrelation der zeitlichen Abhängigkeit, d.h. die Differenzbildung zeitlich aufeinanderfolgender Gitter) eine weitere starke Kompression der Daten erreicht. Dies ist beispielhaft in
Neben einem verlustfreien Kompressionsverfahren oder auch allein kann in einem weiteren Ausführungsbeispiel ein verlustbehaftetes Verfahren zum Einsatz kommen. Dies kann zum einen zu einer weiteren Reduktion der Datenrate verwendet werden, aber auch, um nach der Kompression eine konstante Datenrate zu erhalten, wie es für Automotive Anwendungen üblicherweise gefordert wird. Dabei werden weiter vom Fahrzeug entfernte Bereiche stärker komprimiert, da die erforderliche Genauigkeit der Umfeldmodellierung mit der Entfernung zum Fahrzeug abnimmt (z.B. Einparkassistenz gegenüber Querführung auf Autobahnen). Desweiteren sind insbesondere dem Fahrzeug zugewandte Merkmale relevant, so dass alternativ oder zusätzlich dem Fahrzeug abgewandte Merkmale stärker komprimiert werden können (z.B. Vorder-/Rückseite einer Baustellenwand). Hierfür eignet sich insbesondere ein Viererbaum, bei dem in weiter entfernen bzw. dem Fahrzeug abgewandten Bereichen durch Begrenzung der Baumtiefe eine derartige verlustbehaftete Kompression erzielt werden kann.In addition to a lossless compression method or alone, in another embodiment, a lossy method can be used. This can be used on the one hand for a further reduction of the data rate, but also in order to obtain a constant data rate after the compression, as is usually required for automotive applications. In doing so, areas further from the vehicle are compressed more, since the required accuracy of the environment modeling decreases with the distance to the vehicle (e.g., parking assistance to crossways on highways). In addition, features facing the vehicle are particularly relevant, so that alternatively or additionally the features facing away from the vehicle can be compressed more strongly (for example, front / back of a construction site wall). For this purpose, in particular, a tree of four, in which further loss or the vehicle remote areas by limiting the depth of the tree, such lossy compression can be achieved.
Durch die Übertragung nur von Differenzenwerten lässt sich auf Anwendungsseite, d.h. bei der Auswertung oder Funktionssteuerung, weitere Rechenzeit sparen. Insbesondere können die Differenzdaten für eine Neuberechnungen im Vergleich zu den zuvor gültigen übertragen werden.By transferring only difference values, it is possible on the application side, i. during evaluation or function control, save additional computation time. In particular, the difference data for a recalculation can be transmitted in comparison to the previously valid ones.
Ein wichtiger Punkt der Erfindung ist also die Anwendung einer Schwellwertbildung vor der Anwendung von Kompressionsverfahren, da erst nach der Schwellwertbildung eine hohe räumliche und zeitliche Korrelation, die hohe Kompressionsfaktoren ermöglicht, vorliegt.An important point of the invention is thus the application of a threshold value formation before the application of compression methods, because only after the threshold value formation a high spatial and temporal correlation, which enables high compression factors, is present.
Vorteilhaft ist, dass durch Verlagerung der Schwellwertbildung von der Funktion zur Berechnung des Umfeldmodells Kompressionsverfahren effektiv auf das gitterbasierte Umfeldmodell angewandt werden können, die so eine Übertragung des gitterbasierten Umfeldmodells über Automotive-Bussysteme ermöglichen.It is advantageous that, by shifting the thresholding from the function for calculating the environment model, compression methods can be effectively applied to the grid-based environment model, thus enabling transmission of the grid-based environment model via automotive bus systems.
Weitere Vorteile können Reduktion an Speicherbedarf im Funktionssteuergerät sowie Reduktion an benötigten CPU-Ressourcen sein, da nach Anwendung eines Kompressionsverfahrens zusammenhängende Bereiche, die gleich Klassifiziert sind, mit einer Operation zusammenhängend statt Zelle für Zelle bearbeitet werden können. Beispielsweise wären in
In einer bevorzugten Ausgestaltung der Erfindung die Zuordnung des diskreten Werts anhand der Umfelddaten mit Hilfe einer Zuordnungstabelle erfolgt, wobei die Zuordnungstabelle im Speicher der Auswerteeinheit hintelegt ist. In der Zuordnungstaballe sind also die Schwellwerte hinterlegt, die die Zuordnung zu einem diskreten Wert ermöglicht. Eine Zuordnungstabelle wird hier beispielhaft angegeben.
Insbesondere wird zur Änderung der Werte einer Mehrzahl von Gitterzellen vorzugsweise aller Gitterzellen, nur die Zurdnungstabelle mit den Schwellwerten verändert. Neben einer linearen Abbildung für die Diskretisierung wie in Tabelle 1 gezeigt, kann eine problemangepasste Diskretisierung (z.B. logarithmisch) verwendet werden, die in weniger relevanten Bereichen der kontinuierlichen Eingangswerte große Diskretisierungsschritte verwendet und dadurch bei gleicher Zahl von diskreten Klassen die relevanten Bereiche feiner auflösen kann.In particular, in order to change the values of a plurality of grid cells, preferably of all grid cells, only the table of additions with the threshold values is changed. In addition to a linear map for the discretization as shown in Table 1, a problem-adjusted discretization (e.g., logarithmic) may be used which uses large discretization steps in less relevant regions of the continuous input values, and thereby more finely resolve the relevant regions for an equal number of discrete classes.
Um den Inhalt aller Zellen zu ändern, wird die Zuordnung von Werten in der Tabelle geändert. Durch die Zuordnung der neuen Werte in Tabelle 2 für die Zellen wird die Wahrscheinlichkeit in allen Zellen reduziert, ohne dass die Zellen im Grid angepasst werden müssen.
Da sich bei einem diskreten Definitionsbereich als Eingang der Anwendungsfunktionen für das Update eine endliche Anzahl an Elementen im Wertebereich ergibt, ist es damit möglich, die Werte vorher zu berechnen und in Tabellen zu speichern, dadurch lässt sich weitere Rechenzeit sparen. Ein weiterer Vorteil der Darstellung von Funktionen über Tabellen ist die einfache Überprüfbarkeit der Eingangs - und Ausgangswerte, zudem können Sonderfälle durch entsprechende Einträge in der Tabelle behandelt werden.Since a finite number of elements in the value range results for a discrete domain as input of the application functions for the update, it is thus possible to calculate the values beforehand and to save them in tables, thereby saving additional computation time. A further advantage of displaying functions via tables is the easy verifiability of the input and output values, and special cases can also be handled by corresponding entries in the table.
Claims (13)
- A method for a sensor system for covering the environment for a motor vehicle having an analyzing unit, wherein a grid-based environmental model is calculated and at least a discrete value is assigned to each grid cell, said discrete value being at least a measure for whether or not an object is present in the position represented by the respective grid cell, characterized in that
a discretization is performed by comparing environmental data with at least a threshold value. - The method according to claim 1,
characterized in that
at least a discrete value of the grid is transmitted to an analyzing or control unit via a data transmission system in a vehicle. - The method according to claim 1 or 2,
characterized in that
a lossless compression method is applied to the discrete values of a grid. - The method according to claim 3,
characterized in that
a compression of the data is achieved by a decorrelation of temporal dependence. - The method according to any one of the preceding claims,
characterized in that
a lossy compression method is applied. - The method according to claim 5,
characterized in that
those discrete values of the grid which represent regions farther from the vehicle are subjected to a higher compression. - The method according to claim 5, characterized in that
those discrete values of the grid which represent features facing away from the vehicle are subjected to a higher compression. - The method according to any one of the preceding claims,
characterized in that
the environmental model and preferably a data compression is calculated prior to a data transmission in a vehicle. - The method according to any one of the preceding claims,
characterized in that
the discrete value is assigned on the basis of the environmental data by means of a cross-reference list, wherein the cross-reference list is recorded in the memory of the analyzing unit. - The method according to claim 2,
characterized in that
for changing the values of a plurality of grid cells, in particular of all grid cells, only the cross-reference list is changed. - A device, comprising a sensor system for object detection for a vehicle, a first calculating and analyzing unit on which a method according to any one of claims 1 to 10 is recorded.
- The device according to claim 11, further comprising
a second analyzing or control unit and a data transmission system, wherein the first analyzing or control unit is connected to the second one in a vehicle via the data transmission system. - The device according to claim 10 or 12, wherein the first analyzing or control unit is provided for creating an environmental model and the second analyzing or control unit is provided for controlling a driver assistance system.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| DE102011117138 | 2011-10-28 | ||
| PCT/DE2012/100292 WO2013060323A1 (en) | 2011-10-28 | 2012-09-20 | Grid-based environmental model for a vehicle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| EP2771873A1 EP2771873A1 (en) | 2014-09-03 |
| EP2771873B1 true EP2771873B1 (en) | 2018-04-11 |
Family
ID=47189655
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP12787353.7A Active EP2771873B1 (en) | 2011-10-28 | 2012-09-20 | Grid-based environmental model for a vehicle |
Country Status (5)
| Country | Link |
|---|---|
| US (1) | US20140278049A1 (en) |
| EP (1) | EP2771873B1 (en) |
| DE (1) | DE112012003549A5 (en) |
| ES (1) | ES2669551T3 (en) |
| WO (1) | WO2013060323A1 (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102019203274B3 (en) | 2019-03-11 | 2020-07-09 | Zf Friedrichshafen Ag | Computer-implemented method for creating an environment model for an automated vehicle, control unit and computer program product for automated control of a vehicle and control system for a vehicle |
| DE102020201000B3 (en) | 2020-01-28 | 2021-07-29 | Zf Friedrichshafen Ag | Computer-implemented method and system for obtaining an environment model and control device for an automated vehicle |
| DE102020210379A1 (en) | 2020-08-14 | 2022-02-17 | Zf Friedrichshafen Ag | Computer-implemented method and computer program product for obtaining a representation of surrounding scenes for an automated driving system, computer-implemented method for learning a prediction of surrounding scenes for an automated driving system and control unit for an automated driving system |
Families Citing this family (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102013104257A1 (en) | 2013-04-26 | 2014-10-30 | Continental Teves Ag & Co. Ohg | Grid-based environment model for a vehicle |
| DE102013214631A1 (en) | 2013-07-26 | 2015-01-29 | Bayerische Motoren Werke Aktiengesellschaft | Efficient provision of occupancy information for the environment of a vehicle |
| DE102013214632A1 (en) | 2013-07-26 | 2015-01-29 | Bayerische Motoren Werke Aktiengesellschaft | Efficient provision of occupancy information for the environment of a vehicle |
| DE102013217488A1 (en) | 2013-09-03 | 2015-03-05 | Continental Teves Ag & Co. Ohg | Occupancy grid with compact grid levels |
| DE102013217491A1 (en) * | 2013-09-03 | 2015-03-05 | Continental Teves Ag & Co. Ohg | Transmission of data of a grid-based environment model in a vehicle |
| DE102013018315A1 (en) | 2013-10-31 | 2015-04-30 | Bayerische Motoren Werke Aktiengesellschaft | Environment model with adaptive grid |
| DE102014204430A1 (en) * | 2014-03-11 | 2015-09-17 | Conti Temic Microelectronic Gmbh | Method and device for determining an accuracy of an occupancy grid to be evaluated for a driver assistance system |
| DE102014220687A1 (en) * | 2014-10-13 | 2016-04-14 | Continental Automotive Gmbh | Communication device for a vehicle and method for communicating |
| DE102015201747A1 (en) * | 2015-02-02 | 2016-08-04 | Continental Teves Ag & Co. Ohg | SENSOR SYSTEM FOR A VEHICLE AND METHOD |
| DE102015201867A1 (en) * | 2015-02-03 | 2016-08-04 | Bayerische Motoren Werke Aktiengesellschaft | Optimized storage of an environment map in the memory of a vehicle |
| DE102015205244B3 (en) * | 2015-03-24 | 2015-12-10 | Bayerische Motoren Werke Aktiengesellschaft | Method for providing obstacle cards for vehicles |
| DE102015207978B3 (en) * | 2015-04-30 | 2016-06-23 | Bayerische Motoren Werke Aktiengesellschaft | Analysis of the occupancy of grid-based environment maps of a vehicle |
| DE102016211227A1 (en) | 2016-06-23 | 2017-12-28 | Conti Temic Microelectronic Gmbh | Method and vehicle control system for generating images of an environment model and corresponding vehicle |
| DE102016211453A1 (en) | 2016-06-27 | 2017-12-28 | Conti Temic Microelectronic Gmbh | Method and vehicle control system for generating images of an environment model and corresponding vehicle |
| DE102016225066A1 (en) | 2016-12-15 | 2018-06-21 | Conti Temic Microelectronic Gmbh | All-round visibility system for one vehicle |
| EP3442125A1 (en) * | 2017-08-10 | 2019-02-13 | Continental Automotive GmbH | Sensor network assembly for reducing quantity of data for vehicles |
| DE102018217268A1 (en) * | 2018-10-10 | 2020-04-16 | Zf Friedrichshafen Ag | Device and method for determining height information of an object in the surroundings of a vehicle |
| DE102018219773B4 (en) | 2018-11-19 | 2021-08-12 | Audi Ag | Method for mapping a local distribution of events of a predetermined event type in a predetermined surrounding area of a motor vehicle and a control device and motor vehicle designed for this purpose |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR2890774B1 (en) * | 2005-09-09 | 2007-11-16 | Inst Nat Rech Inf Automat | VEHICLE DRIVING ASSISANCE METHOD AND IMPROVED ASSOCIATED DEVICE |
| DE102007012458A1 (en) * | 2007-03-15 | 2008-09-18 | Robert Bosch Gmbh | Method for object formation |
| DE102007013023B4 (en) * | 2007-03-19 | 2017-10-05 | Sick Ag | Probabilistic raster map |
| US8126642B2 (en) * | 2008-10-24 | 2012-02-28 | Gray & Company, Inc. | Control and systems for autonomously driven vehicles |
| US8559673B2 (en) * | 2010-01-22 | 2013-10-15 | Google Inc. | Traffic signal mapping and detection |
| DE102010006828B4 (en) * | 2010-02-03 | 2021-07-08 | Volkswagen Ag | Method for the automatic creation of a model of the surroundings of a vehicle as well as driver assistance system and vehicle |
| US20140358427A1 (en) * | 2010-12-13 | 2014-12-04 | Google Inc. | Enhancing driving navigation via passive drivers feedback |
-
2012
- 2012-09-20 US US14/352,568 patent/US20140278049A1/en not_active Abandoned
- 2012-09-20 WO PCT/DE2012/100292 patent/WO2013060323A1/en active Application Filing
- 2012-09-20 DE DE112012003549.6T patent/DE112012003549A5/en active Pending
- 2012-09-20 ES ES12787353.7T patent/ES2669551T3/en active Active
- 2012-09-20 EP EP12787353.7A patent/EP2771873B1/en active Active
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102019203274B3 (en) | 2019-03-11 | 2020-07-09 | Zf Friedrichshafen Ag | Computer-implemented method for creating an environment model for an automated vehicle, control unit and computer program product for automated control of a vehicle and control system for a vehicle |
| DE102020201000B3 (en) | 2020-01-28 | 2021-07-29 | Zf Friedrichshafen Ag | Computer-implemented method and system for obtaining an environment model and control device for an automated vehicle |
| DE102020210379A1 (en) | 2020-08-14 | 2022-02-17 | Zf Friedrichshafen Ag | Computer-implemented method and computer program product for obtaining a representation of surrounding scenes for an automated driving system, computer-implemented method for learning a prediction of surrounding scenes for an automated driving system and control unit for an automated driving system |
| WO2022033810A1 (en) | 2020-08-14 | 2022-02-17 | Zf Friedrichshafen Ag | Computer-implemented method and computer programme product for obtaining an environment scene representation for an automated driving system, computer-implemented method for learning an environment scene prediction for an automated driving system, and control device for an automated driving system |
Also Published As
| Publication number | Publication date |
|---|---|
| US20140278049A1 (en) | 2014-09-18 |
| ES2669551T3 (en) | 2018-05-28 |
| WO2013060323A1 (en) | 2013-05-02 |
| EP2771873A1 (en) | 2014-09-03 |
| DE112012003549A5 (en) | 2014-05-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2771873B1 (en) | Grid-based environmental model for a vehicle | |
| DE102016218080B3 (en) | Method and device for determining a collision probability of a vehicle with an object | |
| DE102015217891A1 (en) | Determining a desired trajectory for a vehicle | |
| DE102017218889A1 (en) | Unarmed parameterized AI module and method of operation | |
| WO2019206775A1 (en) | Method and device for determining a network configuration of a neural network | |
| DE102018008685A1 (en) | Method for training an artificial neural network, artificial neural network, use of an artificial neural network and corresponding computer program, machine-readable storage medium and corresponding device | |
| DE102010011629A1 (en) | Method for environmental representation of vehicle, involves recording and storing environment data in hierarchical data structures, which are identified in environmental objects | |
| DE102016109762A1 (en) | COMPRESSION ALGORITHMS FOR VEHICLE MEASUREMENT OF PREVIEW DATA | |
| DE102019212912A1 (en) | Compressing a deep neural network | |
| EP2924585B1 (en) | Method for operating a vehicle crash test data management system | |
| DE102013206274A1 (en) | Method and apparatus for adapting a non-parametric function model | |
| DE102019130484A1 (en) | Method and device for training an ensemble of neural networks | |
| WO2021063572A1 (en) | Device and method for processing data from a neural network | |
| DE102020210700A1 (en) | More flexible iterative operation of artificial neural networks | |
| DE10331431B4 (en) | Data compression of simulation results | |
| WO2004053619A2 (en) | Method for processing digital data values | |
| DE102019214546A1 (en) | Computer-implemented method and device for optimizing an architecture of an artificial neural network | |
| DE102019216184A1 (en) | Method for the robustification of a neural network against adversarial disturbances | |
| DE102013224948A1 (en) | A method of storing an environment model | |
| WO2021213998A1 (en) | Method for operating a controller assembly, and corresponding controller assembly | |
| DE102021100765A1 (en) | Method, system and computer program product for determining safety-critical output values of a technical entity | |
| DE102020208737A1 (en) | Method and device for evaluating and certifying a robustness of an AI-based information processing system | |
| DE102013207066A1 (en) | Method for storing altitude profile data, as well as profile data and method for generating a height profile from these profile data | |
| WO2024078952A1 (en) | Method for operating a control device, control device, and vehicle | |
| DE102019204136A1 (en) | Method and device for training and producing an artificial neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| 17P | Request for examination filed |
Effective date: 20140528 |
|
| AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| DAX | Request for extension of the european patent (deleted) | ||
| RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: GREWE, RALPH Inventor name: HEGEMANN, STEFAN Inventor name: LUEKE, STEFAN Inventor name: HOHM, ANDREE |
|
| GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
| INTG | Intention to grant announced |
Effective date: 20171204 |
|
| GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
| GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
| AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
| REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
| REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 988810 Country of ref document: AT Kind code of ref document: T Effective date: 20180415 |
|
| REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: GERMAN |
|
| REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 502012012528 Country of ref document: DE |
|
| REG | Reference to a national code |
Ref country code: ES Ref legal event code: FG2A Ref document number: 2669551 Country of ref document: ES Kind code of ref document: T3 Effective date: 20180528 |
|
| REG | Reference to a national code |
Ref country code: SE Ref legal event code: TRGR |
|
| REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20180411 |
|
| REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
| RAP2 | Party data changed (patent owner data changed or rights of a patent transferred) |
Owner name: CONTI TEMIC MICROELECTRONIC GMBH Owner name: CONTINENTAL TEVES AG & CO. OHG |
|
| REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 7 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180711 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180711 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180712 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180813 |
|
| REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 502012012528 Country of ref document: DE |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 |
|
| PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 |
|
| 26N | No opposition filed |
Effective date: 20190114 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 |
|
| REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 |
|
| REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20180930 |
|
| REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180920 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180920 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180930 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180930 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180930 |
|
| REG | Reference to a national code |
Ref country code: AT Ref legal event code: MM01 Ref document number: 988810 Country of ref document: AT Kind code of ref document: T Effective date: 20180920 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180920 Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20120920 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180411 Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180411 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180811 |
|
| REG | Reference to a national code |
Ref country code: DE Ref legal event code: R081 Ref document number: 502012012528 Country of ref document: DE Owner name: CONTINENTAL AUTONOMOUS MOBILITY GERMANY GMBH, DE Free format text: FORMER OWNERS: CONTINENTAL TEVES AG & CO. OHG, 60488 FRANKFURT, DE; CONTI TEMIC MICROELECTRONIC GMBH, 90411 NUERNBERG, DE |
|
| REG | Reference to a national code |
Ref country code: GB Ref legal event code: 732E Free format text: REGISTERED BETWEEN 20221208 AND 20221215 |
|
| P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230522 |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20230920 Year of fee payment: 12 |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: SE Payment date: 20230920 Year of fee payment: 12 Ref country code: FR Payment date: 20230928 Year of fee payment: 12 Ref country code: DE Payment date: 20230930 Year of fee payment: 12 |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: ES Payment date: 20231124 Year of fee payment: 12 |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: IT Payment date: 20230927 Year of fee payment: 12 |