EP3799752B1 - Ego-motorrad-on-board-sensibilisierungssystem, verfahren zum erfassen und anzeigen des vorhandenseins von autonomen fahrzeugen - Google Patents

Ego-motorrad-on-board-sensibilisierungssystem, verfahren zum erfassen und anzeigen des vorhandenseins von autonomen fahrzeugen Download PDF

Info

Publication number
EP3799752B1
EP3799752B1 EP19465569.2A EP19465569A EP3799752B1 EP 3799752 B1 EP3799752 B1 EP 3799752B1 EP 19465569 A EP19465569 A EP 19465569A EP 3799752 B1 EP3799752 B1 EP 3799752B1
Authority
EP
European Patent Office
Prior art keywords
motorcycle
ego
board
vehicles
vehicle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP19465569.2A
Other languages
English (en)
French (fr)
Other versions
EP3799752A1 (de
Inventor
Constantin-Florin Caruntu
Alexandru-Daniel Puscasu
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Aumovio Autonomous Mobility Germany GmbH
Original Assignee
Continental Automotive Technologies GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Continental Automotive Technologies GmbH filed Critical Continental Automotive Technologies GmbH
Priority to EP19465569.2A priority Critical patent/EP3799752B1/de
Publication of EP3799752A1 publication Critical patent/EP3799752A1/de
Application granted granted Critical
Publication of EP3799752B1 publication Critical patent/EP3799752B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A42—HEADWEAR
    • A42B—HATS; HEAD COVERINGS
    • A42B3/00—Helmets; Helmet covers ; Other protective head coverings
    • A42B3/04—Parts, details or accessories of helmets
    • A42B3/0406—Accessories for helmets
    • A42B3/0433—Detecting, signalling or lighting devices
    • A42B3/046—Means for detecting hazards or accidents
    • A—HUMAN NECESSITIES
    • A42—HEADWEAR
    • A42B—HATS; HEAD COVERINGS
    • A42B3/00—Helmets; Helmet covers ; Other protective head coverings
    • A42B3/04—Parts, details or accessories of helmets
    • A42B3/0406—Accessories for helmets
    • A42B3/042—Optical devices
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G08G1/161—Decentralised systems, e.g. inter-vehicle communication
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G08G1/161—Decentralised systems, e.g. inter-vehicle communication
    • G08G1/163—Decentralised systems, e.g. inter-vehicle communication involving continuous checking
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G08G1/166—Anti-collision systems for active traffic, e.g. moving vehicles, pedestrians, bikes

Definitions

  • the invention is related to increasing road safety.
  • the invention is related to a system and a method for increasing the awareness of motorcycle riders in respect to the presence of autonomous vehicles driving in their field of view as well as a computer program for carrying out steps of the method.
  • the invention US 5251333 A presents a simplified display system that is mounted on the helmet of the motorcycle rider.
  • the invention US 20130305437 A1 proposes a helmet with a look-down micro-display that projects a virtual image in-line with the helmet's chin bar.
  • the invention US 8638237 B2 discloses a system that alerts a vehicle driver about a motorcycle approaching from the rear.
  • the system consists in a unit located in the car and a unit located on the motorcycle.
  • the second unit transmits signals toward the traveling lane of the motorcycle and the car unit is responsible of receiving the transmitted signals and alerting the driver of the approaching motorcycle from the rear.
  • US 20140273863 A1 provides a system which establishes a communication between a smart helmet and a mobile phone/communicator. It consists in a computer processor, a microphone and one speaker, all connected and integrated in the helmet to be used by the motorcycle rider for mobile calls.
  • the invention US 20160075338 A1 presents a safety device for motorcycle riders that includes a safety helmet and a camera device, which is situated on the motorcycle rider's helmet; the camera device is connected to a warning device that outputs a warning as a function of data collected by the camera device related to the state of the motorcycle rider, e.g., fatigue monitoring.
  • the invention US 20170176746 A1 presents a system with one or more cameras that are physically coupled to a helmet, where each camera is configured to generate a video feed, which is presented to a user by projecting it onto a surface, such as the visor of the helmet, thereby enabling enhanced situational awareness for the user of the helmet to the surroundings, but no processing is done on the received images, they being presented directly as captured by the cameras.
  • the invention US 10,219,571 discloses a motorcycle helmet comprising a plurality of electronic components, including internally mounted sensors for detecting objects present in a blind spot of a wearer.
  • the invention JP 2017004426 A describes a traffic safety system which includes a traffic light system with a plurality of wireless tags that measures the distance and the position of the communication devices detected by their radio tags owned by pedestrians, bicycles, motorcycles, wheelchairs, automobiles in its proximity and then sends this position to the other participants that have radio tags.
  • the system operates based on a combination of sensors using radio signals and light laser/LED emission to detect the position of the vehicles around and to communicate it via V2X (Vehicle-to-X).
  • the traffic information is displayed by means of a head unit display HUD that associates the received position information with the actual position on the road surface.
  • One special category of traffic participants refers to the autonomous vehicles.
  • Fig. 1 depicts a plurality of vehicles driving in the ego motorcycle rider's field of view without any hint of which vehicle among them is autonomous.
  • the problem solved by the invention is to provide a system and a method of detecting and displaying the presence of autonomous vehicles driving in the field of view of the ego motorcycle rider for the purpose of alerting said ego motorcycle rider about such presence enabling him to decide on the precautions to take in respect to the detected autonomous vehicles.
  • an ego motorcycle on-board awareness raising system placed in an ensemble consisting in an ego motorcycle and an ego motorcycle rider smart helmet, said smart helmet comprising an advanced driver assistance systems camera acquiring video images having the field of view facing forward and said smart helmet comprising a smart helmet visor for displaying said video images to the motorcycle rider as well as other awareness information, said ego motorcycle on-board awareness raising system further comprising:
  • a method for detecting and displaying presence of autonomous vehicles driving in the field of view of an ego motorcycle rider wearing an ego motorcycle rider smart helmet provided with a smart helmet visor using the ego motorcycle on-board awareness raising system having the following repetitive sequence of eight steps of the method carried out at regular intervals of time t n :
  • a computer program comprising instructions which, when the program is executed on an ego motorcycle on-board detection processing unit cause said ego motorcycle on-board detection processing unit to execute the steps 2 to 7 of the method.
  • the ego motorcycle on-board awareness raising system is placed in an ensemble consisting in an ego motorcycle and an ego motorcycle rider smart helmet SH. Some components of the system are being placed in the ego motorcycle, whereas other components are being placed in the smart helmet SH of the motorcycle rider, as it will be hereafter detailed.
  • Said ego motorcycle rider's smart helmet SH comprises an advanced driver assistance systems camera ADASC, alternatively called camera, acquiring video images and a smart helmet SH visor for displaying said video images as well as other awareness information such as images and/or warning messages addressed to the motorcycle rider.
  • ADASC advanced driver assistance systems camera
  • a smart helmet SH visor for displaying said video images as well as other awareness information such as images and/or warning messages addressed to the motorcycle rider.
  • the advanced driver assistance systems camera ADASC used in this invention has at least the following characteristics: 2 MP (megapixels) and a rate of 30 fps (frames per second) .
  • Said camera is mounted in/on the motorcycle rider' s smart helmet SH in a known way.
  • the advanced driver assistance systems camera ADASC used in this invention has the field of view facing forward, that is in the direction of movement of the motorcycle rider.
  • the inclination of the head of the motorcycle rider and/ or its rotation shall have as effect the change of the field of view.
  • Said ego motorcycle on-board awareness raising system further comprises the following components:
  • the definition of the field of view of the motorcycle rider facing forward includes the angle of the field of view and the distances in respect to the vehicles driving ahead of the motorcycle rider. Both the angle and the distances are pre-determined depending on the characteristics of the camera and, respectively of the radar module RM.
  • the angle of the field of view may be up to 150° inclusively measured in horizontal plane.
  • the radar module RM is placed in front of the ego motorcycle facing forward. It is configured to measure the distance between the ego motorcycle and the vehicles driving in the ego motorcycle field of view, said distance typically ranging between 5-200m inclusively. Any radar used in the automotive industry may be used in the invention provided that it is configured to send the measurements to the ego motorcycle on-board detection processing unit DPU.
  • the gyroscope GYRO is placed within the smart helmet SH. It is configured to determine the direction and inclination of the field of view of the smart helmet depending on the direction in which the rider looks. Any gyroscope used in the automotive industry may be used in the invention, provided that it fits within the smart helmet SH and provided that it is configured to send the result of determinations to the ego motorcycle on-board detection processing unit.
  • the GPS sensor GPSS is configured to determine the geographical position of the ego motorcycle. Any GPS sensor used in the automotive industry may be used in the invention, provided that it is configured to send the result of determinations to the ego motorcycle on-board detection processing unit DPU.
  • the GPS sensor GPSS may be placed in the ego motorcycle as a component of said motorcycle as provided by the manufacturer of motorcycles.
  • ego motorcycle is not provided with GPS sensor GPSS, but the ego motorcycle on-board unit MOBU is provided by its manufacturer with a GPS sensor GPSS.
  • said GPS sensor GPSS of the ego motorcycle on-board unit MOBU is the one configured to determine the geographical position of the ego motorcycle and to send the result of determinations to the ego motorcycle on-board detection processing unit DPU.
  • neither ego motorcycle nor ego motorcycle on-board unit MOBU is provided with GPS sensor GPSS.
  • the GPS sensor GPSS may be provided by a rider's smartphone configured to determine the geographical position of the ego motorcycle and to send the result of determinations to the ego motorcycle on-board detection processing unit DPU.
  • GPS sensor GPSS as shown in the alternative embodiments above have the advantage of providing flexibility to the system and allowing it to be used in a wider range of situations, irrespective of whether the ego motorcycle is provided with its built-in GPS sensor GPSS.
  • the acceleration sensor ACC is placed in the ego motorcycle in the usual place(s). It is configured to measure the acceleration signal of the ego motorcycle on all three axes X, Y Z. Any acceleration sensor used in the automotive industry measuring the acceleration signal of the ego motorcycle on all three axes X, Y Z. may be used in the invention provided that it is configured to send the measurements to the ego motorcycle on-board detection processing unit DPU.
  • the speedometer SPEEDO is placed in the ego motorcycle in the usual place (s) . It is configured to measure the speed of the ego motorcycle. Any speedometer used in the automotive industry may be used in the invention provided that is configured to send the measurements to the ego motorcycle on-board detection processing unit DPU.
  • motorcycle dynamics unit MDU can be combined with any of the possibilities of use of the GPS sensor GPSS within the system, having the advantage of flexibility.
  • the ego motorcycle on-board unit MOBU is configured to communicate via the Vehicle-to-Everything V2X communication channel with other vehicles, including other motorcycle riders.
  • communication via the Vehicle-to-Everything V2X communication channel requires that all vehicles communicating be provided with a corresponding vehicle on-board units VOBU, said vehicle on-board units VOBU allowing sending and receiving messages through said Vehicle-to-Everything V2X communication channel.
  • the initial configuration of the ego motorcycle on-board unit MOBU is the one commonly known in the state of art for the vehicle on-board unit VOBU.
  • the ego motorcycle on-board unit MOBU used in the invention is further configured to send via the vehicle bus VB the data received from other vehicles' vehicle on-board unit VOBU to the ego motorcycle on-board detection processing unit DPU.
  • autonomous vehicles are provided with vehicle on-board units VOBU. Additionally, it is known that each autonomous vehicle must be provided with an autonomous vehicle identifier AVI.
  • the ego motorcycle on-board unit MOBU is configured to read the autonomous vehicle identifier AVI of each autonomous vehicle.
  • the ego motorcycle on-board unit MOBU is configured to receive periodically from other vehicles' vehicle on-board units VOBU, including autonomous vehicles the following information, for each vehicle: geographical position; speed; acceleration signal on the three axes.
  • the ego motorcycle on-board unit MOBU is configured to receive periodically, apart from the above-captioned information, the following information referring to autonomous vehicles, for each autonomous vehicle for which the autonomous vehicle identifier was received and read: targeted path; estimation of position, speed and acceleration in a subsequent pre-determined prediction interval.
  • Fig. 2 places schematically the ego motorcycle on-board unit MOBU in the same category as the sensors, as the data received by the ego motorcycle on-board unit MOBU about other vehicles provided with vehicle on-board unit VOBU, including autonomous vehicles, is used by the method according to the invention in the same way as the information from the other sensors.
  • the ego motorcycle on-board detection processing unit DPU is specially configured to carry out the detection of the autonomous vehicles. Whilst all the other components of the system according to the invention already exist either on the ego motorcycle or on the ego motorcycle rider smart helmet SH, being specially adapted for the invention, the ego motorcycle on-board detection processing unit DPU does not exist in the absence of the invention.
  • the ego motorcycle on-board detection processing unit DPU is configured to receive input from all the categories of sensors, configured to detect autonomous vehicles driving in the ego motorcycle's field of view and further configured to send said result of detection of the autonomous vehicles to the smart helmet SH visor in order to be displayed, as it will be further detailed in the description of the steps of the method.
  • the ego motorcycle on-board detection processing unit DPU comprises a dedicated processor having a processing power above 1000MHz and a capacity to store information of at least 1-2Gb and also comprises at least one non-volatile memory.
  • the ego motorcycle on-board detection processing unit DPU is placed in the motorcycle, for example, in a location suitable for electronic control units.
  • the at least one ego motorcycle bus system BS is configured to interconnect all the components of the system: the advanced driver assistance camera ADASC, the radar module RM, the gyroscope GYRO, the GPS sensor GPSS, the acceleration sensor ACC, the speedometer SPEEDO, the ego motorcycle on-board unit MOBU, the ego motorcycle on-board detection processing unit DPU and the smart helmet SH visor.
  • the at least one ego motorcycle bus system BS may be any kind of vehicle bus used in the automotive industry, using communication protocols such as but not limited to: CAN bus, FlexRay, Ethernet, Bluetooth, etc. Depending on the particular configuration of said components of the system, more than one motorcycle bus system BS may be used to interconnect various components of the system, including the case when various communication protocols are used respectively for each bus system BS.
  • a method for detecting and displaying presence of autonomous vehicles driving in the field of view of an ego motorcycle rider using the ego motorcycle on-board awareness raising system consists of sequences of 8 steps carried out at regular intervals of time t n when said motorcycle rider is in traffic.
  • a non-limiting example of regular intervals of time t n is between 20ms and 100ms.
  • the ego motorcycle on-board unit MOBU broadcasts messages through Vehicle-to-Everything V2X communication channel, said messages having the purpose to send to other vehicles provided with corresponding vehicle on-board units VOBU data about the position of the ego motorcycle and having the purpose to gather data about the presence of said other vehicles provided with corresponding vehicle on-board units VOBU and to gather data about which ones of said other vehicles are provided with a corresponding autonomous vehicle identifier AVI.
  • the broadcasting is carried out regularly covering a pre-determined broadcasting range.
  • broadcast is carried out every 100ms and the broadcasting range is of around 200m around the ego motorcycle in all directions, thus including the field of view.
  • the ego motorcycle on-board detection processing unit DPU receives via the at least one ego motorcycle bus system BS signals as input data from the sensors:
  • Sensors send data to the ego motorcycle on-board detection processing unit DPU at different rates, depending on the specific configurations of the components. It is not mandatory for the invention that the rates be synchronized.
  • the ego motorcycle on-board detection processing unit DPU performs the processing of video stream acquired in step 2 from the advanced driver assistance systems camera ADASC, including steps such as:
  • Image segmentation and labelling aim to reveal the relevant objects in the image corresponding to each video stream.
  • the relevant objects are all vehicles from the field of view irrespectively of whether they are provided or not with corresponding vehicle on-board units VOBU.
  • the processing of video stream is carried out with the same rate as the acquisition rate of said video stream.
  • the resulting processed video stream with the relevant objects labelled does not contain yet the detection of the autonomous vehicles.
  • the ego motorcycle on-board detection processing unit DPU creates a fused environment road model based on the processed video stream carried out in the third step and on the data received from the radar module RM in the second step.
  • the fused environment road model has thus more information than the processed video stream, as the distance to the relevant objects revealed in the processed video stream is now added, still the detection of the autonomous vehicles is not yet carried out.
  • step 2 based on the fused environment road model created in step 4 and based on the following input data received in step 2:
  • Simultaneous localization and mapping SLAM algorithm includes correlation by the ego motorcycle on-board detection processing unit DPU with data received from the sensors: inclination of the helmet received from the gyroscope GYRO, geographical position from the GPS sensor GPSS, the lateral, longitudinal acceleration signal and the yaw rate from the accelerometer ACC and the speed from the speedometer SPEEDO.
  • One non-limiting example includes the correlation of orientation of the inclination of the helmet received from the gyroscope GYRO with the results provided by the radar module RM that are already processed in the fused environment road model because when the motorcycle rider moves his head, the inclination of the smart helmet SH changes and the video acquired by the camera changes.
  • the result of this step is the simultaneous localization and mapping of the motorcycle rider and its smart helmet SH in the fused environment road model, that includes information about the distances to all vehicles driving in the field of view of the motorcycle rider as measured by the radar module RM, still the detection of the autonomous vehicles is not yet carried out.
  • the ego motorcycle on-board detection processing unit DPU detects in the video stream of step 3 the autonomous vehicles provided with the corresponding autonomous vehicle identifier AVI.
  • the ego motorcycle on-board detection processing unit DPU compares and correlates the data regarding simultaneous localization and mapping of the motorcycle rider and its smart helmet SH in the fused environment road model as resulted from step 5 with the data received in step 2 from the ego motorcycle on-board unit MOBU regarding the other vehicles provided with corresponding vehicle on-board unit VOBU including autonomous vehicles of step 2. Specifically,
  • the autonomous vehicles detected are then marked by known marking techniques on the processed video stream.
  • the result of this step is a processed video stream with detected marked autonomous vehicles driving in the field of view of the motorcycle rider.
  • corrections are applied to the detection carried out in previous step.
  • the correction of the position marked for each autonomous vehicle detected is carried out by using a general kinematic estimation algorithm taking into account the values of the subsequent pre-determined prediction interval and has the purpose of ensuring a greater accuracy of marking on the processed video stream of the detected autonomous vehicles.
  • a general kinematic estimation algorithm taking into account the values of the subsequent pre-determined prediction interval and has the purpose of ensuring a greater accuracy of marking on the processed video stream of the detected autonomous vehicles.
  • the measurement of the radar module RM is considered as the more accurate and the determinations of the GPS sensor GPSS must be adjusted to match the measurements of the radar module RM.
  • the result of this step is a corrected video stream with detected and marked autonomous vehicles driving in the field of view of the motorcycle rider.
  • the ego motorcycle on-board detection processing unit DPU sends via the at least one ego motorcycle bus system BS to the smart helmet SH visor the corrected video stream with detected marked autonomous vehicles driving in the field of view of the motorcycle rider.
  • the smart helmet SH visor displays, in an understandable manner by the motorcycle rider, the autonomous vehicles provided with corresponding autonomous vehicle identifier AVI.
  • a computer program which, when the program is executed on an ego motorcycle on-board detection processing unit DPU of any of the preferred embodiments, cause said ego motorcycle on-board detection processing unit DPU to execute the steps 2 to 7 of the method.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Traffic Control Systems (AREA)

Claims (7)

  1. Ego-Motorrad-On-Board-Sensibilisierungssystem, platziert in einem Ensemble, umfassend ein Ego-Motorrad und einen intelligenten Ego-Motorradfahrerhelm (SH), wobei der intelligente Helm (SH) eine Videobilder erfassende erweiterte Fahrerunterstützungssysteme-Kamera (ADASC) umfasst, die ein nach vorn zeigendes Sichtfeld aufweist, und wobei der intelligente Helm (SH) einen Visor des intelligenten Helms (SH) zum Anzeigen der Videobilder für den Motorradfahrer sowie anderer Aufmerksamkeitsinformationen umfasst, dadurch gekennzeichnet, dass das Ego-Motorrad-On-Board-Sensibilisierungssystem ferner Folgendes umfasst:
    - ein Radarmodul (RM), vor dem Ego-Motorrad platziert und nach vorn zeigend, dazu ausgebildet, den Abstand zwischen dem Ego-Motorrad und den Fahrzeugen, die im Sichtfeld des Ego-Motorrads fahren, zu messen, und dazu ausgebildet, die Messungen an eine Ego-Motorrad-On-Board-Detektionsverarbeitungseinheit (DPU) zu senden;
    - ein Gyroskop (GYRO), innerhalb des intelligenten Helms (SH) platziert, dazu ausgebildet, die Richtung und Neigung des Sichtfelds des intelligenten Helms (SH) in Abhängigkeit von der Richtung zu bestimmen, in die der Fahrer schaut, und dazu ausgebildet, die Messungen an die Ego-Motorrad-On-Board-Detektionsverarbeitungseinheit (DPU) zu senden;
    - einen GPS-Sensor (GPSS), im Ego-Motorrad platziert, dazu ausgebildet, die geografische Position des Ego-Motorrads zu bestimmen, und dazu ausgebildet, die Messungen an die Ego-Motorrad-On-Board-Detektionsverarbeitungseinheit (DPU) zu senden;
    - einen Beschleunigungssensor (ACC), im Ego-Motorrad platziert, dazu ausgebildet, das Beschleunigungssignal des Ego-Motorrads auf drei Achsen (X, Y, Z) zu messen, und dazu ausgebildet, die Messungen an die Ego-Motorrad-On-Board-Detektionsverarbeitungseinheit (DPU) zu senden;
    - einen Geschwindigkeitsmesser (SPEEDO), im Ego-Motorrad platziert, dazu ausgebildet, die Geschwindigkeit des Ego-Motorrads zu messen, und dazu ausgebildet, die Messungen an die Ego-Motorrad-On-Board-Detektionsverarbeitungseinheit (DPU) zu senden;
    - eine Ego-Motorrad-On-Board-Einheit (MOBU), ausgebildet zum:
    - Kommunizieren über den Fahrzeug-zu-Allem- bzw. V2X-Kommunikationskanal mit anderen Fahrzeugen, die mit einer entsprechenden Fahrzeug-On-Board-Einheit (VOBU) versehen sind, zum Zwecke des Empfangens der geografischen Position, Geschwindigkeit und des Beschleunigungssignals auf drei Achsen (X, Y Z) jedes der anderen Fahrzeuge, die mit einer entsprechenden Fahrzeug-On-Board-Einheit (VOBU) versehen sind;
    - Empfangen und Lesen einer autonomen Fahrzeugkennung (AVI), gesendet über den Fahrzeug-zu-Allem- bzw. V2X-Kommunikationskanal durch die entsprechende Fahrzeug-On-Board-Einheit (VOBU) jedes entsprechenden autonomen Fahrzeugs zusammen mit dem beabsichtigten Pfad und Schätzung der Position, Geschwindigkeit und Beschleunigung auf drei Achsen (X, Y Z) in einem nachfolgenden vorbestimmten Vorhersageintervall für jedes autonome Fahrzeug, für das die autonome Fahrzeugkennung empfangen und gelesen wurde;
    - Senden der Messungen an die Ego-Motorrad-On-Board-Detektionsverarbeitungseinheit (DPU);
    - wobei die Ego-Motorrad-On-Board-Detektionsverarbeitungseinheit (DPU) dazu ausgebildet ist, Detektion der autonomen Fahrzeuge auszuführen, die im Sichtfeld des Ego-Motorrads fahren, und ferner dazu ausgebildet ist, das Ergebnis der Detektion an den intelligenten Helm (SH) zu senden;
    - zumindest ein Ego-Motorradbussystem (BS), dazu ausgebildet, die erweiterte Fahrerunterstützungskamera (ADASC), das Radarmodul (RM), das Gyroskop (GYRO), den GPS-Sensor (GPSS), den Beschleunigungssensor (ACC), den Geschwindigkeitsmesser (SPEEDO), die Ego-Motorrad-On-Board-Einheit (MOBU), die Ego-Motorrad-On-Board-Detektionsverarbeitungseinheit (DPU) und den Visor des intelligenten Helms (SH) miteinander zu verbinden.
  2. Ego-Motorrad-On-Board-Sensibilisierungssystem nach Anspruch 1, wobei der GPS-Sensor (GPSS) in dem Ego-Motorrad als eine Komponente des Motorrads platziert werden kann.
  3. Ego-Motorrad-On-Board-Sensibilisierungssystem nach Anspruch 1, wobei der GPS-Sensor (GPSS) eine Komponente der Ego-Motorrad On-Board-Einheit (MOBU) ist.
  4. Ego-Motorrad-On-Board-Sensibilisierungssystem nach Anspruch 1, wobei der GPS-Sensor (GPSS) eine Komponente eines Smartphones des Motorradfahrers ist.
  5. Ego-Motorrad-On-Board-Sensibilisierungssystem nach einem der Ansprüche 1 bis 4, wobei der Beschleunigungssensor (ACC) und der Geschwindigkeitsmesser (SPEEDO) in einem einzelnen Sensor kombiniert sind, der eine Motorraddynamikeinheit (MDU) bildet.
  6. Verfahren zum Detektieren und Anzeigen der Anwesenheit von autonomen Fahrzeugen, die im Sichtfeld eines Ego-Motorradfahrers fahren, der einen intelligenten Ego-Motorradfahrerhelm (SH) trägt, der mit einem Visor des intelligenten Helms (SH) versehen ist, unter Verwendung des Ego-Motorrad-On-Board-Sensibilisierungssystems, dadurch gekennzeichnet, dass die folgende repetitive Sequenz von Schritten des Verfahrens bei regelmäßigen Zeitintervallen tn ausgeführt wird:
    Schritt 1 Rundsenden, durch eine Ego-Motorrad-On-Board-Einheit (MOBU), von Nachrichten durch einen Fahrzeug-zu-Allem- bzw. V2X-Kommunikationskanal, wobei die Nachrichten den Zweck haben, an andere Fahrzeuge, die mit entsprechenden Fahrzeug-On-Board-Einheiten (VOBU) versehen sind, Daten über die Position des Ego-Motorrads zu senden, und den Zweck haben, Daten über die Anwesenheit der anderen Fahrzeuge, die mit entsprechenden Fahrzeug-On-Board-Einheiten (VOBU) versehen sind, zu sammeln und Daten darüber zu sammeln, welche der anderen Fahrzeuge mit einer entsprechenden autonomen Fahrzeugkennung (AVI) versehen sind.
    Schritt 2 Empfangen, durch eine Ego-Motorrad-On-Board-Detektionsverarbeitungseinheit (DPU), von Eingangsdaten von den Sensoren über das zumindest eine Ego-Motorradbussystem (BS):
    - Videostreams, erfasst von einer erweiterten Fahrerunterstützungssysteme-Kamera (ADASC);
    - Abstände zu allen Fahrzeugen, die vor dem Ego-Fahrzeug befindlich sind, von einem Radarmodul (RM), sofern all diese Fahrzeuge innerhalb der Reichweite des Radarmoduls (RM) fahren;
    - Geschwindigkeit des Ego-Motorrads von einem Geschwindigkeitsmesser (SPEEDO);
    - Beschleunigungssignale von X-, Y- und Z-Achsen des Ego-Motorrads von einem Beschleunigungsmesser (ACC);
    - Ausrichtung des intelligenten Helms (SH) des Ego-Motorrads von einem Gyroskop (GYRO);
    - geografische Position des Ego-Motorrads von einem GPS-Sensor (GPSS);
    - Daten von einer Ego-Motorrad-On-Board-Einheit (MOBU) hinsichtlich der geografischen Position, Geschwindigkeit und Beschleunigungssignalen von X-, Y-und Z-Achsen von anderen Fahrzeugen, die mit entsprechenden Fahrzeug-On-Board-Einheiten (VOBU) versehen sind, einschließlich autonome Fahrzeuge;
    - Daten von der Ego-Motorrad-On-Board-Einheit (MOBU) für jedes autonome Fahrzeug: die autonome Fahrzeugkennung (AVI), der beabsichtigte Pfad und die Schätzung von Position, Geschwindigkeit und Beschleunigungssignalen von X-, Y-und Z-Achsen in einem nachfolgenden vorbestimmten Vorhersageintervall.
    Schritt 3 Durchführen, durch die Ego-Motorrad-On-Board-Detektionsverarbeitungseinheit (DPU), der Verarbeitung eines Videostreams, der in Schritt 2 von der erweiterten Fahrerunterstützungssysteme-Kamera (ADASC) erfasst wurde:
    - Anwenden von Bildsegmentierung auf den Videostream zum Zwecke des Identifizierens aller relevanten Objekte darin;
    - Kennzeichnen der relevanten Objekte in dem segmentierten Bild,
    in einem verarbeiteten Videostream resultierend, in dem die relevanten Objekte gekennzeichnet sind.
    Schritt 4 Erzeugen, durch die Ego-Motorrad-On-Board-Detektionsverarbeitungseinheit (DPU), eines fusionierten Straßenumgebungsmodells basierend auf dem vorher verarbeiteten Videostream und auf den von dem Radarmodul (RM) empfangenen Daten.
    Schritt 5 Basierend auf dem fusionierten Straßenumgebungsmodell aus Schritt 4 und basierend auf einem Teil der in Schritt 2 empfangenen Eingangsdaten:
    - Neigung des Helms, empfangen vom Gyroskop (GYRO);
    - geografische Position vom GPS-Sensor (GPSS);
    - das laterale und Längsbeschleunigungssignal und die Gierrate vom Beschleunigungsmesser (ACC) und
    - Geschwindigkeit vom Geschwindigkeitsmesser (SPEEDO),
    Anwenden, durch die Ego-Motorrad-On-Board-Detektionsverarbeitungseinheit (DPU), eines gleichzeitigen Lokalisierungs- und Abbildungsalgorithmus (SLAM) mit dem Zweck des Lokalisierens des Ego-Motorrads im fusionierten Straßenumgebungsmodell und Lokalisierens der entsprechenden Ausrichtung des intelligenten Helms (SH),
    in der gleichzeitigen Lokalisierung und Abbildung des Motorradfahrers und seines intelligenten Helms (SH) in dem fusionierten Straßenumgebungsmodell resultierend.
    Schritt 6 Basierend auf:
    - dem verarbeiteten Videostream mit den in Schritt 3 gekennzeichneten relevanten Objekten;
    - der gleichzeitigen Lokalisierung und Abbildung des Motorradfahrers und seines intelligenten Helms SH im fusionierten Straßenumgebungsmodell aus Schritt 5 und
    - den in Schritt 2 empfangenen Daten von der Ego-Motorrad-On-Board-Einheit (MOBU) hinsichtlich der anderen Fahrzeuge, die mit einer entsprechenden Fahrzeug-On-Board-Einheit (VOBU) versehen sind, einschließlich autonome Fahrzeuge;
    - Vergleichen und Korrelieren der Daten hinsichtlich der gleichzeitigen Lokalisierung und Abbildung des Motorradfahrers und seines intelligenten Helms (SH) in dem fusionierten Straßenumgebungsmodell, wie aus Schritt 5 resultierend, mit den in Schritt 2 von der Ego-Motorrad-On-Board-Einheit (MOBU) empfangenen Daten hinsichtlich der anderen Fahrzeuge, die mit einer entsprechenden Fahrzeug-On-Board-Einheit (VOBU) versehen sind, einschließlich autonome Fahrzeuge,
    - Detektieren, in dem verarbeiteten Videostream, jedes autonomen Fahrzeugs, das mit der entsprechenden autonomen Fahrzeugkennung (AVI) versehen ist, und
    - Markieren jedes detektierten autonomen Fahrzeugs in dem verarbeiteten Videostream,
    in einem verarbeiteten Videostream mit detektierten markierten Fahrzeugen resultierend, die im Sichtfeld des Motorradfahrers fahren.
    Schritt 7 Anwenden einer Korrektur auf die Markierung der einzelnen detektierten autonomen Fahrzeuge im verarbeiteten Videostream zum Sicherstellen einer größeren Genauigkeit der Markierung auf dem verarbeiteten Videostream der detektierten autonomen Fahrzeuge und Senden, über das Ego-Motorradbussystem (BS), von Daten hinsichtlich der markierten autonomen Fahrzeuge an den Visor des intelligenten Helms (SH),
    in einem korrigierten Videostream mit detektierten markierten Fahrzeugen resultierend, die im Sichtfeld des Motorradfahrers fahren.
    Schritt 8 Anzeigen, auf dem Visor des intelligenten Helms (SH), der markierten autonomen Fahrzeuge in einer für den Motorradfahrer verständlichen Weise.
  7. Computerprogramm, umfassend Anweisungen, die, wenn das Programm auf einer Ego-Motorrad-On-Board-Detektionsverarbeitungseinheit (DPU) nach einem der Ansprüche 1 bis 5 ausgeführt wird, die Ego-Motorrad-On-Board-Detektionsverarbeitungseinheit (DPU) veranlassen, die Schritte 2 bis 7 des Verfahrens nach Anspruch 6 auszuführen.
EP19465569.2A 2019-10-02 2019-10-02 Ego-motorrad-on-board-sensibilisierungssystem, verfahren zum erfassen und anzeigen des vorhandenseins von autonomen fahrzeugen Active EP3799752B1 (de)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP19465569.2A EP3799752B1 (de) 2019-10-02 2019-10-02 Ego-motorrad-on-board-sensibilisierungssystem, verfahren zum erfassen und anzeigen des vorhandenseins von autonomen fahrzeugen

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP19465569.2A EP3799752B1 (de) 2019-10-02 2019-10-02 Ego-motorrad-on-board-sensibilisierungssystem, verfahren zum erfassen und anzeigen des vorhandenseins von autonomen fahrzeugen

Publications (2)

Publication Number Publication Date
EP3799752A1 EP3799752A1 (de) 2021-04-07
EP3799752B1 true EP3799752B1 (de) 2022-07-06

Family

ID=68461733

Family Applications (1)

Application Number Title Priority Date Filing Date
EP19465569.2A Active EP3799752B1 (de) 2019-10-02 2019-10-02 Ego-motorrad-on-board-sensibilisierungssystem, verfahren zum erfassen und anzeigen des vorhandenseins von autonomen fahrzeugen

Country Status (1)

Country Link
EP (1) EP3799752B1 (de)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102024205672A1 (de) * 2024-06-19 2025-12-24 Volkswagen Aktiengesellschaft Verfahren und Assistenzsystem zum Unterstützen eines Fahrers in einer gemischten Verkehrsumgebung und entsprechend eingerichtetes Kraftfahrzeug

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114495545A (zh) * 2022-01-28 2022-05-13 常州海蓝利科物联网技术有限公司 车辆控制系统及方法
JP2025513697A (ja) * 2022-03-16 2025-04-30 イージス ライダー エージー ヘルメットの位置及び向きを特定するための方法及びシステム
CN118349885B (zh) * 2024-04-16 2024-11-26 杭州脉尖智能科技有限公司 一种基于神经网络的人员分类识别方法以及系统

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
IL99697A0 (en) 1991-10-09 1993-05-13 Nir Tsook Helmet mounted display device
IL190674A (en) 2008-04-07 2013-08-29 Janus Interface Ltd Vehicle alerting system
US20130311075A1 (en) * 2012-05-18 2013-11-21 Continental Automotive Systems, Inc. Motorcycle and helmet providing advance driver assistance
US20130305437A1 (en) 2012-05-19 2013-11-21 Skully Helmets Inc. Augmented reality motorcycle helmet
US10219571B1 (en) * 2012-11-08 2019-03-05 Peter Aloumanis In helmet sensors providing blind spot awareness
US20140273863A1 (en) 2013-03-15 2014-09-18 Luizzi Bros. Sealcoating & Striping Llc Smart helmet with mobile communicator integration
DE102014218577A1 (de) 2014-09-16 2016-03-17 Robert Bosch Gmbh Sicherheitseinrichtung für Motorradfahrer, Verfahren zum Betreiben einer Sicherheitseinrichtung für Motorradfahrer
KR20170117098A (ko) * 2015-02-10 2017-10-20 라이더 시스템즈 엘엘씨 자동차에 대한 근접성 인식 시스템
JP2017004426A (ja) 2015-06-15 2017-01-05 真人 田村 ヘッドアップディスプレーを備えた交通安全システム
US10324290B2 (en) 2015-12-17 2019-06-18 New Skully, Inc. Situational awareness systems and methods

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102024205672A1 (de) * 2024-06-19 2025-12-24 Volkswagen Aktiengesellschaft Verfahren und Assistenzsystem zum Unterstützen eines Fahrers in einer gemischten Verkehrsumgebung und entsprechend eingerichtetes Kraftfahrzeug
EP4668244A1 (de) * 2024-06-19 2025-12-24 Volkswagen Ag Verfahren und assistenzsystem zum unterstützen eines fahrers in einer gemischten verkehrsumgebung und entsprechend eingerichtetes kraftfahrzeug

Also Published As

Publication number Publication date
EP3799752A1 (de) 2021-04-07

Similar Documents

Publication Publication Date Title
EP3799752B1 (de) Ego-motorrad-on-board-sensibilisierungssystem, verfahren zum erfassen und anzeigen des vorhandenseins von autonomen fahrzeugen
US11361687B2 (en) Advertisement display device, vehicle, and advertisement display method
US10210406B2 (en) System and method of simultaneously generating a multiple lane map and localizing a vehicle in the generated map
US10336257B2 (en) Rear vision system for a vehicle and method of using the same
US12013254B2 (en) Control device
JP4277217B2 (ja) 接近移動体表示装置、システム及び方法、並びに衝突情報提供装置及び方法
JP6496018B2 (ja) 交通標識の有効性確認システム及び方法
US20160297449A1 (en) Method and device for detecting the alertness of a vehicle driver
US12590804B2 (en) Map features delivered via augmented reality (AR)
JP6415583B2 (ja) 情報表示制御システムおよび情報表示制御方法
JP5327025B2 (ja) 車両用走行案内装置、車両用走行案内方法及びコンピュータプログラム
US20180037162A1 (en) Driver assistance system
US20190147269A1 (en) Information processing apparatus, driver monitoring system, information processing method and computer-readable storage medium
CN110544368B (zh) 一种疲劳驾驶增强现实预警装置及预警方法
WO2017115371A1 (en) Apparatus and method for avoiding vehicular accidents
JP6589991B2 (ja) ヒューマンインターフェース
CN113706883A (zh) 一种隧道路段安全行车系统和方法
US20220371533A1 (en) Distributed vehicle body sensors for event detection
KR20150072942A (ko) 차량 운전 보조 장치 및 이를 구비한 차량
CN110319848B (zh) 输出装置
JP6526191B2 (ja) 交通標識支援システム及び方法
JP2008250453A (ja) 運転支援装置および運転支援方法
JP2025004086A (ja) 情報収集装置、情報収集サーバ及び情報収集システム
JP2020051943A (ja) 車両用ステレオカメラ装置
US20240288535A1 (en) Motorcycle radar system

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20211007

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20220209

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1502213

Country of ref document: AT

Kind code of ref document: T

Effective date: 20220715

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R081

Ref document number: 602019016667

Country of ref document: DE

Owner name: CONTINENTAL AUTONOMOUS MOBILITY GERMANY GMBH, DE

Free format text: FORMER OWNER: CONTINENTAL AUTOMOTIVE GMBH, 30165 HANNOVER, DE

Ref country code: DE

Ref legal event code: R081

Ref document number: 602019016667

Country of ref document: DE

Owner name: AUMOVIO AUTONOMOUS MOBILITY GERMANY GMBH, DE

Free format text: FORMER OWNER: CONTINENTAL AUTOMOTIVE GMBH, 30165 HANNOVER, DE

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602019016667

Country of ref document: DE

RAP2 Party data changed (patent owner data changed or rights of a patent transferred)

Owner name: CONTINENTAL AUTONOMOUS MOBILITY GERMANY GMBH

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20220706

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221107

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221006

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221106

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221007

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602019016667

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

26N No opposition filed

Effective date: 20230411

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20221031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221002

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221002

REG Reference to a national code

Ref country code: AT

Ref legal event code: UEP

Ref document number: 1502213

Country of ref document: AT

Kind code of ref document: T

Effective date: 20220706

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20191002

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20231002

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20231002

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20231002

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220706

REG Reference to a national code

Ref country code: DE

Ref legal event code: R081

Ref document number: 602019016667

Country of ref document: DE

Owner name: AUMOVIO AUTONOMOUS MOBILITY GERMANY GMBH, DE

Free format text: FORMER OWNER: CONTINENTAL AUTONOMOUS MOBILITY GERMANY GMBH, 85057 INGOLSTADT, DE

REG Reference to a national code

Ref country code: CH

Ref legal event code: U11

Free format text: ST27 STATUS EVENT CODE: U-0-0-U10-U11 (AS PROVIDED BY THE NATIONAL OFFICE)

Effective date: 20251101

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20251031

Year of fee payment: 7

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: AT

Payment date: 20251017

Year of fee payment: 7

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20251024

Year of fee payment: 7

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: CH

Payment date: 20251101

Year of fee payment: 7