EP2960883B1 - Determination of at least one vehicle feature - Google Patents

Determination of at least one vehicle feature Download PDF

Info

Publication number
EP2960883B1
EP2960883B1 EP14173551.4A EP14173551A EP2960883B1 EP 2960883 B1 EP2960883 B1 EP 2960883B1 EP 14173551 A EP14173551 A EP 14173551A EP 2960883 B1 EP2960883 B1 EP 2960883B1
Authority
EP
European Patent Office
Prior art keywords
vehicle
image
line scan
scan camera
speed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP14173551.4A
Other languages
German (de)
French (fr)
Other versions
EP2960883A1 (en
Inventor
Andreas Jung
Marco Bartiromo
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vitronic Dr Ing Stein Bildverarbeitungssysteme GmbH
Original Assignee
Vitronic Dr Ing Stein Bildverarbeitungssysteme GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to ES14173551.4T priority Critical patent/ES2645504T3/en
Priority to NO14173551A priority patent/NO2960883T3/no
Application filed by Vitronic Dr Ing Stein Bildverarbeitungssysteme GmbH filed Critical Vitronic Dr Ing Stein Bildverarbeitungssysteme GmbH
Priority to EP14173551.4A priority patent/EP2960883B1/en
Priority to PL14173551T priority patent/PL2960883T3/en
Priority to DK14173551.4T priority patent/DK2960883T3/en
Priority to HUE14173551A priority patent/HUE034573T2/en
Priority to LTEP14173551.4T priority patent/LT2960883T/en
Priority to SI201430448T priority patent/SI2960883T1/en
Priority to PT141735514T priority patent/PT2960883T/en
Priority to AU2015279273A priority patent/AU2015279273B2/en
Priority to CA2953411A priority patent/CA2953411A1/en
Priority to RU2017101816A priority patent/RU2017101816A/en
Priority to US15/321,370 priority patent/US10515544B2/en
Priority to PCT/EP2015/064147 priority patent/WO2015197641A1/en
Priority to CN201580033839.8A priority patent/CN106663372A/en
Publication of EP2960883A1 publication Critical patent/EP2960883A1/en
Application granted granted Critical
Publication of EP2960883B1 publication Critical patent/EP2960883B1/en
Priority to CY20171101095T priority patent/CY1119472T1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
    • G08G1/0175—Detecting movement of traffic to be counted or controlled identifying vehicles by photographing vehicles, e.g. when violating traffic rules
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/04—Detecting movement of traffic to be counted or controlled using optical or ultrasonic detectors

Definitions

  • Arrangements and methods for detecting vehicles moving along a traffic lane with the aid of at least one camera and / or other devices for detecting the contour or spatial structure of the vehicle and optionally its speed are known, for example, from various applications for monitoring or traffic detection.
  • the detection and tracking of the vehicle by means of a LIDAR system which is estimated from the LIDAR data path and speed of the vehicle and are assigned to the subsequently determined contour and structure data.
  • the device comprises a LIDAR system, which is aligned to detect a vehicle at a distance from the receiving position of the camera and / or the other devices.
  • the arrangement according to the invention for determining at least one feature of a vehicle moving along a lane has a light-sensitive sensor for recording an image of the vehicle and an evaluation device for detecting the at least one feature on the image.
  • the photosensitive sensor is designed as a line camera, this line camera is arranged laterally next to the lane.
  • a conventional area camera detects a vehicle driving a lane in an image, for example with induction loops as a trigger for the camera, however, such an image is distorted in perspective in both image dimensions.
  • the use of a line camera has the advantage that the distance from the recorded part of the vehicle to the line scan camera remains essentially the same, so that neither a perspective distortion in the direction of the juxtaposed shots, nor a different light yield due to the distance between the vehicle and camera occurs , For this reason, it is provided according to the invention to use a line camera for the arrangement laterally next to the lane.
  • the position to the side of the lane allows a particularly effective recognition of numerous features of vehicles in the image, in particular their length, number of axes, characteristic shape or side lettering, which may be helpful for identification. Many of these features are not visible, for example, from a position above the lane.
  • the image is composed of successive shots of the line camera, the line scan camera works with a predetermined, possibly also with an adjustable sampling rate.
  • a further advantage is that the line scan camera with the evaluation device can be used for image evaluation without further measuring devices. Beginning and end of a vehicle can be detected by evaluating the continuously recorded line scans, so that advantageously no triggering of the recording must be made in order to obtain a complete image of the vehicle.
  • a camera type is to be understood which has only one light-sensitive line, a so-called line sensor, in contrast to the two-dimensional sensor, which has a plurality of lines.
  • Line sensors are light- or radiation-sensitive detectors, usually semiconductor detectors, which consist of a one-dimensional array of photodetectors or other detector elements.
  • the line scan camera is aligned obliquely to the traffic lane.
  • the orientation of the line scan camera is to be understood as the direction in which the light-sensitive line is aligned in order to receive light.
  • the vehicle moves in a direction of travel along the lane.
  • the arranged next to the lane line scan camera can thus record its side surface.
  • the line scan camera is aligned obliquely to the lane in the context of the invention.
  • a line camera oriented at an acute angle to the traffic lane in the direction opposite to the direction of travel of the vehicle, first the front of the vehicle and then its side surface are picked up, which has the advantage that recognition of the license plate and possible dangerous goods markings on the front of the vehicle Vehicle is enabled.
  • the line scan camera is likewise aligned obliquely to the traffic lane in the sense of the invention.
  • the side surface of the vehicle is picked up first and then its rear side, which also allows a license plate identification on the rear license plate.
  • the lines camera is aligned at an angle of about 30 ° to the lane, against the direction of travel of the vehicle, the angle of the line camera to a straight front of the vehicle is thus about 60 °.
  • the alignment of the line scan camera does not mean the orientation of the line sensor itself, which is generally arranged vertically, ie essentially at right angles to a plane formed by the traffic lane. Positioning at an angle of significantly less than 90 ° to the vertical would, however, also be conceivable in principle for the line sensor.
  • a lighting device is provided, wherein the lighting device is aligned at an angular distance of at least 7 ° from the orientation of the line scan camera.
  • the lighting device is used for illumination in low light conditions, especially at night.
  • An angular distance of at least 7 ° from the orientation of the line scan camera means that the light emitted by the illumination device, reflected by the vehicle and detected by the line scan camera, is deflected by at least 7 ° in the reflection.
  • the inventive method for determining at least one feature of a moving along a lane vehicle by means of the arrangement described hereinbefore provides that an image of the vehicle is taken with the line scan camera and the image is evaluated for detecting at least one feature of the vehicle.
  • An advantage of the method is that an identification of vehicles is made possible only with the aid of the line scan camera and an evaluation of the image.
  • the individual recorded lines or line scans are lined up in an image horizon, so that the image horizon corresponds to a time axis. Lines appearing along the horizontal image thus display a picture that is unchanged in time, which indicates that no vehicle passes this pixel of the line scan camera.
  • Such horizontal lines, or in particular an end of a horizontal line can be advantageously used for automatic detection of a moving vehicle.
  • an envelope including the image of the vehicle is created during the evaluation.
  • This can be at least one calculate the height, length and width of the vehicle taking into account a scanning rate of the line scan camera and a speed of the vehicle.
  • the speed of the vehicle can be determined based on an extension or compression of the image in the horizontal image relative to the actual aspect ratios on the vehicle, or be determined by means of a separate speed measurement.
  • a number and / or positions of wheels and / or axles of the vehicle are determined by finding round objects during the evaluation of the image.
  • an ellipticity is preferably determined by round objects found during the evaluation of the image.
  • a correction of the image is made by stretching or compression in the image horizontal.
  • a speed of the vehicle is calculated. This calculated speed is another feature of the vehicle, which can advantageously be determined by means of the line scan camera as the sole measuring device. Alternatively, the calculated speed may also be used to check a speed measured by a separate speed measuring device.
  • a classification and / or identification of the vehicle based on at least one of the following, obtained by evaluation of the image features is made: dimensions, number of axles, license plate, dangerous goods marking, shape of a windscreen and / or a side window and characteristic shapes of the vehicle.
  • FIG. 1 is a plan view of the lane 1 and the vehicle 2, while in the FIG. 2 a side view of the vehicle 2 is shown.
  • the arrangement according to the invention comprises a photosensitive sensor 3 for recording an image of the vehicle 2 and an evaluation device 4 for detecting the at least one feature on the image, wherein the evaluation device 4 can also be arranged remotely from the sensor 3.
  • the photosensitive sensor 3 is designed according to the invention as a line camera 3, wherein the line camera 3 is arranged laterally next to the traffic lane 1.
  • the arrangement according to the invention is particularly advantageously suitable for a method for axle classification or axle counting of vehicles 2 in flowing traffic.
  • the method is generally used for the classification of vehicles 2, in particular for the recognition of distinctive features, such as the vehicle manufacturer or dangerous goods indicator in motion.
  • an image of the mark suitable for automatic mark reading processing is taken.
  • an image of the vehicle 2 is created as an overview image of the entire scene.
  • four tasks from the field of traffic monitoring can be carried out by means of the one Line camera 3 side of the lane 1 are performed.
  • the line camera 3 is laterally arranged next to the traffic lane 1 so that the vehicle front 5 and the vehicle flank 6 are received cyclically by the passing vehicle 2.
  • an illumination 7 is preferably directed to the vehicle 2.
  • the angle between the line camera 3 and the vehicle 2 is generally freely selectable and can, depending on the application, be adjusted.
  • the aim is generally an angle as close as possible to 90 ° to the surface to be monitored. Since in the illustrated embodiment, both the vehicle front 5, and the vehicle edge 6 are to be recorded, an angle of about 60 ° on the front 5 offers, since this contains more detail information, as the edge 6, from which the light accordingly a more acute angle of 30 ° in the line camera 3 occurs.
  • ⁇ of more than 7 °.
  • an overexposure by the so-called retroreflective effect of license plates is advantageously avoided.
  • an image of the vehicle 2 is generated, thereby enabling a simple image processing of the same, since no partial images must be joined together or superimposed.
  • the method according to the invention is also advantageously suitable for determining a speed of the vehicle 2. Depending on how elliptically the wheels 8 are indicated schematically in the image, can be recalculated to the speed of the vehicle 2 from this.
  • the speed sensor 9 and the line camera 3 are located at the same position laterally beside the road 1.
  • the speed sensor 9 preferably provides the following data of the vehicle: position relative to the sensor, distance and speed of the vehicle. 2
  • FIG. 3 an image of a vehicle 2 produced by the method according to the invention is shown.
  • the image horizontals by a temporal sequence of successive line images along the replaced with t time axis and is therefore free from perspective distortion.
  • the distance from one pixel to the next pixel in the direction of the t-axis corresponds to the scanning rate or line frequency of the line camera 3 and is preferably adapted to the speed of the vehicle 2 determined by the speed sensor 9, so that no compressed or stretched images in the direction of the t-axis. Axis arise.
  • the image vertical in the direction of the Y axis corresponds to the orientation of the line sensor. 3
  • FIG. 3 shows the image of a truck 2 in motion.
  • the silhouette or envelope H of the vehicle 2 can be advantageously determined by simple edge filters. Even shadows passing through the image can be easily identified and will not lead to misjudgments, as is usually the case with Still Cameras, as no temporal reference can be made.
  • the geometric ones Information such as length, and / or height can be advantageously determined with the help of this edge image. If the positions of the vehicle axles are already known, it can advantageously be ruled out that the front of the vehicle 2 is also measured. The measurement of the vehicle length starts, for example, with the first detected axis. Depending on the type of vehicle identified, such as truck or car, the length can be corrected by an averaged factor.
  • Line scan cameras can operate at line frequencies up to 80 kHz, and thus advantageously offer more advanced capabilities that leverage data.
  • This invention enables with the line scan camera 3 as a single sensor and with only one data acquisition cycle to generate all the data needed, for example, for toll collection.
  • These are in detail, for example: recording a license plate image, recording an overview image, classification of the vehicle based on its axes and the vehicle geometry. It is also possible to recognize dangerous goods identifiers and to recognize the vehicle manufacturer. The fact that all data is generated in one step results in consistent data records that are integer. A misallocation of the data from multiple sensors can be advantageously avoided.

Description

Die Erfindung betrifft eine Anordnung zur Bestimmung mindestens eines Merkmals eines sich entlang einer Fahrspur bewegenden Fahrzeugs, mit einem lichtempfindlichen Sensor zur Aufnahme eines Abbilds des Fahrzeugs und mit einer Auswertungsvorrichtung zur Erkennung des mindestens einen Merkmals auf dem Abbild, sowie ein Verfahren zur Bestimmung mindestens eines Merkmals eines sich entlang einer Fahrspur bewegenden Fahrzeugs mittels einer solchen Anordnung, wobei ein Abbild des Fahrzeugs mit einer Zeilenkamera aufgenommen wird und das Abbild zur Erkennung mindestens eines Merkmals des Fahrzeugs ausgewertet wird.The invention relates to an arrangement for determining at least one feature of a moving vehicle along a lane, with a photosensitive sensor for receiving an image of the vehicle and with an evaluation device for detecting the at least one feature on the image, and a method for determining at least one feature a moving along a lane vehicle by means of such an arrangement, wherein an image of the vehicle is taken with a line scan camera and the image is evaluated for detecting at least one feature of the vehicle.

Anordnungen und Verfahren zum Erfassen von sich entlang einer Fahrspur bewegenden Fahrzeugen mit Hilfe mindestens einer Kamera und/oder sonstigen Einrichtungen zur Erfassung der Kontur oder räumlichen Struktur des Fahrzeuges und gegebenenfalls seiner Geschwindigkeit sind beispielsweise aus verschiedenen Anwendungen zur Überwachung oder Verkehrserfassung bekannt.Arrangements and methods for detecting vehicles moving along a traffic lane with the aid of at least one camera and / or other devices for detecting the contour or spatial structure of the vehicle and optionally its speed are known, for example, from various applications for monitoring or traffic detection.

Automatische Kontrollsysteme für Straßenmaut, beispielsweise, die auf der Kommunikation zwischen Fahrzeug und einer entsprechenden Kommunikationseinrichtung an einer Mautstelle beruhen, setzen voraus, dass in jedem mautpflichtigen Fahrzeug eine entsprechende Kommunikationseinrichtung vorhanden ist, aus welcher die wesentlichen Daten des Fahrzeuges abgerufen werden können, die seiner Identifikation dienen, wobei außerdem auch noch die benutzten Straßenabschnitte bei jeder Kontrolle erfasst und/oder übermittelt werden sollten. Das System muss in der Lage sein, jedes Fahrzeug, welches mautpflichtig ist, eindeutig als solches zu identifizieren. Das heißt, dass zum einen solche Fahrzeuge nicht erfasst werden dürfen, die der Mautpflicht nicht unterliegen und zum anderen, dass alle mautpflichtigen Fahrzeuge als solche erfasst werden. Um ein Verfahren und eine Vorrichtung zum Erfassen von Fahrzeugen in Bewegung zu schaffen, welche in der Lage sind, eine automatische Unterscheidung zwischen mautpflichtigen und nicht mautpflichtigen Fahrzeugen vorzunehmen, um so die unnötige Erfassung großer Datenmengen zu vermeiden, ist gemäß der Druckschrift DE 101 48 289 A1 vorgesehen, dass vor dem Erfassen der Kontur und/oder räumlichen Struktur die Erfassung und Verfolgung des Fahrzeuges mittels eines LIDAR-Systems erfolgt, wobei aus den LIDAR-Daten Weg und Geschwindigkeit des Fahrzeuges abgeschätzt werden und den anschließend ermittelten Kontur- und Strukturdaten zugeordnet werden. Die Vorrichtung weist ein LIDAR-System auf, welches so ausgerichtet ist, dass es ein Fahrzeug im Abstand vor der Aufnahmeposition der Kamera und/oder der sonstigen Einrichtungen erfasst. Weiterhin ist eine Recheneinheit vorgesehen, welche mit dem LIDAR verbunden ist und aus den LIDAR-Daten Weg und Geschwindigkeit des Fahrzeuges prognostiziert, diese Daten gegebenenfalls korrigiert und auf der Basis dieser Daten eine Zuordnung zu den Daten der sonstigen Einrichtung und/oder dem erfassten Videobild vornimmt. Ein Nachteil besteht darin, dass bei der Kombination aus einem LIDAR-System und einer Video-Bilderfassung die jeweiligen Ergebnisse sicher und nachvollziehbar einander zugeordnet werden müssen, was vergleichsweise aufwändig ist. Bei der Auswertung der Videobilder besteht außerdem das Problem, dass diese durch die perspektivische Verzerrung erschwert wird, zum einen durch die Verzerrung selbst und zum anderen durch die mit Abstand von der Kamera abnehmenden Lichtausbeute, die zu einer deutlich schwächeren Ausleuchtung der entfernter von der Kamera befindlichen Bereiche des Fahrzeugs führt. Dies fällt insbesondere bei längeren Fahrzeugen, wie Lastkraftwagen, ins Gewicht. Eine Anordnung nach dem Oberbegriff des Anspruchs 1 ist sowohl aus der WO 2012/152 596 A1 als auch aus der EP 2 703 823 A1 bekannt. Eine Aufgabe der vorliegenden Erfindung besteht darin, eine vereinfachte Anordnung und ein Verfahren zur Bestimmung mindestens eines Merkmals eines sich entlang einer Fahrspur bewegenden Fahrzeugs anzugeben, welche eine ebenso zuverlässige oder verbesserte Merkmalerkennung erlauben, wie der Stand der Technik, beispielsweise zur automatischen Unterscheidung zwischen mautpflichtigen und nicht mautpflichtigen Fahrzeugen.Automatic control systems for road tolls, for example, based on the communication between the vehicle and a corresponding communication device at a toll booth, require that in each toll vehicle there is a corresponding communication device from which the essential data of the vehicle can be retrieved, that of its identification In addition, the used road sections should also be recorded and / or transmitted during each inspection. The system must be able to clearly identify each vehicle that is subject to toll as such. This means that, on the one hand, vehicles that are not subject to the toll obligation and, on the other, that all toll vehicles are recorded as such. In order to provide a method and a device for detecting vehicles in motion, which are able to make an automatic distinction between toll and non-toll vehicles, so as to avoid the unnecessary detection of large amounts of data, is according to the document DE 101 48 289 A1 provided that before the detection of the contour and / or spatial structure, the detection and tracking of the vehicle by means of a LIDAR system, which is estimated from the LIDAR data path and speed of the vehicle and are assigned to the subsequently determined contour and structure data. The device comprises a LIDAR system, which is aligned to detect a vehicle at a distance from the receiving position of the camera and / or the other devices. Furthermore, an arithmetic unit is provided, which is connected to the LIDAR and predicts the path and speed of the vehicle from the LIDAR data, optionally corrects these data and makes an assignment to the data of the other device and / or the captured video image on the basis of this data , A disadvantage is that the combination of a LIDAR system and a video image capture the respective results must be securely and comprehensibly assigned to each other, which is relatively expensive. In the evaluation of the video images is also the problem that this is complicated by the perspective distortion, on the one hand by the distortion itself and on the other by the distance from the camera decreasing light output, which are located to a much weaker illumination of the remote from the camera Areas of the vehicle leads. This is particularly significant for longer vehicles, such as trucks. An arrangement according to the preamble of claim 1 is known from both WO 2012/152 596 A1 as well as from the EP 2 703 823 A1 known. An object of the present invention is to provide a simplified arrangement and a method for determining at least one feature of a moving along a lane vehicle, which allow an equally reliable or improved feature recognition, such as the prior art, for example, for automatic distinction between toll and non-tolled vehicles.

Die Aufgabe wird durch die Anordnung und das Verfahren gemäß den unabhängigen Ansprüchen gelöst. In den jeweiligen Unteransprüchen sind bevorzugte Ausführungsformen und vorteilhafte Weiterbildungen angegeben.The object is achieved by the arrangement and the method according to the solved independent claims. In the respective subclaims preferred embodiments and advantageous developments are given.

Die erfindungsgemäße Anordnung zur Bestimmung mindestens eines Merkmals eines sich entlang einer Fahrspur bewegenden Fahrzeugs weist einen lichtempfindlichen Sensor zur Aufnahme eines Abbilds des Fahrzeugs und eine Auswertungsvorrichtung zur Erkennung des mindestens einen Merkmals auf dem Abbild auf. Erfindungsgemäß ist vorgesehen, dass der lichtempfindliche Sensor als Zeilenkamera ausgeführt ist, wobei diese Zeilenkamera seitlich neben der Fahrspur angeordnet ist.The arrangement according to the invention for determining at least one feature of a vehicle moving along a lane has a light-sensitive sensor for recording an image of the vehicle and an evaluation device for detecting the at least one feature on the image. According to the invention it is provided that the photosensitive sensor is designed as a line camera, this line camera is arranged laterally next to the lane.

Zwar ist es mit einer herkömmlichen Flächenkamera grundsätzlich möglich, ein eine Fahrspur befahrendes Fahrzeug in einem Bild zu erfassen, z.B. mit Induktionsschleifen als Auslöser für die Kamera, jedoch ist ein solches Bild perspektivisch in beiden Bilddimensionen verzerrt. Die Verwendung einer Zeilenkamera hat den Vorteil, dass der Abstand von dem aufgenommenen Teil des Fahrzeugs zu der Zeilenkamera im Wesentlichen gleich bleibt, so dass weder eine perspektivische Verzerrung in Richtung der aneinander gereihten Aufnahmen, noch eine unterschiedliche Lichtausbeute aufgrund der Entfernung zwischen Fahrzeug und Kamera auftritt. Aus diesem Grund ist es erfindungsgemäss vorgesehen, eine Zeilenkamera für die Anordnung seitlich neben der Fahrspur zu verwenden. Die Position seitlich neben der Fahrspur erlaubt eine besonders effektive Erkennung zahlreicher Merkmale von Fahrzeugen in dem Abbild, wie insbesondere deren Länge, Anzahl der Achsen, charakteristische Form oder seitliche Beschriftung, die zu einer Identifizierung hilfreich sein können. Viele dieser Merkmale sind beispielsweise aus einer Position oberhalb der Fahrspur nicht erkennbar. Das Abbild wird dabei aus aufeinanderfolgenden Aufnahmen der Zeilenkamera zusammengesetzt, wobei die Zeilenkamera mit einer vorgegebenen, gegebenenfalls auch mit einer anpassbaren Abtastrate arbeitet. Ein weiterer Vorteil besteht darin, dass die Zeilenkamera mit der Auswertevorrichtung zur Bildauswertung ohne weitere Messgeräte verwendet werden kann. Beginn und Ende eines Fahrzeugs können durch Auswertung der laufend aufgenommenen Zeilenscans erkannt werden, so dass vorteilhaft keine Auslösung der Aufnahme erfolgen muss, um ein vollständiges Abbild des Fahrzeugs zu erhalten. Da die Abtastrate der Zeilenkamera bekannt ist, kann anhand einer Referenz im Abbild auch die Geschwindigkeit des Fahrzeugs festgestellt werden, da bei einer konstanten Abtastrate das Abbild eines Fahrzeugs, je nach Geschwindigkeit des Fahrzeugs, in der Bildhorizontalen, welche einer Zeitachse entspricht, gestaucht oder gestreckt gegenüber den tatsächlichen Seitenverhältnissen des Fahrzeugs dargestellt wird. Als Referenz kann somit beispielsweise eine bekannte Länge eines Objekts an dem Fahrzeug dienen. Besonders vorteilhaft kann auch jedes Rad als Referenz dienen. Da die mit ausreichender Genauigkeit kreisrunden Räder des Fahrzeugs in dem Abbild, je nach Geschwindigkeit des Fahrzeugs, durch die Stauchung oder Streckung entweder in der Höhe oder in der Breite elliptisch dargestellt werden, lässt sich durch eine Berechnung der Elliptizität auf die Geschwindigkeit des Fahrzeugs schließen. Die Elliptizität wird auch als Exzentrizität bezeichnet, die den Abstand der Brennpunkte einer Ellipse von ihrem Mittelpunkt angibt.Although it is in principle possible with a conventional area camera to detect a vehicle driving a lane in an image, for example with induction loops as a trigger for the camera, however, such an image is distorted in perspective in both image dimensions. The use of a line camera has the advantage that the distance from the recorded part of the vehicle to the line scan camera remains essentially the same, so that neither a perspective distortion in the direction of the juxtaposed shots, nor a different light yield due to the distance between the vehicle and camera occurs , For this reason, it is provided according to the invention to use a line camera for the arrangement laterally next to the lane. The position to the side of the lane allows a particularly effective recognition of numerous features of vehicles in the image, in particular their length, number of axes, characteristic shape or side lettering, which may be helpful for identification. Many of these features are not visible, for example, from a position above the lane. The image is composed of successive shots of the line camera, the line scan camera works with a predetermined, possibly also with an adjustable sampling rate. A further advantage is that the line scan camera with the evaluation device can be used for image evaluation without further measuring devices. Beginning and end of a vehicle can be detected by evaluating the continuously recorded line scans, so that advantageously no triggering of the recording must be made in order to obtain a complete image of the vehicle. Since the sampling rate of the line scan camera is known, can be determined by a Reference can be detected in the image and the speed of the vehicle, since at a constant sampling rate, the image of a vehicle, depending on the speed of the vehicle, in the image horizontal, which corresponds to a time axis, compressed or stretched over the actual aspect ratio of the vehicle is displayed. Thus, for example, a known length of an object on the vehicle can serve as a reference. Particularly advantageous, each wheel can serve as a reference. Since the circularly accurate wheels of the vehicle in the image are elliptical in the image, depending on the speed of the vehicle, due to the compression or extension either in height or in width, the speed of the vehicle can be calculated by calculating the ellipticity. Ellipticity is also referred to as eccentricity, which indicates the distance of the focal points of an ellipse from its center.

Seitlich neben der Fahrspur, ist im Sinne der Erfindung dahingehend zu verstehen, dass die Zeilenkamera nicht vertikal über der in der Regel im Wesentlichen horizontal verlaufenden Fahrspur angeordnet ist. Dennoch kann die Zeilenkamera verständlicherweise neben der Fahrspur in einer bestimmten Höhe über dem Niveau der Fahrspur angeordnet sein.Laterally next to the lane is to be understood in the sense of the invention to the effect that the line scan camera is not arranged vertically above the usually substantially horizontal lane. Nevertheless, the line scan camera can understandably be arranged next to the lane at a certain height above the level of the lane.

Als Zeilenkamera im Sinne der Erfindung ist ein Kameratyp zu verstehen, der nur eine lichtempfindliche Zeile, einen sogenannten Zeilensensor aufweist, im Gegensatz zum zweidimensionalen Sensor, der über eine Vielzahl von Zeilen verfügt. Zeilensensoren sind licht- bzw. strahlungsempfindliche Detektoren, in der Regel Halbleiterdetektoren, die aus einem eindimensionalen Feld aus Fotodetektoren oder anderen Detektorelementen bestehen. Erfindungsgemäß ist vorgesehen, dass die Zeilenkamera schräg zu der Fahrspur ausgerichtet ist. Unter der Ausrichtung der Zeilenkamera ist im Sinne der Erfindung diejenige Richtung zu verstehen, in welche die lichtempfindliche Zeile ausgerichtet ist, um Licht zu empfangen. Das Fahrzeug bewegt sich in einer Fahrtrichtung entlang der Fahrspur. Die neben der Fahrspur angeordnete Zeilenkamera kann somit dessen Seitenfläche aufnehmen. Bei einem Winkel ungleich 90° zwischen Fahrspur und Zeilenkamera ist die Zeilenkamera im Sinne der Erfindung schräg zu der Fahrspur ausgerichtet. In dem bevorzugten Fall einer in einem spitzen Winkel zu der Fahrspur, entgegen der Fahrtrichtung des Fahrzeugs ausgerichteten Zeilenkamera, wird zunächst die Vorderfront des Fahrzeugs und anschließend dessen Seitenfläche aufgenommen, was den Vorteil hat, dass eine Erkennung des Kennzeichens und möglicher Gefahrgutkennzeichnungen an der Vorderfront des Fahrzeugs ermöglicht wird. Bei einer Ausrichtung der Zeilenkamera in einem stumpfen Winkel zur Fahrspur in Fahrtrichtung des Fahrzeugs ist die Zeilenkamera im Sinne der Erfindung ebenfalls schräg zu der Fahrspur ausgerichtet. In diesem ebenfalls bevorzugten Fall wird die Seitenfläche des Fahrzeugs zuerst und anschließend dessen Rückseite aufgenommen, was ebenfalls eine Kennzeichenidentifizierung anhand des hinteren Kennzeichenschilds erlaubt. Besonders bevorzugt ist vorgesehen, dass die Zeilen kamera in einem Winkel von etwa 30° zu der Fahrspur, entgegen der Fahrtrichtung des Fahrzeugs ausgerichtet ist, Der Winkel der Zeilenkamera zu einer geraden Fahrzeugfront beträgt somit etwa 60°.As a line scan camera according to the invention, a camera type is to be understood which has only one light-sensitive line, a so-called line sensor, in contrast to the two-dimensional sensor, which has a plurality of lines. Line sensors are light- or radiation-sensitive detectors, usually semiconductor detectors, which consist of a one-dimensional array of photodetectors or other detector elements. According to the invention, it is provided that the line scan camera is aligned obliquely to the traffic lane. For the purposes of the invention, the orientation of the line scan camera is to be understood as the direction in which the light-sensitive line is aligned in order to receive light. The vehicle moves in a direction of travel along the lane. The arranged next to the lane line scan camera can thus record its side surface. at an angle not equal to 90 ° between the lane and line camera, the line scan camera is aligned obliquely to the lane in the context of the invention. In the preferred case of a line camera oriented at an acute angle to the traffic lane, in the direction opposite to the direction of travel of the vehicle, first the front of the vehicle and then its side surface are picked up, which has the advantage that recognition of the license plate and possible dangerous goods markings on the front of the vehicle Vehicle is enabled. With an alignment of the line scan camera at an obtuse angle to the traffic lane in the direction of travel of the vehicle, the line scan camera is likewise aligned obliquely to the traffic lane in the sense of the invention. In this likewise preferred case, the side surface of the vehicle is picked up first and then its rear side, which also allows a license plate identification on the rear license plate. It is particularly preferred that the lines camera is aligned at an angle of about 30 ° to the lane, against the direction of travel of the vehicle, the angle of the line camera to a straight front of the vehicle is thus about 60 °.

Unter der Ausrichtung der Zeilenkamera ist im Sinne der Erfindung nicht die Ausrichtung des Zeilensensors selbst gemeint, der in der Regel vertikal, also im Wesentlichen rechtwinklig zu einer durch die Fahrspur gebildeten Ebene angeordnet ist. Eine Positionierung unter einem Winkel von deutlich weniger als 90° zur Vertikalen wäre für den Zeilensensor aber grundsätzlich auch denkbar.For the purposes of the invention, the alignment of the line scan camera does not mean the orientation of the line sensor itself, which is generally arranged vertically, ie essentially at right angles to a plane formed by the traffic lane. Positioning at an angle of significantly less than 90 ° to the vertical would, however, also be conceivable in principle for the line sensor.

Gemäß einer weiteren bevorzugten Ausführungsform der Anordnung ist eine Beleuchtungsvorrichtung vorgesehen, wobei die Beleuchtungsvorrichtung in einem Winkelabstand von mindestens 7° von der Ausrichtung der Zeilenkamera ausgerichtet ist. Die Beleuchtungsvorrichtung dient zur Ausleuchtung bei schwachen Lichtverhältnissen, insbesondere bei Nacht. Ein Winkelabstand von mindestens 7° von der Ausrichtung der Zeilenkamera bedeutet, dass das Licht, welches von der Beleuchtungsvorrichtung abgestrahlt, von dem Fahrzeug zurückgeworfen und von der Zeilenkamera detektiert wird, um mindestens 7° bei der Reflexion umgelenkt wird. Dadurch lässt sich ein übermäßiger, eine Auswertung behindernder Lichteinfall von Kennzeichenschildern mit retroreflektierender Oberfläche vorteilhaft vermeiden. Für die Beleuchtung eignet sich sowohl sichtbares Licht, als auch Infrarotstrahlung.According to a further preferred embodiment of the arrangement, a lighting device is provided, wherein the lighting device is aligned at an angular distance of at least 7 ° from the orientation of the line scan camera. The lighting device is used for illumination in low light conditions, especially at night. An angular distance of at least 7 ° from the orientation of the line scan camera means that the light emitted by the illumination device, reflected by the vehicle and detected by the line scan camera, is deflected by at least 7 ° in the reflection. As a result, it is possible to advantageously avoid an excessive incidence of light from license plates with a retroreflecting surface, which hinders an evaluation. For the illumination, both visible light and infrared radiation are suitable.

Gemäß einer weiteren bevorzugten Ausführungsform der Anordnung ist eine Steuerung der Zeilenkamera vorgesehen, wobei eine Abtastrate der aufgenommenen Zeilen durch die Steuerung einstellbar ist. Weiterhin bevorzugt ist eine Geschwindigkeitsmessvorrichtung zur Messung einer Geschwindigkeit des Fahrzeugs bei der Aufnahme des Abbilds vorgesehen. Dadurch lässt sich besonders vorteilhaft eine Ausführungsform der Anordnung realisieren, bei der die Abtastrate durch die Steuerung an die mittels einer Geschwindigkeitsmessvorrichtung gemessene Geschwindigkeit des Fahrzeugs anpassbar ist. Dadurch wird eine Stauchung oder Streckung des Abbilds in der Bildhorizontalen zumindest weitgehend vermieden, so dass eine nachträgliche Korrektur, beispielsweise anhand der Elliptizität der abgebildeten Räder, entfallen kann oder zumindest wesentlich weniger umfangreich ausfallen kann. In einer alternativen Ausgestaltung kann die Messung der Geschwindigkeit durch eine Geschwindigkeitsmessvorrichtung bei einer festen Abtastrate der Zeilenkamera anhand der rechnerisch aus dem Abbild ermittelbaren Geschwindigkeit verifiziert werden.According to a further preferred embodiment of the arrangement, a control of the line camera is provided, wherein a sampling rate of the recorded lines is adjustable by the controller. Further preferably, a speed measuring device is provided for measuring a speed of the vehicle when taking the image. As a result, an embodiment of the arrangement can be realized particularly advantageously in which the sampling rate can be adapted by the controller to the speed of the vehicle measured by means of a speed measuring device. As a result, a compression or extension of the image in the horizontal image is at least largely avoided, so that a subsequent correction, for example on the basis of the ellipticity of the imaged wheels, can be omitted or at least substantially less extensive. In an alternative embodiment, the measurement of the speed can be verified by a speed measuring device at a fixed scanning rate of the line camera on the basis of the computationally determined from the image speed.

Das erfindungsgemäße Verfahren zur Bestimmung mindestens eines Merkmals eines sich entlang einer Fahrspur bewegenden Fahrzeugs mittels der hierin zuvor beschriebenen Anordnung sieht vor, dass ein Abbild des Fahrzeugs mit der Zeilenkamera aufgenommen wird und das Abbild zur Erkennung mindestens eines Merkmals des Fahrzeugs ausgewertet wird. Ein Vorteil des Verfahrens besteht darin, dass eine Identifizierung von Fahrzeugen nur mit Hilfe der Zeilenkamera und einer Auswertung des Abbilds ermöglicht wird. Dazu werden insbesondere die einzelnen aufgenommenen Zeilen oder Zeilenscans in einer Bildhorizontalen aneinander gereiht, so dass die Bildhorizontale einer Zeitachse entspricht. Entlang der Bildhorizontalen auftretende Linien zeigen somit ein zeitlich unverändertes Bild an, was darauf schließen lässt, dass kein Fahrzeug diesen Bildpunkt der Zeilenkamera passiert. Derartige horizontale Linien, bzw. insbesondere ein Ende einer horizontalen Linie lassen sich vorteilhaft zu einer automatischen Detektierung eines sich bewegenden Fahrzeugs heranziehen.The inventive method for determining at least one feature of a moving along a lane vehicle by means of the arrangement described hereinbefore provides that an image of the vehicle is taken with the line scan camera and the image is evaluated for detecting at least one feature of the vehicle. An advantage of the method is that an identification of vehicles is made possible only with the aid of the line scan camera and an evaluation of the image. For this purpose, in particular the individual recorded lines or line scans are lined up in an image horizon, so that the image horizon corresponds to a time axis. Lines appearing along the horizontal image thus display a picture that is unchanged in time, which indicates that no vehicle passes this pixel of the line scan camera. Such horizontal lines, or in particular an end of a horizontal line can be advantageously used for automatic detection of a moving vehicle.

Gemäß einer bevorzugten Ausführungsform wird bei der Auswertung eine das Abbild des Fahrzeugs einschließende Hüllkurve erstellt. Daraus lässt sich mindestens eine der Abmessung Höhe, Länge und Breite des Fahrzeugs unter Berücksichtigung einer Abtastrate der Zeilenkamera und einer Geschwindigkeit des Fahrzeugs berechnen. Die Geschwindigkeit des Fahrzeugs kann dabei anhand einer Streckung oder Stauchung des Abbilds in der Bildhorizontalen gegenüber den tatsächlichen Seitenverhältnissen am Fahrzeug bestimmt werden, oder mittels einer separaten Geschwindigkeitsmessung ermittelt werden.According to a preferred embodiment, an envelope including the image of the vehicle is created during the evaluation. This can be at least one calculate the height, length and width of the vehicle taking into account a scanning rate of the line scan camera and a speed of the vehicle. The speed of the vehicle can be determined based on an extension or compression of the image in the horizontal image relative to the actual aspect ratios on the vehicle, or be determined by means of a separate speed measurement.

Gemäß einer weiteren bevorzugten Ausführungsform ist vorgesehen, dass eine Anzahl und/oder Positionen von Rädern und/oder Achsen des Fahrzeugs durch Auffinden runder Objekte bei der Auswertung des Abbilds bestimmt werden. Weiterhin bevorzugt wird eine Elliptizität von bei der Auswertung des Abbilds aufgefundenen, runden Objekten bestimmt. Besonders bevorzugt wird unter Berücksichtigung der bestimmten Elliptizität eine Korrektur des Abbilds durch eine Streckung oder Stauchung in der Bildhorizontalen vorgenommen. Gemäß einer weiteren bevorzugten Ausführungsform ist vorgesehen, dass unter Berücksichtigung der Elliptizität und der Abtastrate der Zeilenkamera eine Geschwindigkeit des Fahrzeugs berechnet wird. Diese berechnete Geschwindigkeit ist ein weiteres Merkmal des Fahrzeugs, welches vorteilhaft mittels der Zeilenkamera als alleinigem Messgerät bestimmt werden kann. Alternativ kann die berechnete Geschwindigkeit auch zur Überprüfung einer durch eine separate Geschwindigkeitsmessvorrichtung gemessene Geschwindigkeit verwendet werden.According to a further preferred embodiment it is provided that a number and / or positions of wheels and / or axles of the vehicle are determined by finding round objects during the evaluation of the image. Furthermore, an ellipticity is preferably determined by round objects found during the evaluation of the image. With particular regard to the specific ellipticity, a correction of the image is made by stretching or compression in the image horizontal. According to a further preferred embodiment, it is provided that, taking into account the ellipticity and the sampling rate of the line camera, a speed of the vehicle is calculated. This calculated speed is another feature of the vehicle, which can advantageously be determined by means of the line scan camera as the sole measuring device. Alternatively, the calculated speed may also be used to check a speed measured by a separate speed measuring device.

Gemäß einer weiteren bevorzugten Ausführungsform ist vorgesehen, dass eine Klassifizierung und/oder Identifizierung des Fahrzeugs anhand mindestens eines der folgenden, durch Auswertung des Abbilds erhaltenen Merkmale vorgenommen wird: Abmessungen, Anzahl der Achsen, Kennzeichen, Gefahrgutkennzeichnung, Form einer Frontscheibe und/oder einer Seitenscheibe und charakteristische Formen des Fahrzeugs.According to a further preferred embodiment, it is provided that a classification and / or identification of the vehicle based on at least one of the following, obtained by evaluation of the image features is made: dimensions, number of axles, license plate, dangerous goods marking, shape of a windscreen and / or a side window and characteristic shapes of the vehicle.

Nachfolgend wird die Erfindung anhand eines Ausführungsbeispiels mit Bezug auf die beigefügte Zeichnung näher erläutert. Die Ausführungen beziehen sich gleichermaßen auf die erfindungsgemäße Anordnung und auf das Verfahren. Sie sind lediglich beispielhaft und schränken den allgemeinen Erfindungsgedanken nicht ein.The invention will be explained in more detail with reference to an embodiment with reference to the accompanying drawings. The statements relate equally to the arrangement according to the invention and to the method. They are merely exemplary and do not limit the general idea of the invention.

Es zeigen

  • Figur 1 eine erste schematische Ansicht der erfindungsgemäßen Anordnung;
  • Figur 2 eine zweite schematische Ansicht der Anordnung gemäß Figur 1;
  • Figur 3 ein nach dem erfindungsgemäßen Verfahren erstelltes Abbild eines Fahrzeugs.
Show it
  • FIG. 1 a first schematic view of the arrangement according to the invention;
  • FIG. 2 a second schematic view of the arrangement according to FIG. 1 ;
  • FIG. 3 an image of a vehicle created by the method according to the invention.

In den Figuren 1 und 2 sind zwei verschiedene Ansichten der erfindungsgemäßen Anordnung zur Bestimmung mindestens eines Merkmals eines sich entlang einer Fahrspur 1 bewegenden Fahrzeugs 2 dargestellt, die nachfolgend gemeinsam beschrieben werden. In der Figur 1 ist eine Draufsicht der Fahrspur 1 und des Fahrzeugs 2 dargestellt, während in der Figur 2 eine Seitenansicht des Fahrzeugs 2 dargestellt ist. Die erfindungsgemäße Anordnung umfasst einen lichtempfindlichen Sensor 3 zur Aufnahme eines Abbilds des Fahrzeugs 2 und eine Auswertungsvorrichtung 4 zur Erkennung des mindestens einen Merkmals auf dem Abbild, wobei die Auswertevorrichtung 4 auch entfernt von dem Sensor 3 angeordnet sein kann. Der lichtempfindliche Sensor 3 ist erfindungsgemäß als Zeilenkamera 3 ausgeführt, wobei die Zeilenkamera 3 seitlich neben der Fahrspur 1 angeordnet ist.In the Figures 1 and 2 show two different views of the arrangement according to the invention for determining at least one feature of a moving along a lane 1 vehicle 2, which will be described together below. In the FIG. 1 is a plan view of the lane 1 and the vehicle 2, while in the FIG. 2 a side view of the vehicle 2 is shown. The arrangement according to the invention comprises a photosensitive sensor 3 for recording an image of the vehicle 2 and an evaluation device 4 for detecting the at least one feature on the image, wherein the evaluation device 4 can also be arranged remotely from the sensor 3. The photosensitive sensor 3 is designed according to the invention as a line camera 3, wherein the line camera 3 is arranged laterally next to the traffic lane 1.

Die erfindungsgemäße Anordnung ist besonders vorteilhaft für ein Verfahren zur Achsklassifikation bzw. Achszählung von Fahrzeugen 2 im fließenden Verkehr geeignet. Das Verfahren dient generell zur Klassifikation von Fahrzeugen 2, insbesondere zur Erkennung von markanten Merkmalen, wie z.B. den Fahrzeughersteller oder Gefahrgutkennzeichen in Bewegung. Zusätzlich wird mit dem gleichen Sensor 3 ein Bild des Kennzeichens aufgenommen, welches zur automatischen Kennzeichenleseverarbeitung geeignet ist. Des Weiteren wird ein Abbild des Fahrzeugs 2 als Übersichtsbild der Gesamtszene erstellt. Durch die erfindungsgemäße Anordnung und das erfindungsgemäße Verfahren können so vier Aufgaben aus dem Bereich der Verkehrsüberwachung mittels der einen Zeilenkamera 3 seitlich der Fahrspur 1 durchgeführt werden. Die Zeilenkamera 3 wird seitlich so neben der Fahrspur 1 angeordnet, dass von dem vorbeifahrenden Fahrzeug 2 die Fahrzeugfront 5 und die Fahrzeugflanke 6 zyklisch aufgenommen werden. Für den Fall, dass das natürliche Umgebungslicht, insbesondere bei Nacht, nicht ausreichend sein sollte, wird vorzugsweise eine Beleuchtung 7 auf das Fahrzeug 2 gerichtet. Der Winkel zwischen der Zeilenkamera 3 und dem Fahrzeug 2 ist dabei generell frei wählbar und kann, je nach Anwendung, angepasst werden. Anzustreben ist generell ein Winkel möglichst nahe an 90° auf die zu überwachende Fläche. Da im dargestellten Ausführungsbeispiel sowohl die Fahrzeugfront 5, als auch die Fahrzeugflanke 6 aufgenommen werden sollen, bietet sich ein Winkel von ca. 60° auf die Front 5 an, da diese mehr Detailinformationen enthält, als die Flanke 6, von der das Licht dementsprechend unter einem spitzeren Winkel von 30° in die Zeilenkamera 3 einfällt. In der Figur 2 ist erkennbar, dass das von der Beleuchtung 7 ausgehende, schematisch durch die mit L bezeichneten Pfeile dargestellte Licht, das von dem Fahrzeug 2 zurückgeworfen wird, bevorzugt unter einem Winkel α von mehr als 7° in die Zeilenkamera 3 fällt. Bei einem Winkelabstand von mehr als 7° zwischen der Beleuchtung 7 und der Kamera 3 wird vorteilhaft eine Überbelichtung durch den sogenannten Retroreflektiveffekt von Kennzeichenschildern vermieden.The arrangement according to the invention is particularly advantageously suitable for a method for axle classification or axle counting of vehicles 2 in flowing traffic. The method is generally used for the classification of vehicles 2, in particular for the recognition of distinctive features, such as the vehicle manufacturer or dangerous goods indicator in motion. In addition, with the same sensor 3, an image of the mark suitable for automatic mark reading processing is taken. Furthermore, an image of the vehicle 2 is created as an overview image of the entire scene. As a result of the arrangement according to the invention and the method according to the invention, four tasks from the field of traffic monitoring can be carried out by means of the one Line camera 3 side of the lane 1 are performed. The line camera 3 is laterally arranged next to the traffic lane 1 so that the vehicle front 5 and the vehicle flank 6 are received cyclically by the passing vehicle 2. In the event that the natural ambient light, especially at night, should not be sufficient, an illumination 7 is preferably directed to the vehicle 2. The angle between the line camera 3 and the vehicle 2 is generally freely selectable and can, depending on the application, be adjusted. The aim is generally an angle as close as possible to 90 ° to the surface to be monitored. Since in the illustrated embodiment, both the vehicle front 5, and the vehicle edge 6 are to be recorded, an angle of about 60 ° on the front 5 offers, since this contains more detail information, as the edge 6, from which the light accordingly a more acute angle of 30 ° in the line camera 3 occurs. In the FIG. 2 It can be seen that the light emanating from the illumination 7, represented schematically by the arrows denoted by L, which is reflected by the vehicle 2, preferably falls into the line scan camera 3 at an angle α of more than 7 °. At an angular distance of more than 7 ° between the illumination 7 and the camera 3, an overexposure by the so-called retroreflective effect of license plates is advantageously avoided.

Mit dem erfindungsgemäßen Verfahren wird ein Abbild des Fahrzeugs 2 erzeugt und dadurch eine einfache Bildverarbeitung desselben ermöglicht, da keine Teilbilder zusammengefügt oder überlagert werden müssen. Es treten vorteilhaft in der Bildhorizontalen keine Verzerrungen auf der Flanke 6 des Fahrzeugs 2 auf, da der Abstand zwischen Kamera 3 und Fahrzeug 2 im Wesentlichen unverändert bleibt, abgesehen von Abweichungen durch Lenkbewegungen des Fahrzeugs 2 oder Unebenheiten der Flanke 6 des Fahrzeugs 2. Es entstehen weiterhin vorteilhaft keine stürzenden Linien in dem Abbild auf der Flanke 6 des Fahrzeugs 2. Das erfindungsgemäße Verfahren eignet sich darüber hinaus vorteilhaft zur Bestimmung einer Geschwindigkeit des Fahrzeugs 2. Je nachdem, wie elliptisch die hier nur schematisch angedeuteten Räder 8 in dem Abbild dargestellt werden, kann daraus auf die Geschwindigkeit des Fahrzeugs 2 rückgerechnet werden.With the method according to the invention, an image of the vehicle 2 is generated, thereby enabling a simple image processing of the same, since no partial images must be joined together or superimposed. There are advantageously no distortions in the image horizontal on the flank 6 of the vehicle 2, since the distance between camera 3 and vehicle 2 remains substantially unchanged, apart from deviations by steering movements of the vehicle 2 or unevenness of the flank 6 of the vehicle 2. It arise Furthermore, advantageously no falling lines in the image on the flank 6 of the vehicle 2. The method according to the invention is also advantageously suitable for determining a speed of the vehicle 2. Depending on how elliptically the wheels 8 are indicated schematically in the image, can be recalculated to the speed of the vehicle 2 from this.

Alle hier genannten Merkmale können mit der erfindungsgemäßen Anordnung mit nur einem Sensor 3 erkannt werden. In den Figuren 1 und 2 ist der Sensor 3 geringfügig oberhalb des Fahrbahnniveaus platziert. Denkbar ist auch die Montage des Sensors 3 an einer Brücke (nicht dargestellt), jedoch in jedem Fall seitlich versetzt zu dem zu überwachenden Bereich, also der Fahrspur 1.All features mentioned here can be detected with the inventive arrangement with only one sensor 3. In the Figures 1 and 2 is the sensor 3 placed slightly above the roadway level. Also conceivable is the mounting of the sensor 3 on a bridge (not shown), but in each case offset laterally to the area to be monitored, ie the lane. 1

Ein laserbasierter Geschwindigkeitssensor 9, wie z.B. ein LIDAR-Sensor, wie er in Geschwindigkeitsmessanlagen eingesetzt wird, kann vorzugsweise als zusätzlicher Detektor für Fahrzeuge 2 vorgesehen sein. Der Geschwindigkeitssensor 9 und die Zeilen kamera 3 befinden sich an der gleichen Position seitlich neben der Straße 1. Zum Detektionszeitpunkt liefert der Geschwindigkeitssensor 9 vorzugsweise die folgenden Daten des Fahrzeugs: Position relativ zur Sensorik, Entfernung und Geschwindigkeit des Fahrzeugs 2.A laser-based speed sensor 9, e.g. a LIDAR sensor, as used in speed measuring systems, may preferably be provided as an additional detector for vehicles 2. The speed sensor 9 and the line camera 3 are located at the same position laterally beside the road 1. At the detection time, the speed sensor 9 preferably provides the following data of the vehicle: position relative to the sensor, distance and speed of the vehicle. 2

In der Figur 3 ist ein nach dem erfindungsgemäßen Verfahren erstelltes Abbild eines Fahrzeugs 2 dargestellt. Im Gegensatz zu herkömmlichen Kameras, sogenannten Still Cameras oder Area Scan Cameras, deren beide Koordinatenachsen im Abbild räumliche Positionen verzerrt durch die Perspektive darstellen, wird bei Verwendung einer Zeilenkamera 3 nach der erfindungsgemäßen Anordnung, die Bildhorizontale durch eine zeitliche Abfolge der nacheinander entstandenen Linienaufnahmen entlang der mit t bezeichneten Zeitachse ersetzt und ist daher frei von perspektivischer Verzerrung. Der Abstand von einem Pixel zum nächsten Pixel in Richtung der t-Achse entspricht der Abtastrate oder Zeilenfrequenz der Zeilenkamera 3 und ist vorzugsweise auf die durch den Geschwindigkeitssensor 9 ermittelte Geschwindigkeit des Fahrzeugs 2 angepasst, damit keine gestauchten bzw. gedehnten Bilder in Richtung der t-Achse entstehen. Die Bildvertikale in Richtung der mit Y bezeichneten Achse entspricht der Ausrichtung des Zeilensensors 3.In the FIG. 3 an image of a vehicle 2 produced by the method according to the invention is shown. In contrast to conventional cameras, so-called still cameras or area scan cameras whose two coordinate axes in the image distort spatial positions distorted by the perspective, when using a line camera 3 according to the inventive arrangement, the image horizontals by a temporal sequence of successive line images along the replaced with t time axis and is therefore free from perspective distortion. The distance from one pixel to the next pixel in the direction of the t-axis corresponds to the scanning rate or line frequency of the line camera 3 and is preferably adapted to the speed of the vehicle 2 determined by the speed sensor 9, so that no compressed or stretched images in the direction of the t-axis. Axis arise. The image vertical in the direction of the Y axis corresponds to the orientation of the line sensor. 3

Die Zeilenkamera 3 befindet sich an einem festen Ort und wird nicht bewegt. Ortsfeste Objekte werden demnach durch horizontale Linien mit nur geringer Dynamik, evtl. durch Pixelrauschen oder Schatten dargestellt. Figur 3 zeigt das Abbild eines LKW 2 in Bewegung. Die Silhouette oder Hüllkurve H des Fahrzeugs 2 lässt sich vorteilhaft durch einfache Kantenfilter bestimmen. Selbst durch das Bild laufende Schatten können leicht identifiziert werden und führen nicht zu Fehleinschätzungen, wie dies bei Aufnahmen mit Still Cameras regelmäßig der Fall ist, da kein zeitlicher Bezug hergestellt werden kann. Die geometrischen Informationen, wie Länge, und/oder Höhe lassen sich mit Hilfe dieses Kantenbildes vorteilhaft bestimmen. Wenn die Positionen der Fahrzeugachsen bereits bekannt sind, kann vorteilhaft ausgeschlossen werden, dass die Front des Fahrzeugs 2 mitgemessen wird. Die Messung der Fahrzeuglänge beginnt beispielsweise mit der ersten erkannten Achse. Je nach identifizierten Fahrzeugtyp, wie LKW oder PKW, kann die Länge um einen gemittelten Faktor korrigiert werden.The line camera 3 is at a fixed location and is not moved. Fixed objects are therefore represented by horizontal lines with only little dynamics, possibly by pixel noise or shadows. FIG. 3 shows the image of a truck 2 in motion. The silhouette or envelope H of the vehicle 2 can be advantageously determined by simple edge filters. Even shadows passing through the image can be easily identified and will not lead to misjudgments, as is usually the case with Still Cameras, as no temporal reference can be made. The geometric ones Information such as length, and / or height can be advantageously determined with the help of this edge image. If the positions of the vehicle axles are already known, it can advantageously be ruled out that the front of the vehicle 2 is also measured. The measurement of the vehicle length starts, for example, with the first detected axis. Depending on the type of vehicle identified, such as truck or car, the length can be corrected by an averaged factor.

Andere Sensoren, die zur Überwachung im Verkehr eingesetzt werden, wie LIDAR Scanner, arbeiten üblicherweise mit einer Abtastrate von 100 bis 200 Hz und liefern somit keine so detaillierten Daten, beispielsweise wird ein Rad nur mit etwa drei Pixeln Breite dargestellt, was keine so umfangreiche Auswertung erlaubt, wie das mit einer Zeilenkamera erstellte Abbild. Zeilenkameras, wie sie gemäß dieser Erfindung benutzt werden, können mit Zeilenfrequenzen bis zu 80 kHz arbeiten und bieten daher vorteilhaft weitergehende Möglichkeiten, die Daten zunutzen. Diese Erfindung befähigt dazu, mit der Zeilenkamera 3 als einzigen Sensor und mit nur einem Datenaufnahmezyklus alle Daten zu erzeugen, die z.B. zur Mauterhebung nötig sind. Dies sind im Einzelnen beispielsweise: Aufnahme eines Kennzeichenbildes, Aufnahme eines Übersichtsbildes, Klassifikation des Fahrzeugs auf Grund seiner Achsen und der Fahrzeuggeometrie. Zudem ist es möglich, Gefahrgutkennzeichen zu erkennen und den Fahrzeughersteller zu erkennen. Dadurch, dass alle Daten in einem Schritt erzeugt werden, entstehen konsistente Datensätze, die integer sind. Eine Falschzuordnung der Daten von mehreren Sensoren kann so vorteilhaft vermieden werden.Other sensors used for traffic surveillance, such as LIDAR scanners, typically operate at a sampling rate of 100 to 200 Hz and thus do not provide such detailed data, for example, a wheel is only displayed at about three pixels wide, which is not as extensive an evaluation allowed, like the image created with a line camera. Line scan cameras, as used in accordance with this invention, can operate at line frequencies up to 80 kHz, and thus advantageously offer more advanced capabilities that leverage data. This invention enables with the line scan camera 3 as a single sensor and with only one data acquisition cycle to generate all the data needed, for example, for toll collection. These are in detail, for example: recording a license plate image, recording an overview image, classification of the vehicle based on its axes and the vehicle geometry. It is also possible to recognize dangerous goods identifiers and to recognize the vehicle manufacturer. The fact that all data is generated in one step results in consistent data records that are integer. A misallocation of the data from multiple sensors can be advantageously avoided.

BezugszeichenlisteLIST OF REFERENCE NUMBERS

11
Fahrspurlane
22
Fahrzeugvehicle
33
Zeilenkameraline camera
44
Auswertevorrichtungevaluation
55
Fahrzeugfrontvehicle front
66
Fahrzeugflankevehicle edge
77
Beleuchtungsvorrichtunglighting device
88th
Räderbikes
99
GeschwindigkeitsmessvorrichtungSpeed measuring device
LL
Pfeilearrows
αα
Winkelangle
tt
Zeitachse, BildhorizontaleTimeline, image horizontal
YY
Achse, BildvertikaleAxis, image vertical
HH
Hüllkurveenvelope

Claims (15)

  1. Assembly for determining at least one feature of a vehicle (2) moving along a traffic lane (1), having a light sensitive sensor (3) for recording an image of the vehicle, and having an evaluation device (4) for determining the at least one feature on the image, wherein
    the light sensitive sensor is provided as a line scan camera, wherein the line scan camera is arranged laterally beside the traffic lane,
    characterized in that the line scan camera is aligned aslant to the traffic lane either against a driving direction of the vehicle, or in the driving direction of the vehicle.
  2. Assembly according to claim 1, characterized in that the line scan camera is aligned at an angle of approximately 30° to the traffic lane, against a driving direction of the vehicle.
  3. Assembly according to one of the preceding claims, characterized in that a lighting device (7) is provided, wherein the lighting device is aligned at an angular distance of at least 7° from the alignment of the line scan camera.
  4. Assembly according to one of the preceding claims, characterized in that a speed measuring device for measuring a speed of the vehicle during the recording of the image is provided.
  5. Assembly according to one of the preceding claims, characterized in that a controller for the line scan camera is provided, wherein a sampling rate of the recorded lines is adjustable by the controller.
  6. Assembly according to claim 5, characterized in that the sampling rate is adaptable by the controller to the speed of the vehicle measured by a speed measuring device.
  7. Method for determining at least one feature of a vehicle (2) moving along a traffic lane (1) by means of an assembly according to one of the preceding claims, wherein an image of the vehicle is recorded with the line scan camera (3) and the image for determining the at least one feature of the vehicle is analysed.
  8. Method according to claim 7, characterized in that the recorded lines are attached on after another along a horizontal axis of the image, wherein lines occurring along the horizontal axis of the image are used for detecting the vehicle.
  9. Method according to claim 8, characterized in that an envelope enclosing the image of the vehicle is generated.
  10. Method according to claim 9, characterized in that at least one of the dimensions height, length and width of the vehicle are calculated from the envelope, taking into consideration a sampling rate of the line scan camera and a speed of the vehicle.
  11. Method according to one of claims 7 to 10, characterized in that a number and/or a position of the wheels and/or axles of the vehicle are determined by finding round objects during the analysis of the image.
  12. Method according to one of claims 7 to 11, characterized in that an ellipticity of round objects found during the analysis of the image is determined.
  13. Method according to claim 12, characterized in that a correction of the image is carried out by means of stretching or compressing along the horizontal axis of the image under consideration of the ellipticity.
  14. Method according to claim 13, characterized in that a speed of the vehicle is calculated under consideration of the ellipticity and the sampling rate of the line scan camera.
  15. Method according to one of claims 8 to 14, characterized in that a classification of the vehicle is carried out by means of at least one of the following features obtained by the analysis of the image:
    dimensions, number of axles, vehicle registration, dangerous goods sign, shape of a front windscreen and/or of a side window and characteristic shapes of the vehicle.
EP14173551.4A 2014-06-23 2014-06-23 Determination of at least one vehicle feature Active EP2960883B1 (en)

Priority Applications (16)

Application Number Priority Date Filing Date Title
NO14173551A NO2960883T3 (en) 2014-06-23 2014-06-23
EP14173551.4A EP2960883B1 (en) 2014-06-23 2014-06-23 Determination of at least one vehicle feature
PL14173551T PL2960883T3 (en) 2014-06-23 2014-06-23 Determination of at least one vehicle feature
DK14173551.4T DK2960883T3 (en) 2014-06-23 2014-06-23 Determination of at least one vehicle characteristic
HUE14173551A HUE034573T2 (en) 2014-06-23 2014-06-23 Determination of at least one vehicle feature
LTEP14173551.4T LT2960883T (en) 2014-06-23 2014-06-23 Determination of at least one vehicle feature
SI201430448T SI2960883T1 (en) 2014-06-23 2014-06-23 Determination of at least one vehicle feature
PT141735514T PT2960883T (en) 2014-06-23 2014-06-23 Determination of at least one vehicle feature
ES14173551.4T ES2645504T3 (en) 2014-06-23 2014-06-23 Determination of at least one characteristic of a vehicle
CN201580033839.8A CN106663372A (en) 2014-06-23 2015-06-23 Determination of at least one feature of vehicle
AU2015279273A AU2015279273B2 (en) 2014-06-23 2015-06-23 Determination of at least one feature of a vehicle
CA2953411A CA2953411A1 (en) 2014-06-23 2015-06-23 Determination of at least one feature of a vehicle
RU2017101816A RU2017101816A (en) 2014-06-23 2015-06-23 DETERMINATION AT LEAST ONE SIGN OF A VEHICLE
US15/321,370 US10515544B2 (en) 2014-06-23 2015-06-23 Determination of at least one feature of a vehicle
PCT/EP2015/064147 WO2015197641A1 (en) 2014-06-23 2015-06-23 Determination of at least one feature of a vehicle
CY20171101095T CY1119472T1 (en) 2014-06-23 2017-10-20 DEFINITION OF ATTACHMENT TO A VEHICLE IDENTIFICATION

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP14173551.4A EP2960883B1 (en) 2014-06-23 2014-06-23 Determination of at least one vehicle feature

Publications (2)

Publication Number Publication Date
EP2960883A1 EP2960883A1 (en) 2015-12-30
EP2960883B1 true EP2960883B1 (en) 2017-08-09

Family

ID=51167584

Family Applications (1)

Application Number Title Priority Date Filing Date
EP14173551.4A Active EP2960883B1 (en) 2014-06-23 2014-06-23 Determination of at least one vehicle feature

Country Status (16)

Country Link
US (1) US10515544B2 (en)
EP (1) EP2960883B1 (en)
CN (1) CN106663372A (en)
AU (1) AU2015279273B2 (en)
CA (1) CA2953411A1 (en)
CY (1) CY1119472T1 (en)
DK (1) DK2960883T3 (en)
ES (1) ES2645504T3 (en)
HU (1) HUE034573T2 (en)
LT (1) LT2960883T (en)
NO (1) NO2960883T3 (en)
PL (1) PL2960883T3 (en)
PT (1) PT2960883T (en)
RU (1) RU2017101816A (en)
SI (1) SI2960883T1 (en)
WO (1) WO2015197641A1 (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102014210759A1 (en) * 2014-06-05 2015-12-17 Siemens Aktiengesellschaft Location system for determining the position of a vehicle in a charging station
DE102018200991A1 (en) * 2018-01-23 2019-07-25 Friedrich-Alexander-Universität Erlangen-Nürnberg Method and device for determining the speed of a vehicle
JP7108486B2 (en) * 2018-07-20 2022-07-28 三菱重工機械システム株式会社 vehicle detector
CN111161542B (en) * 2018-11-08 2021-09-28 杭州海康威视数字技术股份有限公司 Vehicle identification method and device
CN116453075B (en) * 2023-06-14 2023-09-08 山东省科学院海洋仪器仪表研究所 Truck wheel axle identification method and system based on image processing

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4908705A (en) * 1988-01-21 1990-03-13 Fairchild Weston Systems, Inc. Steerable wide-angle imaging system
DE60037360T2 (en) 1999-05-28 2008-12-04 Nippon Telegraph And Telephone Corp. Method and device for measuring the speed of vehicles with an image processing system
JP3828349B2 (en) * 2000-09-27 2006-10-04 株式会社日立製作所 MOBILE BODY DETECTION MEASUREMENT METHOD, DEVICE THEREOF, AND RECORDING MEDIUM CONTAINING MOBILE BODY DETECTION MEASUREMENT PROGRAM
DE10148289A1 (en) 2001-09-29 2003-04-24 Vitronic Dr Ing Stein Bildvera Detector for classifying moving vehicles has camera mounted on motorway bridge with computer to determine vehicle speed and path
US6974532B2 (en) * 2003-05-01 2005-12-13 New York University Method for producing adherent coatings of calcium phosphate phases on titanium and titanium alloy substrates by electrochemical deposition
US7415335B2 (en) * 2003-11-21 2008-08-19 Harris Corporation Mobile data collection and processing system and methods
WO2006011141A2 (en) * 2004-07-25 2006-02-02 Israel Aerospace Industries Ltd. Method and system for the acquisition of data and for the display of data
SG139579A1 (en) * 2006-07-20 2008-02-29 Cyclect Electrical Engineering A foreign object detection system
US7869621B1 (en) * 2007-06-07 2011-01-11 Aydin Arpa Method and apparatus for interpreting images in temporal or spatial domains
US8150105B2 (en) * 2008-05-22 2012-04-03 International Electronic Machines Corporation Inspection using three-dimensional profile information
US20100238290A1 (en) * 2008-08-27 2010-09-23 Kachemak Research Development, Inc. Drive over vehicle inspection systems and methods
FR2975186B1 (en) * 2011-05-11 2014-01-10 Morpho METHOD AND DEVICE FOR MEASURING THE SPEED OF A VEHICLE CIRCULATING ON A TRACK.
US8424387B2 (en) * 2011-08-22 2013-04-23 Herzog Services, Inc. Apparatus for detecting defects
DK2565860T3 (en) 2011-08-30 2014-04-22 Kapsch Trafficcom Ag DEVICE AND METHOD OF DETECTING VEHICLE IDENTIFICATION PLATES
DE102011053284B8 (en) * 2011-09-05 2012-09-06 Jenoptik Robot Gmbh Method for verifying the speed of an adequate vehicle by means of a camera
CN102682602B (en) * 2012-05-15 2014-05-07 华南理工大学 Collecting method for road traffic parameters based on video technique
DE102012017441A1 (en) * 2012-09-04 2014-03-06 Christoph Münz wheel detection
DE102012017962A1 (en) * 2012-09-12 2014-03-13 Christoph Münz Symmetrical and asymmetric speed detection and distributed data processing
DE102013212495A1 (en) * 2013-06-27 2014-12-31 Robert Bosch Gmbh Method and device for inspecting a contoured surface, in particular the underbody of a motor vehicle

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
AU2015279273A1 (en) 2017-01-19
DK2960883T3 (en) 2017-11-13
SI2960883T1 (en) 2018-02-28
RU2017101816A3 (en) 2018-07-23
HUE034573T2 (en) 2018-02-28
RU2017101816A (en) 2018-07-23
CY1119472T1 (en) 2018-03-07
EP2960883A1 (en) 2015-12-30
LT2960883T (en) 2017-12-27
AU2015279273B2 (en) 2018-07-05
CA2953411A1 (en) 2015-12-30
WO2015197641A1 (en) 2015-12-30
NO2960883T3 (en) 2018-01-06
PT2960883T (en) 2017-11-14
US10515544B2 (en) 2019-12-24
US20170148315A1 (en) 2017-05-25
CN106663372A (en) 2017-05-10
PL2960883T3 (en) 2018-02-28
ES2645504T3 (en) 2017-12-05

Similar Documents

Publication Publication Date Title
DE102008004632A1 (en) Device and method for measuring a parking space
EP2960883B1 (en) Determination of at least one vehicle feature
DE602005000532T2 (en) Monitoring system for the exterior of a vehicle
DE10164346B4 (en) Road surveillance method for a vehicle and system therefor
EP2234081B1 (en) Method for producing a known fixed spatial relationship between a laser scanner and a digital camera for traffic monitoring
EP3183721A1 (en) Method and axle-counting device for contact-free axle counting of a vehicle and axle-counting system for road traffic
DE112017007467T5 (en) Method for providing interference reduction and a dynamic range of interest in a LIDAR system
EP2338062B1 (en) Method and system for determining a vehicle speed
EP1395852A1 (en) Method for preparing image information
EP2799903B1 (en) Method for detecting speeding offences with restrictive data storage
EP2483882B1 (en) Method for immediate penalisation of a traffic offense
EP0290633B1 (en) Method for detecting changes in the driving range of an unmanned vehicle
EP3435030A1 (en) Method for creating a 3d model of an object
WO2012160056A1 (en) Device and method for measuring the running gear of a motor vehicle
EP3082119B1 (en) Distance measurement of vehicles
CN110779451A (en) Vehicle size measuring device and method based on single camera
DE102011083745B4 (en) A method for monocular motion stereo-based automatic automatic parking free parking, computer program product and device
DE19517026B4 (en) Method for determining the speed of a vehicle with the aid of a vehicle-mounted video camera and device for carrying out the method
AT517658B1 (en) System for recording vehicle side views
EP3049757B1 (en) Chassis measurement under ambient light
DE102017208810A1 (en) Device and method for measuring the tread depth of a tire
WO2012089205A2 (en) Method for safely identifying a vehicle captured by a radiation sensor in a photograph
DE19517029B9 (en) Method for determining the height of a vehicle with the aid of a video camera recording the vehicle and device for carrying out the method
EP2449343A1 (en) Apparatus, sensor and method for visual determination of the range to an object
WO2023247395A1 (en) Method for operating a lidar system with stray light correction, corresponding lidar system, and vehicle

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

17P Request for examination filed

Effective date: 20160630

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

17Q First examination report despatched

Effective date: 20160804

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20170213

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: AT

Ref legal event code: REF

Ref document number: 917610

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170815

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502014004895

Country of ref document: DE

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

Effective date: 20171107

REG Reference to a national code

Ref country code: PT

Ref legal event code: SC4A

Ref document number: 2960883

Country of ref document: PT

Date of ref document: 20171114

Kind code of ref document: T

Free format text: AVAILABILITY OF NATIONAL TRANSLATION

Effective date: 20171030

REG Reference to a national code

Ref country code: NL

Ref legal event code: FP

REG Reference to a national code

Ref country code: SE

Ref legal event code: TRGR

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2645504

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20171205

REG Reference to a national code

Ref country code: EE

Ref legal event code: FG4A

Ref document number: E014658

Country of ref document: EE

Effective date: 20171101

REG Reference to a national code

Ref country code: NO

Ref legal event code: T2

Effective date: 20170809

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170809

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170809

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170809

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20171110

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20171209

REG Reference to a national code

Ref country code: HU

Ref legal event code: AG4A

Ref document number: E034573

Country of ref document: HU

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170809

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502014004895

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170809

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 5

26N No opposition filed

Effective date: 20180511

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170809

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170809

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20190619

Year of fee payment: 6

Ref country code: PL

Payment date: 20190618

Year of fee payment: 6

Ref country code: NO

Payment date: 20190620

Year of fee payment: 6

Ref country code: CZ

Payment date: 20190620

Year of fee payment: 6

Ref country code: PT

Payment date: 20190617

Year of fee payment: 6

Ref country code: LT

Payment date: 20190613

Year of fee payment: 6

Ref country code: IE

Payment date: 20190621

Year of fee payment: 6

Ref country code: NL

Payment date: 20190619

Year of fee payment: 6

Ref country code: DK

Payment date: 20190624

Year of fee payment: 6

Ref country code: CY

Payment date: 20190612

Year of fee payment: 6

Ref country code: LU

Payment date: 20190619

Year of fee payment: 6

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GR

Payment date: 20190620

Year of fee payment: 6

Ref country code: BG

Payment date: 20190619

Year of fee payment: 6

Ref country code: SI

Payment date: 20190618

Year of fee payment: 6

Ref country code: BE

Payment date: 20190619

Year of fee payment: 6

Ref country code: TR

Payment date: 20190617

Year of fee payment: 6

Ref country code: SE

Payment date: 20190624

Year of fee payment: 6

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: SK

Payment date: 20190614

Year of fee payment: 6

Ref country code: CH

Payment date: 20190624

Year of fee payment: 6

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: AT

Payment date: 20190618

Year of fee payment: 5

Ref country code: ES

Payment date: 20190723

Year of fee payment: 6

Ref country code: GB

Payment date: 20190624

Year of fee payment: 6

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: HU

Payment date: 20190612

Year of fee payment: 6

Ref country code: AL

Payment date: 20190621

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170809

REG Reference to a national code

Ref country code: EE

Ref legal event code: MM4A

Ref document number: E014658

Country of ref document: EE

Effective date: 20200630

REG Reference to a national code

Ref country code: DK

Ref legal event code: EBP

Effective date: 20200630

Ref country code: NO

Ref legal event code: MMEP

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200623

Ref country code: CZ

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200623

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: NL

Ref legal event code: MM

Effective date: 20200701

REG Reference to a national code

Ref country code: SK

Ref legal event code: MM4A

Ref document number: E 25715

Country of ref document: SK

Effective date: 20200623

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 917610

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200623

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20200623

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200623

REG Reference to a national code

Ref country code: LT

Ref legal event code: MM4D

Effective date: 20200623

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20200630

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200624

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200623

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200623

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200630

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200630

Ref country code: EE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200630

Ref country code: LT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200623

Ref country code: PT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210125

Ref country code: NO

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200630

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200701

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20201231

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200623

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200630

Ref country code: SE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200624

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200623

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200624

Ref country code: DK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200630

REG Reference to a national code

Ref country code: SI

Ref legal event code: KO00

Effective date: 20210811

REG Reference to a national code

Ref country code: SE

Ref legal event code: EUG

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200623

REG Reference to a national code

Ref country code: ES

Ref legal event code: FD2A

Effective date: 20211105

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200624

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200623

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200623

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20230622

Year of fee payment: 10

Ref country code: DE

Payment date: 20230620

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: LV

Payment date: 20230616

Year of fee payment: 10

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200623