EP2812652A1 - Determining the characteristics of a road surface by means of a 3d camera - Google Patents

Determining the characteristics of a road surface by means of a 3d camera

Info

Publication number
EP2812652A1
EP2812652A1 EP13709741.6A EP13709741A EP2812652A1 EP 2812652 A1 EP2812652 A1 EP 2812652A1 EP 13709741 A EP13709741 A EP 13709741A EP 2812652 A1 EP2812652 A1 EP 2812652A1
Authority
EP
European Patent Office
Prior art keywords
road surface
camera
height
image data
vehicle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP13709741.6A
Other languages
German (de)
French (fr)
Inventor
Stefan Hegemann
Stefan Heinrich
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Conti Temic Microelectronic GmbH
Original Assignee
Conti Temic Microelectronic GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Conti Temic Microelectronic GmbH filed Critical Conti Temic Microelectronic GmbH
Publication of EP2812652A1 publication Critical patent/EP2812652A1/en
Withdrawn legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
    • B60W40/06—Road conditions
    • B60W40/064—Degree of grip
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
    • B60W40/06—Road conditions
    • B60W40/068—Road friction coefficient
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/30—Measuring arrangements characterised by the use of optical techniques for measuring roughness or irregularity of surfaces
    • G01B11/306—Measuring arrangements characterised by the use of optical techniques for measuring roughness or irregularity of surfaces for measuring evenness
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C7/00—Tracing profiles
    • G01C7/02—Tracing profiles of land surfaces
    • G01C7/04—Tracing profiles of land surfaces involving a vehicle which moves along the profile to be traced
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/42—Simultaneous measurement of distance and other co-ordinates
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/46—Indirect determination of position data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/40—Analysis of texture
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W10/00—Conjoint control of vehicle sub-units of different type or different function
    • B60W10/18—Conjoint control of vehicle sub-units of different type or different function including control of braking systems
    • B60W10/184—Conjoint control of vehicle sub-units of different type or different function including control of braking systems with wheel brakes
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W10/00—Conjoint control of vehicle sub-units of different type or different function
    • B60W10/20—Conjoint control of vehicle sub-units of different type or different function including control of steering systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/40—Photo or light sensitive means, e.g. infrared sensors
    • B60W2420/403—Image sensing, e.g. optical camera
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G01S17/894—3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G06T2207/10012—Stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30261—Obstacle

Definitions

  • the invention relates to a method and a device for detecting the condition of a road surface with ⁇ means of a (spatially resolving) 3D camera.
  • DE 102009033219 Al discloses a method and an on ⁇ device for determining a vehicle vorausdorf- a road profile of a lane.
  • a Babyfas ⁇ sungsvorraum or from the vehicle's own motion data a street height profile of the lane ahead the vehicle is determined.
  • the image capture device may in this case be a camera which is fixedly arranged in the front region of the vehicle and comprises two image acquisition units.
  • an active suspension control or adjustment damping can be regulated.
  • Accident prevention is becoming increasingly important in driver assistance systems.
  • emergency braking systems slightest ⁇ th this an important contribution.
  • Their effect depends ever ⁇ but crucially on the coefficient of friction and friction coefficient of the surface with respect to the tires of the vehicle from. In particular, wetness considerably reduces the coefficient of friction compared with that on a dry road.
  • WO 2011/007015 A1 shows a laser-based method for friction coefficient classification in motor vehicles. Signals from a lidar or CV sensor that are placed on the road surface. are judged are evaluated for this purpose and then a friction coefficient is assigned in particular on the basis of the amplitude of the measured road surface. It can be estimated at ⁇ play as whether snow, asphalt or ice form the road surface.
  • the object of the present invention to overcome these drawbacks and to provide a more favorable, more reliable and vo ⁇ out looking recognition of the nature of a roadway surface.
  • the object is achieved by Minim ⁇ least a picture of the surroundings of the vehicle is recorded with a 3D camera. From the image data of the 3D camera, height profiles of the road surface transversely to the direction of travel of the vehicle are determined along a plurality of lines. This Li ⁇ nien also referred to as scan lines. Along each line of thus height profile is "scanned".
  • the condition of the vehicle is detected ground surface from which he ⁇ mediated level courses. As a condition here in particular, the material which the road surface bil ⁇ det (tar, sand, ice, snow), the shape of the surface (flat, rough, bumpy) (as well as local changes in the material or the flatness of the road surface oil track, puddle, pothole, ruts, etc. .) Understood.
  • the condition of the road surface is detected from a comparison of the ascertained height courses with stored height courses of known nature.
  • a Beillesheitsklassifikator can be vortrai ⁇ ned which thereby learns which detected height profiles correspond to which textures of road surfaces.
  • a quantitative evaluation of the level curves determined can be performed, for example, egg ⁇ ne frequency analysis (or width and / or expansion in the direction of travel of individual recesses or projections of the road surface), amplitude determination, gradient or the like. From this quantitative analysis, typical textures can be identified on the basis of characteristic quantities, eg strong noise of the height gradients in a gravel path.
  • water or a layer of ice on the road leads to a smooth road surface. Ice often occurs along with snow along with the roadway.
  • At least one monocular camera of the 3D camera can be evaluated and into the recognition of the condition of the road surface.
  • the evaluation of the 2D image data in this case need not be limited to the area of the road surface, but may in particular include areas next to the roadway.
  • the 2D image data can be evaluated by means of a texture or pattern analysis.
  • the texture and / or pattern analysis may in particular be a classification based on trained textures and / or patterns, e.g. using a neural network. For example, due to the characteristic textures or patterns, a head-stone patch can be well recognized in the 2D image.
  • the 2D image data are evaluated by means of an edge and / or color analysis.
  • edge detection algorithms known per se for edge detection can be used.
  • the position of the edges in particular enables or supports the recognition of textures of the road surface whose edge profile is known.
  • a gray scale or color analysis for example, snow can be detected from white areas on or next to the roadway.
  • ⁇ Licher texture difference may be estimated by a respective local coefficient of friction.
  • the at least one estimated coefficient of friction is used for a precontrol of steering and / or braking interventions (ABS, ESP, ACC, emergency brake assistant, lane keeping, backup and / or emergency steering assistant).
  • ABS steering and / or braking interventions
  • ESP ESP
  • ACC emergency brake assistant
  • lane keeping lane keeping
  • backup and / or emergency steering assistant a precontrol of steering and / or braking interventions
  • the 3D camera is preferably a stereo camera or a photonic mixer camera or PMD sensor (English: Photonic Mixing Device).
  • the invention further comprises a device for detecting the condition of a road surface.
  • a 3D camera, evaluation and detection means are pre ⁇ see.
  • the 3D camera allows the recording of at least ei ⁇ Nes image of the lying in front of the vehicle environment including the road ahead.
  • the evaluation means serve to determine height profiles of the road surface along a plurality of lines transversely to the direction of travel of the vehicle from the image data of the 3D camera.
  • the Erken ⁇ voltage medium used for detecting the condition of the road surface from the determined height gradients.
  • Fig. 1 lines transverse to the direction of travel, along which the height profile of the road surface is determined;
  • FIG. 2 shows an exemplary height profile of the road surface transversely to the direction of travel in the presence of a striking hole
  • Fig. 7 is a 2D camera recording in wet conditions
  • Fig. 8 is a 2D camera recording on dry roads.
  • Fig. 1 the scanning of the height profile (h) along lines (5) transversely (y) to the direction of travel (1) of the vehicle is shown schematically.
  • the road surface is monitored from right to left, left to right or in both transverse directions (y) along the scan lines (5) having any given density.
  • a raised lane boundary such as a curb limits the road surface left (3) and right (4).
  • the substantially flat road surface is lowered locally in the left front of the region ⁇ a pothole (2) or a similar interruption.
  • This lowering is clearly visible in the course of the height (h) of the road surface transversely (y) to the direction of travel (1) as shown in FIG.
  • This height profile (h) has, in addition to two jumps due to the raised roadway boundaries (3, 4) a dotted lowering in the left area, which can be assigned to the pothole (2) of FIG.
  • a roadway is shown, which is formed by a KopfSteinpflaster (6).
  • chassis components could be controlled accordingly and / or measures to improve the driving noise (counter-noise or tire pressure changes) could be taken.
  • FIG. 4 An exemplary height profile (h) of the head plaster (6) in the transverse direction (y) is shown in FIG. 4. More height profiles along other scan lines (5) in Fig. Wür ⁇ look similar to Figure 3.
  • the typical width and height of the variations in altitude (h) allows the assignment of such characteristic height gradients to a head-stone pavement as the nature of the road surface.
  • the recognition can be additionally supported by a texture or pattern analysis of the 2D camera image (see Fig. 3). This analysis also allows the road surface to be recognized as a head stone patch, thus confirming the recognition from the height profiles from the 3D camera data.
  • Fig. 5 is a roadway with two ruts (7, 8) parallel to the direction of travel (1) in the right part of the road surface to see.
  • two Absen ⁇ kungen (dotted) can be seen in the right part of the road surface.
  • Fig. 7 a wet road surface is in a 2D camera image (gray scale image of a monocular camera) to se ⁇ hen.
  • mirroring effects on the wet road surface can be recognized from the 2D image since trees (or in general the environment) appear mirrored on the road surface.
  • mirroring effects lead to a greater perceived distance in the disparity image of the stereo camera.
  • This supposed distance ent ⁇ speaks the actual distance to the trees next to the roadway ermit- from the disparity of the environment can be telt.

Abstract

The invention relates to a method and to a device for detecting the characteristics of a road surface by means of a 3D camera. At least one image of the environment in front of the vehicle is recorded by the 3D camera. From the image data of the 3D camera, height variations (h) in the road surface transverse (y) to the travel direction of the vehicle are determined along a plurality of lines (5). The characteristics (2; 6; 7; 8) of the road surface are identified from the determined height variations (h).

Description

Bestimmung einer Beschaffenheit einer Fahrbahnoberfläche  Determining a condition of a road surface
mittels einer 3D-Kamera  using a 3D camera
Die Erfindung betrifft ein Verfahren und eine Vorrichtung zur Erkennung der Beschaffenheit einer Fahrbahnoberfläche mit¬ tels einer (räumlich auflösenden) 3D-Kamera. The invention relates to a method and a device for detecting the condition of a road surface with ¬ means of a (spatially resolving) 3D camera.
Die DE 102009033219 AI zeigt ein Verfahren und eine Vor¬ richtung zur Ermittlung eines einem Fahrzeug vorausliegen- den Straßenprofils einer Fahrspur. Mittels einer Bilderfas¬ sungsvorrichtung bzw. aus Fahrzeugeigenbewegungsdaten wird ein Straßenhöhenprofil der dem Fahrzeug vorausliegenden Fahrspur ermittelt. Die Bilderfassungsvorrichtung kann hierbei eine Kamera sein, die im Frontbereich des Fahrzeugs fest angeordnet ist und zwei Bildaufnahmeeinheiten umfasst. In Abhängigkeit des ermittelten Straßenhöhenprofils kann eine aktive Fahrwerksregelung oder Verstelldämpfung geregelt werden. Die Unfallvermeidung gewinnt bei Fahrerassistenzsystemen zunehmend an Bedeutung. Insbesondere Notbremssysteme leis¬ ten hierzu einen wichtigen Beitrag. Ihre Wirkung hängt je¬ doch entscheidend vom Reibwert bzw. Reibungskoeffizienten des Untergrundes in Bezug zum Reifen des Fahrzeugs ab. Ins- besondere Nässe erniedrigt den Reibwert gegenüber dem auf einer trockenen Fahrbahn erheblich. DE 102009033219 Al discloses a method and an on ¬ device for determining a vehicle vorausliegen- a road profile of a lane. By means of a Bilderfas ¬ sungsvorrichtung or from the vehicle's own motion data a street height profile of the lane ahead the vehicle is determined. The image capture device may in this case be a camera which is fixedly arranged in the front region of the vehicle and comprises two image acquisition units. Depending on the determined road height profile, an active suspension control or adjustment damping can be regulated. Accident prevention is becoming increasingly important in driver assistance systems. In particular, emergency braking systems slightest ¬ th this an important contribution. Their effect depends ever ¬ but crucially on the coefficient of friction and friction coefficient of the surface with respect to the tires of the vehicle from. In particular, wetness considerably reduces the coefficient of friction compared with that on a dry road.
WO 2011/007015 AI zeigt ein laserbasiertes Verfahren zur Reibwertklassifikation in Kraftfahrzeugen. Signale eines Lidar- bzw. CV-Sensors, die auf die Fahrbahnoberfläche ge- richtet sind, werden hierzu ausgewertet und anschließend wird insbesondere anhand der Amplitude der vermessenen Fahrbahnoberfläche ein Reibwert zugeordnet. Es kann bei¬ spielsweise geschätzt werden, ob Schnee, Asphalt oder Eis die Fahrbahnoberfläche bilden. WO 2011/007015 A1 shows a laser-based method for friction coefficient classification in motor vehicles. Signals from a lidar or CV sensor that are placed on the road surface. are judged are evaluated for this purpose and then a friction coefficient is assigned in particular on the basis of the amplitude of the measured road surface. It can be estimated at ¬ play as whether snow, asphalt or ice form the road surface.
Es zeigt sich, dass Verfahren zur Erkennung der Beschaffenheit einer Fahrbahnoberfläche nach dem Stand der Technik Nachteile mit sich bringen, da entweder die Bilderfassung und -auswertung ausschließlich zur Fahrwerksregelung eingesetzt wird oder Lidarsensoren erforderlich sind, die teuer sind und gezielt auf die Fahrbahn gerichtet werden müssen, so dass die Beschaffenheit der Fahrbahnoberfläche unter Um¬ ständen nicht ausreichend weit im Voraus eigeschätzt wer- den kann. It is found that methods for detecting the condition of a road surface according to the prior art have disadvantages, since either the image acquisition and evaluation is used exclusively for chassis control or Lidarsensoren are required, which are expensive and must be targeted to the roadway so that the condition of the road surface under order ¬ stands not far enough in advance eigeschätzt advertising can to.
Es ist Aufgabe der vorliegenden Erfindung diese Nachteile zu überwinden und eine günstigere, zuverlässigere und vo¬ rausschauende Erkennung der Beschaffenheit einer Fahrbahn- Oberfläche zu ermöglichen. The object of the present invention to overcome these drawbacks and to provide a more favorable, more reliable and vo ¬ out looking recognition of the nature of a roadway surface.
Die Aufgabe wird gelöst, indem mit einer 3D-Kamera mindes¬ tens ein Bild von der Umgebung des Fahrzeugs aufgenommen wird. Aus den Bilddaten der 3D-Kamera werden entlang einer Mehrzahl von Linien Höhenverläufe der Fahrbahnoberfläche quer zur Fahrtrichtung des Fahrzeugs ermittelt. Diese Li¬ nien werden auch als Scanlines bezeichnet. Entlang jeder Linie wird der somit Höhenverlauf „gescannt". Aus den er¬ mittelten Höhenverläufen wird die Beschaffenheit der Fahr- bahnoberfläche erkannt. Als Beschaffenheit wird hierbei insbesondere das Material, das die Fahrbahnoberfläche bil¬ det (Teer, Sand, Eis, Schnee) , die Form der Oberfläche (eben, rau, buckelig) sowie lokale Änderungen des Materials oder der Ebenheit der Fahrbahnoberfläche (Ölspur, Pfütze, Schlagloch, Spurrillen usw.) verstanden. The object is achieved by Minim ¬ least a picture of the surroundings of the vehicle is recorded with a 3D camera. From the image data of the 3D camera, height profiles of the road surface transversely to the direction of travel of the vehicle are determined along a plurality of lines. This Li ¬ nien also referred to as scan lines. Along each line of thus height profile is "scanned". The condition of the vehicle is detected ground surface from which he ¬ mediated level courses. As a condition here in particular, the material which the road surface bil ¬ det (tar, sand, ice, snow), the shape of the surface (flat, rough, bumpy) (as well as local changes in the material or the flatness of the road surface oil track, puddle, pothole, ruts, etc. .) Understood.
Bevorzugt wird die Beschaffenheit der Fahrbahnoberfläche aus einem Vergleich der ermittelten Höhenverläufe mit gespeicherten Höhenverläufen bekannter Beschaffenheit er- kannt . Dazu kann ein Beschaffenheitsklassifikator vortrai¬ niert werden, der dadurch lernt, welche detektierten Höhenverläufe welchen Beschaffenheiten von Fahrbahnoberflächen entsprechen . Alternativ oder kumulativ kann eine quantitative Auswertung der ermittelten Höhenverläufe durchgeführt werden, z.B. ei¬ ne Frequenzanalyse (bzw. Breite und/oder Ausdehnung in Fahrtrichtung einzelner Vertiefungen oder Erhebungen der Fahrbahnoberfläche) , Amplitudenbestimmung, Gradientenbildung oder ähnliches. Aus dieser quantitativen Auswertung können typische Beschaffenheiten aufgrund charakteristischer Größen erkannt werden, z.B. starkes Rauschen der Höhenverläufe bei einem Kiesweg. Wasser oder eine Eisschicht auf der Fahrbahn führen dagegen zu einer glatten Fahrbahnoberfläche. Eis tritt häufig zusammen mit Schnee neben der Fahrbahn auf. Preferably, the condition of the road surface is detected from a comparison of the ascertained height courses with stored height courses of known nature. For this, a Beschaffenheitsklassifikator can be vortrai ¬ ned which thereby learns which detected height profiles correspond to which textures of road surfaces. Alternatively or cumulatively, a quantitative evaluation of the level curves determined can be performed, for example, egg ¬ ne frequency analysis (or width and / or expansion in the direction of travel of individual recesses or projections of the road surface), amplitude determination, gradient or the like. From this quantitative analysis, typical textures can be identified on the basis of characteristic quantities, eg strong noise of the height gradients in a gravel path. On the other hand, water or a layer of ice on the road leads to a smooth road surface. Ice often occurs along with snow along with the roadway.
In einer bevorzugten Ausführungsform können zusätzlich zu den ermittelten Höhenverläufen 2D-Bilddaten mindestens ei- ner monokularen Kamera der 3D-Kamera ausgewertet werden und in die Erkennung der Beschaffenheit der Fahrbahnoberfläche einfließen. Die Auswertung der 2D-Bilddaten muss hierbei nicht auf den Bereich der Fahrbahnoberfläche begrenzt sein, sondern kann insbesondere Bereiche neben der Fahrbahn um- fassen. In a preferred embodiment, in addition to the ascertained height profiles, 2D image data, at least one monocular camera of the 3D camera can be evaluated and into the recognition of the condition of the road surface. The evaluation of the 2D image data in this case need not be limited to the area of the road surface, but may in particular include areas next to the roadway.
Gemäß einer vorteilhaften Weiterbildung der Erfindung können die 2D-Bilddaten mittels einer Textur- bzw. Musteranalyse ausgewertet werden. Die Textur- und/oder Musteranalyse kann insbesondere eine Klassifikation anhand trainierter Texturen und/oder Muster sein, z.B. mittels eines neuronalen Netzwerks. Beispielsweise kann ein KopfSteinpflaster aufgrund der charakteristischen Texturen bzw. Muster gut im 2D-Bild erkannt werden. According to an advantageous development of the invention, the 2D image data can be evaluated by means of a texture or pattern analysis. The texture and / or pattern analysis may in particular be a classification based on trained textures and / or patterns, e.g. using a neural network. For example, due to the characteristic textures or patterns, a head-stone patch can be well recognized in the 2D image.
Bevorzugt werden die 2D-Bilddaten mittels einer Kanten- und/oder Farbanalyse ausgewertet. Zur Kantendetektion können an sich bekannte Algorithmen zur Kantenerkennung verwendet werden. Die Lage der Kanten ermöglicht oder unter- stützt insbesondere die Erkennung von Beschaffenheiten der Fahrbahnoberfläche, deren Kantenprofil bekannt ist. Mittels einer Grauwert- oder Farbanalyse kann beispielsweise Schnee aus weißen Bereichen auf oder neben der Fahrbahn erkannt werden . Preferably, the 2D image data are evaluated by means of an edge and / or color analysis. For edge detection, algorithms known per se for edge detection can be used. The position of the edges in particular enables or supports the recognition of textures of the road surface whose edge profile is known. By means of a gray scale or color analysis, for example, snow can be detected from white areas on or next to the roadway.
Vorteilhaft wird aus der erkannten Beschaffenheit der Fahr¬ bahnoberfläche ein (lokal aufgelöster) Reibwert zwischen vorausliegender Fahrbahnoberfläche und Fahrzeugreifen abgeschätzt . Für vorausliegende Fahrbahnoberflächenbereiche unterschied¬ licher Beschaffenheit kann dadurch jeweils ein lokaler Reibwert abgeschätzt werden. A (locally resolved) coefficient of friction between the road surface lying ahead of the vehicle tire and is advantageously estimated from the detected nature of the driving ¬ web surface. For lying ahead road surface areas ¬ Licher texture difference may be estimated by a respective local coefficient of friction.
In einer bevorzugten Ausführungsform wird der mindestens eine abgeschätzte Reibwert zu einer Vorsteuerung von Lenk- und/oder Bremseingriffen (ABS, ESP, ACC, Notbremsassistent, Spurhalte-, Ausweich- und/oder Notlenkassistent) genutzt. In a preferred embodiment, the at least one estimated coefficient of friction is used for a precontrol of steering and / or braking interventions (ABS, ESP, ACC, emergency brake assistant, lane keeping, backup and / or emergency steering assistant).
Die 3D-Kamera ist bevorzugt eine Stereokamera oder eine Photomischdetektor-Kamera bzw. PMD-Sensor (englisch: Photonic Mixing Device) . The 3D camera is preferably a stereo camera or a photonic mixer camera or PMD sensor (English: Photonic Mixing Device).
Die Erfindung umfasst ferner eine Vorrichtung zur Erkennung der Beschaffenheit einer Fahrbahnoberfläche. Hierzu sind eine 3D-Kamera, Auswertemittel und Erkennungsmittel vorge¬ sehen. Die 3D-Kamera ermöglicht die Aufnahme mindestens ei¬ nes Bildes der vor dem Fahrzeug liegenden Umgebung inklusive der vorausliegenden Fahrbahn. Die Auswertemittel dienen zum Ermitteln von Höhenverläufen der Fahrbahnoberfläche entlang einer Mehrzahl von Linien quer zur Fahrtrichtung des Fahrzeugs aus den Bilddaten der 3D-Kamera. Die Erken¬ nungsmittel dienen zum Erkennen der Beschaffenheit der Fahrbahnoberfläche aus den ermittelten Höhenverläufen. The invention further comprises a device for detecting the condition of a road surface. For this purpose, a 3D camera, evaluation and detection means are pre ¬ see. The 3D camera allows the recording of at least ei ¬ Nes image of the lying in front of the vehicle environment including the road ahead. The evaluation means serve to determine height profiles of the road surface along a plurality of lines transversely to the direction of travel of the vehicle from the image data of the 3D camera. The Erken ¬ voltage medium used for detecting the condition of the road surface from the determined height gradients.
Im Folgenden wird die Erfindung anhand von Figuren und Ausführungsbeispielen näher erläutert. Es zeigen: Fig. 1 Linien quer zur Fahrtrichtung, entlang derer der Höhenverlauf der Fahrbahnoberfläche ermittelt wird; In the following the invention will be explained in more detail with reference to figures and exemplary embodiments. Show it: Fig. 1 lines transverse to the direction of travel, along which the height profile of the road surface is determined;
Fig. 2 einen exemplarischen Höhenverlauf der Fahrbahnoberfläche quer zur Fahrtrichtung bei Vorliegen eines Schlag- lochs; FIG. 2 shows an exemplary height profile of the road surface transversely to the direction of travel in the presence of a striking hole; FIG.
Fig. 3 die Ermittlung von Höhenverläufen quer zur Fahrtrichtung bei einem KopfSteinpflaster ;  3 shows the determination of height profiles transverse to the direction of travel in a head stone plaster;
Fig. 4 einen Höhenverlauf bei KopfSteinpflaster ; 4 shows a height profile in the case of KopfSteinpflaster;
Fig. 5 eine Fahrbahnoberfläche mit Spurrillen; 5 shows a road surface with ruts.
Fig. 6 einen exemplarischen Höhenverlauf quer zur Fahrtrichtung bei Spurrillen; 6 shows an exemplary height profile transverse to the direction of travel in track grooves;
Fig. 7 eine 2D-Kameraaufnähme bei nasser Fahrbahn und  Fig. 7 is a 2D camera recording in wet conditions and
Fig. 8 eine 2D-Kameraaufnähme bei trockener Fahrbahn. Fig. 8 is a 2D camera recording on dry roads.
In Fig. 1 ist das Scannen des Höhenverlaufs (h) entlang von Linien (5) quer (y) zur Fahrtrichtung (1) des Fahrzeugs schematisch dargestellt. Die Fahrbahnoberfläche wird von rechts nach links, links nach rechts oder in beiden Querrichtungen (y) entlang der Scan-Linien (5) , die eine beliebige vorgegebene Dichte aufweisen, überwacht. Eine erhabene Fahrbahnbegrenzung wie beispielsweise ein Bordstein begrenzt die Fahrbahnoberfläche links (3) und rechts (4) . Die im Wesentlichen ebene Fahrbahnoberfläche ist im linken vor¬ deren Bereich durch ein Schlagloch (2) oder eine vergleichbare Unterbrechung lokal abgesenkt. In Fig. 1, the scanning of the height profile (h) along lines (5) transversely (y) to the direction of travel (1) of the vehicle is shown schematically. The road surface is monitored from right to left, left to right or in both transverse directions (y) along the scan lines (5) having any given density. A raised lane boundary such as a curb limits the road surface left (3) and right (4). The substantially flat road surface is lowered locally in the left front of the region ¬ a pothole (2) or a similar interruption.
Diese Absenkung ist im Höhenverlauf (h) der Fahrbahnoberfläche quer (y) zur Fahrtrichtung (1) wie in Fig. 2 dargestellt deutlich zu erkennen. Hier ist exemplarisch der Höhenverlauf (h) entlang einer Linie (5) in Querrichtung (y) aufgetragen. Dieser Höhenverlauf (h) weist neben zwei Sprüngen aufgrund der erhabenen Fahrbahnbegrenzungen (3, 4) eine gepunktet dargestellte Absenkung im linken Bereich auf, die dem Schlagloch (2) aus Fig. 1 zugeordnet werden kann . This lowering is clearly visible in the course of the height (h) of the road surface transversely (y) to the direction of travel (1) as shown in FIG. Here is an example of the height profile (h) along a line (5) in the transverse direction (y) applied. This height profile (h) has, in addition to two jumps due to the raised roadway boundaries (3, 4) a dotted lowering in the left area, which can be assigned to the pothole (2) of FIG.
In Fig. 3 ist eine Fahrbahn dargestellt, die durch ein KopfSteinpflaster (6) gebildet ist. Bei der Erkennung eines vorausliegenden KopfSteinpflasters könnten Fahrwerkskompo- nenten entsprechend angesteuert werden und/oder Maßnahmen zur Fahrgeräuschverbesserung (Gegenschall bzw. Reifendruckveränderungen) ergriffen werden. In Fig. 3, a roadway is shown, which is formed by a KopfSteinpflaster (6). When detecting a head-mounted plaster in front, chassis components could be controlled accordingly and / or measures to improve the driving noise (counter-noise or tire pressure changes) could be taken.
Ein exemplarischer Höhenverlauf (h) des KopfSteinpflasters (6) in Querrichtung (y) ist in Fig. 4 dargestellt. Weitere Höhenverläufe entlang weiterer Scanlines (5) in Fig. 3 wür¬ den ähnlich aussehen. Die typische Breite und Höhe der Schwankungen im Höhenverlauf (h) erlaubt die Zuordnung derartiger charakteristischer Höhenverläufe zu einem Kopf- Steinpflaster als Beschaffenheit der Fahrbahnoberfläche. Unterstützt werden kann die Erkennung zusätzlich durch eine Textur- bzw. Musteranalyse des 2D-Kamerabilds (vgl. Fig. 3) . Auch durch diese Analyse kann die Fahrbahnoberfläche als KopfSteinpflaster erkannt und somit die Erkennung aus den Höhenverläufen aus den 3D-Kameradaten bestätigt werden. An exemplary height profile (h) of the head plaster (6) in the transverse direction (y) is shown in FIG. 4. More height profiles along other scan lines (5) in Fig. Wür ¬ look similar to Figure 3. The typical width and height of the variations in altitude (h) allows the assignment of such characteristic height gradients to a head-stone pavement as the nature of the road surface. The recognition can be additionally supported by a texture or pattern analysis of the 2D camera image (see Fig. 3). This analysis also allows the road surface to be recognized as a head stone patch, thus confirming the recognition from the height profiles from the 3D camera data.
In Fig. 5 ist eine Fahrbahn mit zwei Spurrillen (7, 8) parallel zur Fahrtrichtung (1) im rechten Teil der Fahrbahnoberfläche zu sehen. Entsprechend sind im Höhenverlauf (h) in Fig. 6 zwei Absen¬ kungen (gepunktet) im rechten Teil der Fahrbahnoberfläche zu erkennen. In Fig. 7 ist eine nasse Fahrbahnoberfläche in einem 2D- Kamerabild (Grauwertbild einer monokularen Kamera) zu se¬ hen. Spiegelungseffekte auf der nassen Fahrbahnoberfläche können zum einen aus dem 2D-Bild erkannt werden, da Bäume (oder allgemein die Umgebung) auf der Fahrbahnoberfläche gespiegelt erscheinen. Zum anderen führen Spiegelungseffekte zu einem größeren vermeintlichen Abstand im Disparitätsbild der Stereokamera. Dieser vermeintliche Abstand ent¬ spricht dem tatsächlichen Abstand zu den Bäumen, die aus dem Disparitätsbild der Umgebung neben der Fahrbahn ermit- telt werden können. Somit kann erkannt werden, dass eine nasse Fahrbahnoberfläche vorausliegt und der Reibwert kann auf μ=0,4 geschätzt werden. In Fig. 5 is a roadway with two ruts (7, 8) parallel to the direction of travel (1) in the right part of the road surface to see. Correspondingly, in the height curve (h) in FIG. 6, two Absen ¬ kungen (dotted) can be seen in the right part of the road surface. In Fig. 7 a wet road surface is in a 2D camera image (gray scale image of a monocular camera) to se ¬ hen. On the one hand, mirroring effects on the wet road surface can be recognized from the 2D image since trees (or in general the environment) appear mirrored on the road surface. On the other hand, mirroring effects lead to a greater perceived distance in the disparity image of the stereo camera. This supposed distance ent ¬ speaks the actual distance to the trees next to the roadway ermit- from the disparity of the environment can be telt. Thus, it can be seen that a wet road surface is ahead and the coefficient of friction can be estimated to μ = 0.4.
In Fig. 8 ist ein 2D-Kamerabild einer trockenen Fahrbahn- Oberfläche zu sehen. Diese kann unter anderem daraus er¬ kannt werden, dass keine Spiegelungseffekte auftreten und der Reibwert kann auf μ=0,9 geschätzt werden. FIG. 8 shows a 2D camera image of a dry road surface. This may be it ¬ known among other things from the fact that no mirror effects occur and the friction coefficient can be estimated at μ = 0.9.

Claims

Patentansprüche claims
1. Verfahren zur Erkennung der Beschaffenheit einer Fahrbahnoberfläche mittels einer 3D-Kamera, wobei A method for detecting the condition of a road surface by means of a 3D camera, wherein
mit der 3D-Kamera mindestens ein Bild von der vor dem Fahrzeug liegenden Umgebung aufgenommen wird, dadurch gekennzeichnet, dass  is recorded with the 3D camera at least one image of the lying in front of the vehicle environment, characterized in that
aus den Bilddaten der 3D-Kamera entlang einer Mehrzahl von Linien (5) Höhenverläufe (h) der Fahrbahnoberflä¬ che quer (y) zur Fahrtrichtung (1) des Fahrzeugs ermittelt wird und aus den ermittelten Höhenverläufen (h) die Beschaffenheit (2; 6; 7; 8) der Fahrbahnoberfläche erkannt wird. From the image data of the 3D camera along a plurality of lines (5) height curves (h) of the Fahrbahnoberflä ¬ che transverse (y) to the direction of travel (1) of the vehicle is determined and from the determined height gradients (h) the nature (2; ; 7; 8) of the road surface is detected.
2. Verfahren nach Anspruch 1, wobei die Beschaffenheit (2; 6; 7; 8) der Fahrbahnoberfläche aus einem Ver¬ gleich der ermittelten Höhenverläufe (h) mit gespei¬ cherten Höhenverläufen bekannter Beschaffenheit erkannt wird. 2. The method of claim 1, wherein the composition (2; 6; 7; 8) of the road surface is detected from a ¬ Ver equal to the height profiles detected (h) with vomit ¬ cherten height curves of known nature.
3. Verfahren nach Anspruch 1 oder 2, wobei zusätzlich 2D- Bilddaten mindestens einer monokularen Kamera der 3D- Kamera ausgewertet werden und in die Erkennung der Be¬ schaffenheit der Fahrbahnoberfläche einfließen. 3. The method of claim 1 or 2, wherein additionally 2D image data of at least one monocular camera of the 3D camera are evaluated and incorporated into the detection of Be ¬ creation of the road surface.
4. Verfahren nach Anspruch 3, wobei die 2D-Bilddaten mittels einer Textur- oder Musteranalyse ausgewertet wer¬ den . 4. The method of claim 3, wherein the 2D image data evaluated by means of a texture or pattern analysis ¬ the.
5. Verfahren nach Anspruch 3 oder 4, wobei die 2D- Bilddaten mittels einer Kanten und/oder Farbanalyse ausgewertet werden. 5. The method of claim 3 or 4, wherein the 2D image data are evaluated by means of an edge and / or color analysis.
6. Verfahren einem der vorhergehenden Ansprüche, wobei aus der erkannten Beschaffenheit zumindest eines Be¬ reichs der vorausliegenden Fahrbahnoberfläche ein Reibwert abgeschätzt wird. 6. The method any of the preceding claims, wherein a friction value is estimated from the detected characteristics of at least one Be ¬ realm of the forward roadway surface.
7. Verfahren nach Anspruch 6, wobei der mindestens eine abgeschätzte Reibwert zu einer Vorsteuerung von Lenk- und/oder Bremseingriffen genutzt wird. 7. The method of claim 6, wherein the at least one estimated coefficient of friction is used for a feedforward control of steering and / or braking interventions.
8. Verfahren nach einem der vorhergehenden Ansprüche, wobei die 3D-Kamera eine Stereokamera ist. 8. The method according to any one of the preceding claims, wherein the 3D camera is a stereo camera.
9. Vorrichtung zur Erkennung der Beschaffenheit einer Fahrbahnoberfläche umfassend eine 3D-Kamera, dadurch gekennzeichnet, dass Auswertemittel zum Ermitteln von Höhenverläufen (h) der Fahrbahnoberfläche entlang einer Mehrzahl von Linien (5) quer (y) zur Fahrtrichtung (1) des Fahrzeugs aus den Bilddaten der 3D-Kamera und Erkennungsmittel zum Erkennen der Beschaffenheit (2;9. A device for detecting the condition of a road surface comprising a 3D camera, characterized in that evaluation means for determining height gradients (h) of the road surface along a plurality of lines (5) transversely (y) to the direction of travel (1) of the vehicle from the Image data of the 3D camera and recognition means for recognizing the condition (2;
6; 7; 8) der Fahrbahnoberfläche aus den ermittelten Höhenverläufen (h) . 6; 7; 8) of the road surface from the determined height gradients (h).
EP13709741.6A 2012-02-10 2013-01-28 Determining the characteristics of a road surface by means of a 3d camera Withdrawn EP2812652A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102012101085A DE102012101085A1 (en) 2012-02-10 2012-02-10 Determining a condition of a road surface by means of a 3D camera
PCT/DE2013/100028 WO2013117186A1 (en) 2012-02-10 2013-01-28 Determining the characteristics of a road surface by means of a 3d camera

Publications (1)

Publication Number Publication Date
EP2812652A1 true EP2812652A1 (en) 2014-12-17

Family

ID=47891329

Family Applications (1)

Application Number Title Priority Date Filing Date
EP13709741.6A Withdrawn EP2812652A1 (en) 2012-02-10 2013-01-28 Determining the characteristics of a road surface by means of a 3d camera

Country Status (5)

Country Link
US (1) US9679204B2 (en)
EP (1) EP2812652A1 (en)
JP (1) JP6117245B2 (en)
DE (2) DE102012101085A1 (en)
WO (1) WO2013117186A1 (en)

Families Citing this family (48)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102012112724A1 (en) 2012-12-20 2014-06-26 Continental Teves Ag & Co. Ohg Method for determining a road condition from environmental sensor data
DE102012112725A1 (en) 2012-12-20 2014-06-26 Continental Teves Ag & Co. Ohg Friction estimation from camera and wheel speed data
DE102013101639A1 (en) 2013-02-19 2014-09-04 Continental Teves Ag & Co. Ohg Method and device for determining a road condition
DE102013221713A1 (en) 2013-10-25 2014-12-11 Continental Automotive Gmbh Method for displaying image information on a head-up instrument of a vehicle
DE102013223014A1 (en) * 2013-11-12 2015-05-13 Robert Bosch Gmbh Driver assistance system for a motor vehicle for the anticipatory detection of a road surface
DE102013018926B4 (en) * 2013-11-13 2021-06-02 Audi Ag Method and system for determining a value for an altitude
DE102013223367A1 (en) 2013-11-15 2015-05-21 Continental Teves Ag & Co. Ohg Method and device for determining a road condition by means of a vehicle camera system
DE102013224791A1 (en) * 2013-12-03 2015-06-03 Continental Teves Ag & Co. Ohg A method of detecting at least one lane marking of a lane ahead of a vehicle
WO2015104816A1 (en) * 2014-01-09 2015-07-16 川崎重工業株式会社 Vehicle, and method of assisting driving of same
DE102014207084A1 (en) * 2014-04-14 2015-10-15 Siemens Aktiengesellschaft Method, device and system for detecting road damage
EP2982951B1 (en) * 2014-08-05 2018-10-10 Joseph Vögele AG Thermography module for road finishers
EP2990531A1 (en) 2014-08-27 2016-03-02 Joseph Vögele AG System for road finisher with a temperature measuring device, method for determining a cooling behaviour und computer-readable storage medium
DE102014223259B4 (en) 2014-11-14 2021-10-14 Conti Temic Microelectronic Gmbh Method for estimating the course of a lane in a lane
JP6425243B2 (en) * 2014-12-22 2018-11-21 国立研究開発法人産業技術総合研究所 Identification device and identification method
US20160292518A1 (en) * 2015-03-30 2016-10-06 D-Vision C.V.S Ltd Method and apparatus for monitoring changes in road surface condition
DE102015206335A1 (en) 2015-04-09 2016-10-13 Bayerische Motoren Werke Aktiengesellschaft Procedure for warning a road user
DE102015007592A1 (en) * 2015-06-16 2016-12-22 Audi Ag Trajectory-based suspension control
US9566963B2 (en) 2015-06-25 2017-02-14 Robert Bosch Gmbh Method of decreasing braking distance
JP6468162B2 (en) 2015-10-19 2019-02-13 株式会社デンソー Obstacle notification device
DE102015222926A1 (en) * 2015-11-20 2017-05-24 Robert Bosch Gmbh Automatic parking
US20210080242A1 (en) * 2015-12-04 2021-03-18 Pavement Analytics Limited Method and system for measuring deformation of a surface
DE102016203959A1 (en) * 2016-03-10 2017-09-14 Robert Bosch Gmbh Infrastructure recognition apparatus for a vehicle, method for generating a signal, and method for providing repair information
DE102016110461A1 (en) * 2016-06-07 2017-12-07 Connaught Electronics Ltd. Method for detecting an inclination in a roadway for a motor vehicle, driver assistance system and motor vehicle
DE102016210545A1 (en) * 2016-06-14 2017-12-14 Continental Automotive Gmbh Method and system for detecting properties of accumulated water on a track
US10339391B2 (en) * 2016-08-24 2019-07-02 Gm Global Technology Operations Llc. Fusion-based wet road surface detection
JP6838340B2 (en) * 2016-09-30 2021-03-03 アイシン精機株式会社 Peripheral monitoring device
US10078334B2 (en) * 2016-12-07 2018-09-18 Delphi Technologies, Inc. Vision sensing compensation
DE102016224539A1 (en) 2016-12-08 2018-06-14 Brose Fahrzeugteile Gmbh & Co. Kommanditgesellschaft, Bamberg Method for controlling an adjusting device with a anti-trapping function
DE102016225352B4 (en) 2016-12-16 2018-10-04 Volkswagen Aktiengesellschaft Method for estimating a coefficient of friction of a roadway by means of a motor vehicle and control device and motor vehicle
US10696227B2 (en) 2017-01-12 2020-06-30 Mobileye Vision Technologies Ltd. Determining a road surface characteristic
US11266054B2 (en) 2017-01-24 2022-03-08 Cnh Industrial America Llc System and method for automatically estimating and adjusting crop residue parameters as a tillage operation is being performed
US10123475B2 (en) 2017-02-03 2018-11-13 Cnh Industrial America Llc System and method for automatically monitoring soil surface roughness
US10262206B2 (en) 2017-05-16 2019-04-16 Cnh Industrial America Llc Vision-based system for acquiring crop residue data and related calibration methods
US11364915B2 (en) * 2017-12-07 2022-06-21 Nissan Motor Co., Ltd. Road condition determination method and road condition determination device
KR102133973B1 (en) * 2018-11-27 2020-07-15 넥센타이어 주식회사 Tire abrasion confirmation system and method thereof
US11373532B2 (en) 2019-02-01 2022-06-28 Hitachi Astemo, Ltd. Pothole detection system
GB2584383B (en) * 2019-02-08 2022-06-15 Jaguar Land Rover Ltd Vehicle control system and method
WO2020198134A1 (en) * 2019-03-22 2020-10-01 Vergence Automation, Inc. Lighting-invariant sensor system for object detection, recognition, and assessment
DE102019110216A1 (en) * 2019-04-17 2020-10-22 Zf Friedrichshafen Ag Method for detecting future lanes available for a vehicle as well as control device, vehicle, computer program and computer-readable data carrier
EP3959485B1 (en) * 2019-04-23 2024-04-03 Google LLC Generation of surface maps to improve navigation
CN112304293B (en) * 2019-08-02 2022-09-13 北京地平线机器人技术研发有限公司 Road height detection method and device, readable storage medium and electronic equipment
CN113015887A (en) * 2019-10-15 2021-06-22 谷歌有限责任公司 Navigation directions based on weather and road surface type
US11514594B2 (en) 2019-10-30 2022-11-29 Vergence Automation, Inc. Composite imaging systems using a focal plane array with in-pixel analog storage elements
RU2757037C1 (en) * 2020-04-23 2021-10-11 Общество с ограниченной ответственностью «Яндекс Беспилотные Технологии» Method and system for identifying the presence of a track on the current terrain
CN112504181B (en) * 2020-12-25 2021-12-03 山东大学 Inspection robot and method for measuring pavement evenness
CN113104033B (en) * 2021-05-11 2022-06-03 东风柳州汽车有限公司 Low-speed automatic driving method, device, equipment and storage medium
DE102022211241A1 (en) 2022-10-24 2024-04-25 Continental Autonomous Mobility Germany GmbH Detecting road surface coverage on a roadway
CN116879866B (en) * 2023-09-06 2023-11-14 四川华腾公路试验检测有限责任公司 Road surface flatness evaluation method based on laser radar three-dimensional data

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2012110030A2 (en) * 2011-02-14 2012-08-23 Conti Temic Microelectronic Gmbh Estimation of coefficients of friction using a 3-d camera

Family Cites Families (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0735522A (en) 1993-07-23 1995-02-07 Nippon Doro Kodan Method of measuring profile of cross-section of paved road surface with the use of laser
DE19856510C2 (en) 1998-02-20 2002-10-24 Cegelec Anlagen Und Automatisi Method and system for determining unevenness and damaged areas in the surface of a traffic area
JP2001334921A (en) * 2000-05-30 2001-12-04 Fuji Heavy Ind Ltd Estimating device for surface friction coefficient of vehicle
US20020176608A1 (en) 2001-05-23 2002-11-28 Rose David Walter Surface-profiling system and method therefor
DE10155488A1 (en) 2001-11-13 2003-05-28 Wilhelm Caspary Method for recording the condition of a road surface uses a vehicle heading along a road in a preset direction with a scanner emitting pulsed oscillating laser beams at predefined angular stages
AU2003263131A1 (en) 2002-08-09 2004-03-19 Automotive Distance Control Systems Gmbh Means of transport with a three-dimensional distance camera and method for the operation thereof
US6807473B1 (en) 2003-04-09 2004-10-19 Continental Teves, Inc. Road recognition system
JP4453382B2 (en) 2004-02-10 2010-04-21 トヨタ自動車株式会社 Vehicle travel control device
JP4092308B2 (en) 2004-06-02 2008-05-28 トヨタ自動車株式会社 Boundary line detection device
DE102004055069B4 (en) * 2004-07-15 2007-02-15 Daimlerchrysler Ag Multidimensional road survey
DE102004048637A1 (en) * 2004-10-04 2006-04-06 Daimlerchrysler Ag Roadway surface`s high sections detecting method, involves measuring concurrent surface area of roadway using sensor arrangement that is arranged in driving direction of measuring vehicle
DE102004057296A1 (en) 2004-11-26 2006-06-08 Daimlerchrysler Ag Lane departure warning with distinction between lane markings and the construction boundary of the lane
DE102005039167A1 (en) 2005-08-17 2007-02-22 Daimlerchrysler Ag Lane departure warning driver assistance device for vehicle, has evaluating unit evaluating present surrounding situation by comparing location of boundary with markings or edges, and adjusting unit adjusting warning output to situation
DE102006012289A1 (en) * 2006-03-17 2007-09-20 Man Nutzfahrzeuge Ag Motor vehicle e.g. commercial vehicle, driving assistance system controlling method, involves transferring parameter into driving assistance system, and sensing roadbed for determining visually observable parameter in space of motor vehicle
US8306747B1 (en) * 2007-01-19 2012-11-06 Starodub, Inc. Travel way measurement system
RU2009148504A (en) 2007-06-08 2011-07-20 Теле Атлас Б.В. (NL) METHOD AND DEVICE FOR CREATING A PANORAMA WITH MULTIPLE SURVEILLANCE POINTS
EP2048476B1 (en) 2007-10-08 2013-12-11 Delphi Technologies, Inc. Driver assistance method
JP5089545B2 (en) 2008-09-17 2012-12-05 日立オートモティブシステムズ株式会社 Road boundary detection and judgment device
DE102009033219A1 (en) 2009-01-23 2010-07-29 Daimler Ag Method for determining a road profile of a traffic lane ahead of a vehicle
JP5188452B2 (en) * 2009-05-22 2013-04-24 富士重工業株式会社 Road shape recognition device
US8564657B2 (en) 2009-05-29 2013-10-22 Honda Research Institute Europe Gmbh Object motion detection system based on combining 3D warping techniques and a proper object motion detection
KR20120050451A (en) 2009-07-17 2012-05-18 콘티넨탈 엔지니어링 서비시스 게엠베하 Laser-based method for the friction coefficient classification of motor vehicles
DE102010011093A1 (en) * 2010-03-11 2011-09-15 Daimler Ag Method for determining a vehicle body movement
DE102010013339A1 (en) * 2010-03-30 2011-01-05 Daimler Ag Device for controlling active/semi-active engine mounting for passenger car, has evaluation unit determining unevenness in course of road surface, where adjustment of operating parameter of engine mounting is taken place based on unevenness
DE102010020688A1 (en) 2010-05-15 2011-05-19 Daimler Ag Driving track course determining method for motor vehicle, involves identifying structures limiting driving tracks based on length, contrast, direction and three dimensional position of continuous edge courses of contours of structures
DE102010045162A1 (en) * 2010-09-11 2012-03-15 Volkswagen Ag Method for transverse guidance of motor car, involves determining voidance trajectory, where wheel contact with determined pothole is avoided when predicted trajectory of motor car leads to wheel contact with pothole
DE102010063017A1 (en) 2010-12-14 2012-06-14 Robert Bosch Gmbh Method in a driver assistance system for detecting wetness on a road
DE102011100907A1 (en) * 2011-05-09 2012-01-12 Daimler Ag Device for determining road condition of roadway located in front of vehicle, comprises imaging sensor for detecting road surface, and evaluation unit for evaluation of image data
DE102011056671A1 (en) 2011-12-20 2013-06-20 Conti Temic Microelectronic Gmbh Determining a height profile of a vehicle environment using a 3D camera
DE102012024874B4 (en) 2012-12-19 2014-07-10 Audi Ag Method and device for predicatively determining a parameter value of a vehicle passable surface

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2012110030A2 (en) * 2011-02-14 2012-08-23 Conti Temic Microelectronic Gmbh Estimation of coefficients of friction using a 3-d camera

Also Published As

Publication number Publication date
DE102012101085A1 (en) 2013-08-14
DE112013000914A5 (en) 2014-10-23
JP2015510119A (en) 2015-04-02
US20140347448A1 (en) 2014-11-27
JP6117245B2 (en) 2017-04-19
US9679204B2 (en) 2017-06-13
WO2013117186A1 (en) 2013-08-15

Similar Documents

Publication Publication Date Title
WO2013117186A1 (en) Determining the characteristics of a road surface by means of a 3d camera
EP2675685B1 (en) Estimation of coefficients of friction using a 3-d camera
EP3069296A1 (en) Method and device for determining a roadway state by means of a vehicle camera system
DE102004018088B4 (en) Lane recognition system
DE102005044486B4 (en) Method for detecting the surface condition of a road, as well as detection system and driver assistance system for implementing the method
WO2014094766A1 (en) Method for determining a carriageway condition from environment sensor data
WO2013091620A1 (en) Determining a vertical profile of a vehicle environment by means of a 3d camera
EP2850384B1 (en) Method and system for the automatic optical inspection of the tread pattern of at least one wheel of a vehicle
EP2889583A1 (en) Method and device for creating an adapted safety map for a motor vehicle
EP2561419A1 (en) Method for determining the course of the road for a motor vehicle
DE102013112459A1 (en) An apparatus and method for assessing a probability of a collision between a vehicle and a target, vehicle collision avoidance system, and methods for avoiding a collision between a vehicle and a target
DE102016000209A1 (en) A control system and method for determining a pavement irregularity
DE102011007608A1 (en) Method for active chassis controlling of vehicle i.e. car, involves determining gear control information for regulating wheel suspension system, and performing active controlling of suspension system based on regulated control information
DE102018008788A1 (en) Method for determining a road surface roughness
DE102021124810A1 (en) NEURAL VEHICLE NETWORK
EP2715666A1 (en) Method for determining a pitching movement in a camera installed in a vehicle, and method for controlling a light emission from at least one headlamp on a vehicle
DE112020000735T5 (en) VEHICLE CONTROL SYSTEM AND METHODS
DE102014223259B4 (en) Method for estimating the course of a lane in a lane
WO2019002482A1 (en) Method and device for determining a quality of a surface in the surroundings of a vehicle
DE102011083610A1 (en) Speed control for a motor vehicle
DE102020007770B4 (en) Method for determining uneven ground on a road by means of a motor vehicle system, and system
DE102013221713A1 (en) Method for displaying image information on a head-up instrument of a vehicle
DE102022100013A1 (en) System and method for vehicle lighting based roadway disruption warning
DE102019118106B4 (en) Method for determining a visibility
DE102018103060B3 (en) A method for estimating a wall of a wall and for activating an active triangulation of a matrix headlight system of a motor vehicle

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20140910

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20170216

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: CONTI TEMIC MICROELECTRONIC GMBH

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN WITHDRAWN

18W Application withdrawn

Effective date: 20210126