DE102012112164A1 - VIDEO-BASED DETECTION OF OBSTACLES ON A TRACK - Google Patents

Abstract

The invention relates to a method for video-based detection of obstacles on a roadway with the following steps: Receiving image data (S10) that are generated by a 3D camera provided for capturing an area of the surroundings in front of a vehicle, the 3D camera in this way is aligned and designed so that the area covered by it encompasses at least the entire width of the lane in front of the vehicle, evaluating the image data (S12) to detect obstacles across the direction of travel of the vehicle, and outputting data (S14) on detected obstacles.

Description

The invention relates to a method and a device for video-based detection of obstacles on a roadway, which are particularly suitable for use in driver assistance systems.

The DE 10 2009 033 219 A1 describes a method for determining a road profile of a traffic lane in front of a vehicle on the basis of captured image data. In this case, a road height profile of the traffic lane is determined by an estimation device, for example a Kalman filter. In particular, the image data are generated by a camera fixedly arranged, for example, in the front region of the vehicle. By measuring the road height profile in front of the vehicle and integrating time, expected unevenness in the floor of a respective vehicle wheel of the vehicle can be determined when the vehicle passes over the measured area. Based on this, the chassis of the vehicle can be regulated accordingly. Formally, this process is a stochastic measurement process.

Object of the present invention is therefore to propose a method and an apparatus for video-based detection of obstacles on a roadway compared to that from the DE 10 2009 033 219 A1 are further improved.

This object is solved by the subject matters of the independent claims. Further embodiments of the invention will become apparent from the dependent claims.

An underlying idea of the invention is to detect obstacles on a roadway by evaluating image data of a 3D camera, that is to implement a video-based recognition, in which

is used by means of a 3D camera, an active evaluation zone for the detection of obstacles on a roadway, which is not limited to the immediate area of the vehicle or a vehicle tire, but comprises the entire road surface (width) in the Vorschaubereich of the vehicle. By using a 3D camera, the image data also contain spatial information of the Vorschaubereichs of the vehicle. Due to the active evaluation zone and the spatial information, it is now possible to detect obstacles transverse to the direction of travel, such as bumps, edges (discontinuities) on the vehicle trajectory and / or road curvatures, which extend in particular over the entire width of a road and with the previously used algorithms impossible or very difficult to detect, since they do not cause jumps in a height profile transverse to the direction of travel, and therefore can not be detected with a specialized on the detection of such jumps image processing algorithm in the rule.

An embodiment of the invention now relates to a method for video-based detection of obstacles on a roadway comprising the steps of: receiving image data generated by a 3D camera provided for detecting a region of an environment in front of a vehicle, the 3D camera thus is aligned and formed such that the area covered by it comprises at least the entire width of the roadway in front of the vehicle, evaluating the image data for detecting obstacles transverse to the direction of travel of the vehicle, and outputting data to detected obstacles.

The evaluation of the image data for detecting obstacles transverse to the direction of travel of the vehicle may include the detection of several pixels in space, which are arranged approximately on a horizontal line, and determining an approximate height of the obstacle depending on the detected pixels.

The evaluation of the image data for detecting obstacles transverse to the direction of travel of the vehicle may include the detection of several pixels in space, which are arranged approximately on a vertical line, and the calculation of a roadway curvature depending on the detected pixels.

Outputting data on detected obstacles may include determining whether the vehicle can run over a detected obstacle and / or whether a detected obstacle may cause damage to the vehicle when crossing it, and outputting the result of the determining.

Outputting data on detected obstacles depending on the result of the determination may include issuing a warning, an intervention in the vehicle control and / or a change of vehicle characteristics.

A further embodiment of the invention relates to a device for video-based detection of obstacles on a roadway having first means for receiving image data generated by a 3D camera provided for detecting a region of an environment in front of a vehicle, the 3D camera being aligned in this way and is configured so that the area covered by it comprises at least the entire width of the roadway in front of the vehicle, second means for evaluating the image data for detecting obstacles transversely to the direction of travel of the vehicle, and third means for outputting data to detected obstacles.

The second means may be adapted to carry out a method according to the invention and as described herein, and the third means may be adapted to carry out a method according to the invention and as described herein.

Finally, another embodiment of the invention relates to a driver assistance system having a device according to the invention and as described herein.

Further advantages and possible applications of the present invention will become apparent from the following description in conjunction with the / in the drawing (s) illustrated embodiment (s).

In the description, in the claims, in the abstract and in the drawing (s), the terms and associated reference numerals used in the list of reference numerals recited below are used.

The drawing (s) show / show in

1 a flowchart of an embodiment of a method for video-based detection of obstacles on a roadway according to the invention;

2 an example of an image of a detected area of an environment in front of a vehicle with an area of a raised road surface at a construction site as an obstacle;

3 an example of an image of a detected area of an environment in front of a vehicle having an area of a raised road surface for traffic calming as an obstacle;

4 two examples of lanes with obstacles transverse to the direction of travel of a vehicle in a side view; and

5 An embodiment of a device for video-based detection of obstacles on a roadway according to the invention.

In the following description, identical, functionally identical and functionally connected elements may be provided with the same reference numerals. Absolute values are given below by way of example only and are not to be construed as limiting the invention.

In the present invention, starting from an in-vehicle stereo or. 3D camera system observing the road surface in the surrounding area in front of the vehicle. The image data of the acquired environmental region generated by the 3D camera system are processed by an algorithm whose flow chart is shown in FIG 1 is shown and will be explained below.

The algorithm receives in a first step S10 the image data from the 3D camera system for further processing. In a subsequent step S12, the received image data for detecting obstacles transverse to the direction of travel are evaluated.

The evaluation may include various steps to detect various obstacles, as explained below.

In order to detect discontinuities in the course of the surface to be traversed, a plurality of pixels arranged on a horizontal line are detected in a step S120, as is the case with the in 2 and 3 shown exemplary captured images of detected environmental areas in front of a vehicle.

In 2 is the course of a roadway 14 shown in front of a vehicle as captured by a 3D camera system. On the roadway 14 there is an area 10 (framed by the dashed rectangle), in which a section 100 the road surface is lifted, as is often the case for example when renewing the asphalt surface of a roadway. Due to the section 100 with the lifted road surface arise a front edge 102 and a back edge 104 each between the surface of the roadway 14 and the section 100 , A crossing of these two edges 102 and 104 , which are transverse to the direction of travel of the vehicle, could lead to damage to the vehicle. Therefore, a speed limit usually applies in the area of such sections. Through step S120, the algorithm can now use the two edges 102 and 104 recognize and in a subsequent step S122 determine the approximate height of these edges on the basis of in the 3D Kemerasystem generated image data contained space or 3D information, in particular calculate or estimate.

At the in 3 example shown is in one area 12 the roadway 14 a section 120 what is done, for example, to calm traffic on certain roads. Alternatively, so-called speed bumper (brake threshold for traffic calming) be mounted on the road surface, something shorter than the section 120 are. To override the raised section of roadway 120 to allow, are at the front of the section 120 a front driveway 122 and at the back of the section 120 a back departure 124 provided an oblique transition between the section 120 and the surface of the roadway 14 define. Also the driveway 122 and the departure 124 may be recognized by their representation in captured by the 3D camera system images as multiple, arranged on a horizontal line pixels from the algorithm in step S120. The approximate height of the section 120 can in step S122 from the space information to the pixels of the driveway 122 and the departure 124 determined, in particular calculated or estimated.

Furthermore, in the course of the evaluation, the algorithm can determine the curvature of the surface to be traversed, i. the roadway are determined. For this purpose, in a step S124 pixels are recognized on a vertical line, for example the central strip of a road. Subsequently, in a step S126, the curvature of the road is calculated with the spatial positions of the pixels recognized on the vertical line.

The algorithm proceeds to a step S14 after the image data is evaluated in step S12, as explained above, in which data on detected obstacles are output. Also, the step S14 has a plurality of sub-steps, as will be explained in detail below.

First, in a step S140, the algorithm determines whether the vehicle can overrun an obstacle detected in step S12, or whether a detected obstacle when overrunning can cause damage to the vehicle.

• – ob das Fahrzeug in der Lage ist die Fahrbahnkrümmung ohne Beschädigungen zu befahren (siehe beispielsweise die in 4 oben unter 1 gezeigte Situation), oder
• – ob es eine Kante (Unstetigkeit) auf der Fahrzeugtrajektorie gibt, die entweder nicht überfahren werden kann oder Beschädigungen am Fahrzeug hervorrufen könnte (siehe beispielsweise die in 4 unten unter 2 gezeigte Situation).

For example, it can be determined

• - whether the vehicle is able to negotiate the road curvature without damage (see, for example, the in 4 the situation shown above under 1), or
• Whether there is an edge (discontinuity) on the vehicle trajectory that either can not be run over or could cause damage to the vehicle (see, for example, the in 4 below situation 2).

• – Ausgeben einer Warnung im Schritt S1420 oder Eingriff in die Fahrzeugsteuerung (beispielsweise aktiver Brems- oder Lenkeingriff) im Schritt S1422 bei unmöglich befahrbaren Fahrbahnkrümmungen (z.B. tiefliegende Garagen, zu hohe Kanten vor Garagen, zu hohe Speedbumper etc.);
• – Veränderung der Fahrzeugeigenschaften im Schritt S1424, so dass Überfahrbarkeit wieder gegeben ist und/oder Komfort beim Überfahren verbessert wird.

Subsequent to step S142, the algorithm may perform various functions based on obstacle detection:

• Issuing a warning in step S1420 or intervention in the vehicle control (for example active braking or steering intervention) in step S1422 in the case of impassable road curves (eg low-lying garages, edges too high in front of garages, speed bumpers too high, etc.);
• - Changing the vehicle characteristics in step S1424 so that override is restored and / or comfort when driving over is improved.

5 shows a block diagram of a video-based obstacle detection device 20 According to the invention, the data of a stereo-vision camera with left and right imagers 28 respectively. 30 processed. The two imagers 28 and 30 provide image data of the environment in front of a vehicle. This image data becomes an image data receiving unit 22 which may be configured to calculate the position of pixels in the space. The image data and possibly the calculated pixel positions in space are sent to an image data evaluation unit 24 transmitted, which evaluates the obtained image data to detect obstacles transverse to the direction of travel of the vehicle according to the method step S12 explained above. An obstacle issuing unit 26 then outputs data on detected obstacles, for example in the form of a warning (acoustically, visually and / or haptically), by an intervention in the vehicle control and / or by a change in the vehicle characteristics, for example by outputting a corresponding control signal for control devices of the vehicle that is responsible for the vehicle control are responsible (eg active brakes), according to the method step S14 explained above.

• – Berechnung ob eine Fläche mit den gegebenen Fahrzeugeigenschaften (Radstand, Federwege, Abstand Chassis zum Boden) überfahrbar ist (wie in 4 beispielhaft gezeigt ist).
• – Anzeige oder Warnung, falls Bereiche nicht überfahren werden können.
• – Aus der Erkennung abgeleitete Eingriffe in die Fahrzeugeigenschaften (z.B. Fahrwerkanhebung), so dass die Bereiche dennoch befahren werden können.
• – Aus der Erkennung abgeleitete Eingriffe in die Fahrzeugeigenschaften (z.B. Feder/Dämpfer-Verstellung), so dass die Bereiche (z.B. Übergangsbereiche an Parkhausrampen) möglichst „komfortabel“ überfahren werden können.

The essence of the invention is the detection of areas in the vehicle environment that can not or should not be driven due to the vehicle characteristics or whose trafficability is limited or can cause damage to the vehicle when driving. The areas in the vehicle are characterized by obstacles that are transverse to the direction of travel and therefore can not be determined by conventional algorithms for environment assessment difficult or difficult, especially since they usually do not cause jumps in a height gradient transverse to the direction of travel. Further aspects of the invention are listed below again:

• - Calculation of whether an area with the given vehicle characteristics (wheelbase, suspension travel, distance chassis to the ground) can be driven over (as in 4 is shown by way of example).
• - Display or warning if areas can not be overrun.
• - Interference in the vehicle characteristics derived from the recognition (eg suspension lift), so that the areas can still be traveled.
• - Interference derived from the recognition in the vehicle characteristics (eg spring / damper adjustment), so that the areas (eg transition areas on parking garage ramps) as possible "comfortable" can be run over.

LIST OF REFERENCE NUMBERS

10

Area with raised road surface (obstacle)

100

 Road section with raised / elevated road surface

102

leading edge of the carriageway section 100

104

rear edge of the carriageway section 100

12

Area with raised lane for traffic calming (obstacle)

120

 raised / elevated roadway section

122

front driveway to raised / elevated carriageway section 120

124

rear departure from the raised / elevated carriageway section 120

14

roadway

20

Video-based obstacle detection device

22

Image data receiving unit

24

Bilddatenauswerteeinheit

26

Obstacle output unit

28

left imager of a 3D camera

30

right imager of a 3D camera

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102009033219 A1 [0002, 0003]

Claims (8)

Method for video-based detection of obstacles ( 10 ; 12 ) on a roadway ( 14 ) comprising the steps of: receiving image data (S10) generated by a 3D camera provided to detect a region of an environment in front of a vehicle, the 3D camera being oriented and configured such that the region covered thereby comprises at least the entire width of the road ahead of the vehicle, - evaluating the image data (S12) for detecting obstacles ( 10 transverse to the direction of travel of the vehicle, and - outputting data (S14) to identified obstacles ( 10 ). A method according to claim 1, characterized in that the evaluation of the image data (S12) for detecting obstacles transverse to the direction of travel of the vehicle, the detection of multiple pixels in space (S120), which are arranged approximately on a horizontal line, and determining an approximate Height of the obstacle depending on the detected pixels (S122). A method according to claim 1 or 2, characterized in that the evaluation of the image data (S12) for detecting obstacles transverse to the direction of travel of the vehicle, the detection of several pixels in the space (S124), which are arranged approximately on a vertical line, and calculating a roadway curvature depending on the detected pixels (S126). The method of claim 1, 2 or 3, characterized in that the outputting of data (S14) to detected obstacles determining (S140), whether the vehicle can run over a detected obstacle and / or whether a detected obstacle when driving over cause damage to the vehicle , and outputting the result of the determination (S142). A method according to claim 4, characterized in that the outputting of data (S142) to detected obstacles depending on the result of the determination, outputting a warning (S1420), an intervention in the vehicle control (S1422) and / or a change of vehicle characteristics (S1424) having. Contraption ( 20 ) for the video-based detection of obstacles on a roadway with - first means ( 22 ) for receiving image data generated by a 3D camera provided for detecting a region of an environment in front of a vehicle, wherein the 3D camera is oriented and configured so that the area covered by it is at least the entire width of the road ahead of Vehicle comprises, - second means ( 24 ) for evaluating the image data for detecting obstacles transverse to the direction of travel of the vehicle, and - third means ( 26 ) for outputting data on detected obstacles. Device according to claim 6, characterized in that the second means ( 24 ) are adapted to carry out a method according to claim 2 or 3, and the third means ( 26 ) are adapted to carry out a method according to claim 4 or 5.  Driver assistance system with a device according to claim 6 or 7.

Images