DE102007017293A1 - Evaluation device for use in motor vehicle, has determination unit determining roadway limits using both image information captured by black-and-white camera and color camera which is aligned in rear side of vehicle - Google Patents

Abstract

The device (S) has a input recording image information (I-K1) captured by a black-and-white camera (K1) e.g. complementary metal oxide semiconductor camera, aligned in forward direction of a motor vehicle (I). An input records image information (I-K2) captured by a color camera (K2) aligned in backwards direction of the vehicle. A determination unit determines roadway limits e.g. road marking of specific color, using the both image information. The determination unit verifies a validity of the detected roadway limit based on the information (I-K2). Independent claims are also included for the following: (1) a method for operating a driver assistance system for a vehicle (2) a computer program product comprising instructions for operating a driver assistance system for a vehicle.

Description

The The invention relates to an evaluation device for a driver assistance system for a vehicle.

Under the concept of driver assistance systems (English: ADAS, Advanced Driver Assistance Systems) functions are summarized which to assist the driver of a motor vehicle. The aim of the driver assistance systems is often the increase safety by avoiding dangerous situations their emergence and the support of the driver for accident prevention in critical situations. Other goals are the increase of comfort through stress reduction and relief the driver in standard situations, facilitating orientation by situation-dependent prepared and driver-appropriate mediated Environment information, as well as the increase of driving pleasure.

Examples for driver assistance functions are traction control or traction control such as ABS (antilock braking system), ASR (traction control), ESP (Electronic Stability Program), EDS (Electronic Stability Program) Differential lock), as well as adaptive headlights, fade-in and dimming assist for driving lights, night vision systems, Cruise Control, Park Assist, Brake Assist, ACC (Adaptive Cruise Control) distance control, distance warning, turn assistant, traffic jam assistant, Lane Detection System, Lane Keeping Assist, Lane Keeping Assist, Lane Change Assistant, ISA (Intelligent Speed Adaptation), ANB (Automatic Emergency braking), curve assistant, tire pressure monitoring system, driver condition detection, Traffic sign recognition, Platooning.

Of the Invention is based on the object, an efficient evaluation device for a driver assistance system, and a corresponding Driver assistance system, a method for operating a driver assistance system and to show a computer program product.

These Task is performed by an evaluation device with the characteristics of Claim 1, and by a driver assistance system, by a method and a computer program product with features of siblings Claims solved. Advantageous embodiments and further developments are the subject of dependent claims.

The Inventive evaluation device comprises a Input for receiving first image information, wherein the first Image information from a forward-facing black and white camera and an input for receiving second image information, the second image information being from one in the reverse direction aligned color camera, and an investigator component for determining at least one lane lane boundary using the first image information and the second image information.

The Acquisition of the image information evaluated by the evaluation device takes place via two different cameras: a black and white camera and a color camera. Accordingly, the correspond first image information a black and white or grayscale image, while the second image information corresponds to a color image. The black and white camera is facing forward. This means that the black and white camera at least partially the when driving in a forward direction in front of the vehicle or when driving in the backward direction behind the environment surrounding the vehicle. On the other hand is the color camera aligned to the rear, so that the color camera at least partially when driving in a forward direction behind the vehicle lying or when driving in the reverse direction detected in front of the vehicle environment. The attachment of the two Cameras can be based on such driver assistance systems Cameras known manner done.

The Evaluation device receives first and second image information. This image information may be the original one act on the respective camera captured data, or by a already processed version of the originally recorded Dates.

By an evaluation of the first and second image information determined the evaluation device at least one lane boundary. The Determining a lane boundary corresponds to the decision that a lane lane boundary is present, and optionally the determination of parameters or properties of this lane boundary. The evaluation device can have an output for outputting information concerning the determined road lane boundary, such as z. For example, the indication of whether a lane lane boundary exists, which way the lane lane boundary is, which position the lane lane boundary at single or multiple points has, what course the lane boundary having. Preferably, the location or the course of the determined Lane lane boundary of the evaluation device relative to the Vehicle specified; alternatively, an indication of the absolute position of position or course of the lane lane boundary possible.

In a development of the invention, the at least one lane lane boundary is a lane marking. Fahrbahnmar cations are also referred to as road marking or ground marking; they are signs on the road surface. Examples are continuous or broken centerlines, or even two or more parallel lines. The lane marker may be a lane marker of a particular color. In this way, z. B. a certain color can be specified, and the evaluation only determines road markings of this color. Furthermore, z. For example, a ranking of colors can be specified, wherein the evaluation device searches for lane markings in the order of this ranking. Additionally or alternatively to the lane marking, the at least one lane lane boundary may be a lane edge, such as a lane. B. a strip of grass and / or earth, a guard or guardrail or a wall.

In Development of the invention, the determination component is formed for detecting at least one road lane boundary below Using the first image information and checking the validity of the at least one detected lane lane boundary based on the second image information. In the detection of at least a lane lane boundary will be this lane boundary in found the first image information. Preferably, the determining component designed to check the validity the at least one detected lane lane boundary based on a the second image information taken from the color of at least one detected lane lane boundary. The validity relates in this case on the further use of the detected lane lane boundary: in a recognized as lane line boundary can Information regarding this road lane boundary another Component to be made available for use; at a lane boundary recognized as invalid preferably no information is transmitted. In this way can z. B. a lane lane boundary based on the first image information be detected, this lane boundary only as is assumed if it has a certain color. The validation can be checked under Use of a coordinate transformation, where pixels the first image information in the range of the second image information be transformed or vice versa.

one Embodiment of the invention according to the determining component configured to detect at least one lane lane boundary below Using the first image information and at least one Lane lane boundary using the second image information, and for matching the at least one using the first one Image information detected lane boundary with the at least one detected using the second image information Roadway lane boundary.

one Embodiment of the invention according to the determining component configured to detect a plurality of lane lane boundaries using the first image information and to select one of the lane lane boundaries based on the second image information.

The inventive driver assistance system comprises an evaluation device of the type described above, as well as the black and white camera and the color camera, and an application unit for using the at least one determined by the evaluation device Lane lane boundary as part of a driver assistance function. In the driver assistance function, it may be z. B. a lane detection system, a lane departure warning, a lane keeping assistance, or a lane change assistant.

According to the Methods for operating a driver assistance system are of a black-and-white camera oriented in the forward direction of the vehicle captured first image information, from one in the reverse direction the vehicle-oriented color camera become second image information detected, and using the first and the second image information At least one road lane boundary is determined.

The inventive computer program product has an input for receiving first image information, which from a front-facing vehicle of the black-and-white camera an input for receiving second image information, which of a in the reverse direction of the vehicle aligned color camera were detected, as well as an investigation component for determining at least one lane lane boundary using the first and the second image information. Under a computer program product may be in the context of the present invention in addition to the actual Computer program (with his about the normal physical interaction technical effect between program and arithmetic unit) in particular a record carrier for the computer program, a file library, a configured arithmetic unit, as well For example, a storage device or a server on the or the files associated with the computer program are to be understood.

The computer program product according to the invention and the method according to the invention are particularly suitable for the evaluation device according to the invention, and this may also apply to the refinements and developments. For this purpose, they may comprise further suitable means or steps.

in the The following is the invention with reference to an embodiment explained in more detail. Showing:

1 : a motor vehicle with a driver assistance system,

2 : a section of a driver assistance system.

This in 1 illustrated motor vehicle 1 drive currently on the right lane SP1, in 1 to the right. At the right edge of the right lane SP1 is the edge of the road R. This may be, for. B. act a strip of grass and / or earth. As a rule, the roadway edge R differs in color from the roadway SP1.

Between the right lane SP1 and the left lane SP2 are the lane markers MW and MG. In this case, the lane marking MW is a normally used white lane marking, and at the lane marking MG by a z. B. due to a construction site mounted yellow road marking. In this case, the yellow lane marker MG is MG, not the white lane marker MW from the driver of the vehicle 1 and therefore also to be observed by the driver assistance system.

The vehicle 1 has a black and white video camera K1, z. B. a CMOS camera. A black and white camera is used because it is less expensive than a color camera. Moreover, the resolution of a black-and-white camera is larger for a given number of pixels than for a color camera. For the driver of the vehicle 1 the use of a black-and-white camera is not disadvantageous, as it is from the camera 1 captured image is not displayed. The camera K1 is directed in the direction of travel, thus taking pictures of the front of the vehicle 1 lying on the environment. For this purpose, it may, for. B. near the roof and the windshield of the vehicle 1 be mounted. With B1 the schematically represented by the camera K1 detected area is designated.

Using a lane detection algorithm suitable for this purpose, the images taken by the camera K1 are evaluated and the lane edge R and the lane markings MW and MG are thereby detected. Because of this detection, the course of the lane markings MW and MG, as well as the road edge R with respect to the own vehicle 1 known. For this purpose, known methods can be used, for. B. described in WO 2007/002964 ,

Of the detected lane R and the detected lane markings MW and MG can z. B. for the driver assistance function Lane departure assistant (English: Lane Departure War ning) used become. Here, the driver should inadvertently leave be warned of its lane.

There are different lane markings MW and MG, of which only one is valid, as it is z. B. in a construction site area is often the case, it must be decided by the driver assistance system, which of the lane markers MW and MG should be used. One way of doing this is to derive the valid lane from past driving behavior by selecting that lane marking MW or MG in whose lane the driver had been located at the beginning of the jobsite area. However, this is often not unique, for. B. because the vehicle 1 was located both within the lane marked by the lane marking MW and within the lane delimited by the lane marking MG, or because the driver had initially misclassified when entering the construction site area.

The vehicle 1 In addition to the black and white camera K1 has a color video camera K2. The color camera K2 is oriented to the rear and thus absorbs the environment behind the vehicle. With B2 is the schematically represented captured by the camera K2 area designated. The color camera K2 can z. As the support of the driver when driving in the reverse direction, z. B. when parking, help. For this purpose, the images taken by the color camera K2 images can be displayed on a display, eg. B. on the display of the navigation system. The resolution of the images taken by the color camera K2 is usually reduced compared to the black-and-white camera K1.

2 shows a section of the driver assistance system of the vehicle 1 , The evaluation component S is provided by the camera K1 with the information I-K1 and the camera K2 with the information I-K2. The information I-K1 and I-K2 can be the image data captured by the respective camera K1 or K2, or data resulting from a processing of this image data.

The evaluation component S evaluates the information mations I-K1 and I-K2 together and decides based on this evaluation, which lane marking MW or MG is currently used. The evaluation unit S outputs to the application unit A the information IS which shows the profile of the roadway marking MW or MG to be used relative to the own vehicle 1 can be seen. The application unit A uses the information IS in terms of the respective driver assistance function, in a lane departure warning driver assistance function z. B. to determine an unintended lane change of the vehicle 1 ,

The application unit A is connected to an interface I. This may be a man-machine interface and / or an interface for the autonomous control of the vehicle 1 act. About a man-machine interface can be issued in an audible, visual or haptic manner to the driver information or warning, z. B. due to leaving the lane SP1. Through an interface to the autonomous control of the vehicle can be used in the driving behavior of the vehicle 1 be intervened without the need for the involvement of the driver, z. B. by slowing down or steering the vehicle 1 ,

The joint evaluation of the information I-K1 and I-K2 by the evaluation component S can, for. B. be done as follows:
The information I-K1 are evaluated and thereby the lane markings MW and MG detected. Individual points of the detected course of the road markings MW and MG or the entire course of the road markings MW and MG is transformed by a coordinate transformation into the image of the camera K2. Such a coordinate transformation is possible by using the positions of the cameras K1 and K2 relative to each other, the relative orientation of the cameras K1 and K2, and the aperture angle of the cameras K1 and K2. A reverse transformation is possible, ie from the image of the camera K2 in the image of the camera K1. After the coordinate transformation, it is possible to check which color the road markings MW and MG detected in the black and white image of the camera K1 have in the color image of the camera K2.

The points of the gradients of the road markings MW and MG detected in the black-and-white image of the camera K1 only lie exactly on the road markings of the color image of the camera K2 if the roadway track SP1 has no horizontal curvature. In the case of cornering, it is therefore possible, the current steering wheel adjustment of the vehicle 1 to be considered in the coordinate transformation. As a result, a road curvature can be compensated.

It is also possible to evaluate both the information I-K1 and thereby detect the lane markers MW and MG in the black and white image of the camera K1, as well as to evaluate the information I-K2 and thereby the lane markers MW and MG in the color image of the camera K2 to detect. The road markings MW and MG detected in the color image of the camera K2 are not sufficient for the application unit A, as is usually the course of the front of the vehicle 1 lying lane track SP1 is required, while using only the camera K2 only the course of the behind the vehicle 1 lying lane track SP1 can be determined.

The two sets of the detected lane markings MW and MG can then be compared with one another to effect an assignment of the lane marking MW from the image of the camera K1 to the lane marking MW from the image of the camera K2, and an assignment of the lane marking MG from the image of the camera Camera K1 to the lane marker MG from the image of the camera K2. This assignment takes place via a comparison of road marking parameters, such as. B. the position of the lane marking MW or MG relative to the vehicle 1 , Because of this association, it is known what color the road markings MG and MW have.

By the described joint evaluation of the information I-K1 and I-K2 - using both the coordinate transformation as well as using the balance of the two images detect lane marking - can thus be detected what color are in the picture of the black and white camera K1 detected pavement marking MG and MW have. Based on this Color can be decided which road marking MG or MW is the valid.

The procedure applied with respect to the lane markings MG and MW can also be used for the lane edge R. By evaluating the information I-K1, the roadside edge R is detected. In this case, it is utilized that the roadway edge R has a different texture than the roadway track SP1. In this case, however, the possible range of manifestations of road edges makes the detection of the roadway edge R on the basis of a black-and-white image very unreliable. The detected roadway edge R can be checked by a coordinate transformation - as explained above from the image of the camera K1 in the image of the camera K2 or vice versa - and adjusted if necessary. For this purpose, for. B. green or brown spots associated with the roadway edge R, and gray to black areas of the lane SP1. As part of the adaptation of the camera K1 detected in the image Fahrbahnrandes R only those points of the detected in the image of the camera K1 roadway edge R are accepted as the actual lane edge, which have the corresponding color within the images of the color camera K2. In this way, the reliability of the detection of the roadway edge R is significantly increased.

Of the Evaluation component S thus leads to a merger of the high-resolution Pictures of the black and white camera K1 with the colored pictures of the Camera K2 through. This can be a reliable detection of lane lane boundaries, d. H. of lane marking MW or MG and road edge R, be effected. Due to a coordinate transformation or a different balance of black and white images and the color images, it is not necessary, one to the front use aligned color camera.

The The invention has been described in an embodiment described. It is understood that many changes and modifications are possible without the scope of the Is left invention.

QUOTES INCLUDE IN THE DESCRIPTION

This list The documents listed by the applicant have been automated generated and is solely for better information recorded by the reader. The list is not part of the German Patent or utility model application. The DPMA takes over no liability for any errors or omissions.

Cited patent literature

• - WO 2007/002964 [0024]

Claims (12)

Evaluation device (S) for a driver assistance system for a vehicle ( 1 ) having an input for receiving first image information (I-K1), which is from a forward direction of the vehicle ( 1 a black and white camera (K1), an input for receiving second image information (I-K2), which is from a vehicle in the backward direction (K1). 1 ) detected color camera (K2), a determination component for determining at least one lane boundary (MW, MG, R) using the first image information (I-K1) and the second image information (I-K2). Evaluation device (S) according to claim 1, wherein it at least one road lane boundary (MW, MG, R) is a lane marking (MW, MG). Evaluation device (S) according to claim 1, wherein it at least one road lane boundary (MW, MG, R) is a lane marking (MW, MG) of a particular color. Evaluation device (S) according to one of the claims 1 to 3, wherein it is in the at least one lane boundary (MW, MG, R) is about a lane edge (R). Evaluation device (S) according to one of the claims 1 to 4, wherein the determination component is configured to detect of at least one lane lane boundary (MW, MG, R) using the first image information (I-K1) and to check the validity of the at least one de tektierten lane boundary (MW, MG, R) based on the second image information (I-K2). Evaluation device (S) according to claim 5, wherein the Determination component is designed to check the validity of the at least one detected lane lane boundary (MW, MG, R) based on a second image information (I-K2) taken Color of the at least one detected lane lane boundary (MW, MG, R). Evaluation device (S) according to claim 5 or 6, wherein checking the validity using a coordinate transformation takes place. Evaluation device (S) according to one of the claims 1 to 7, wherein the determination component is configured to detect of at least one lane lane boundary (MW, MG, R) using the first image information (I-K1) and at least one lane lane boundary (MW, MG, R) using the second image information (I-K2), and for matching the at least one using the first one Image information (I-K1) detected road lane boundary (MW, MG, R) with the at least one using the second image information (I-K2) detected lane lane boundary (MW, MG, R). Evaluation device (S) according to one of the claims 1 to 8, wherein the determination component is configured to detect a plurality of lane lane boundaries (MW, MG, R) using of the first image information (I-K1) and to select one the detected lane lane boundaries (MW, MG, R) on the basis of second image information (I-K2). Driver assistance system for a vehicle ( 1 ), with an evaluation device (S) according to one of claims 1 to 9, the black and white camera (K1) and the color camera (K2), an application unit (A) for using the at least one of the evaluation device (S) determined lane boundary (MW, MG, R) as part of a driver assistance function. Method for operating a driver assistance system for a vehicle ( 1 ), wherein from one in the forward direction of the vehicle ( 1 ) aligned black and white camera (K1) first image information (I-K1) are detected from a vehicle in the backward direction ( 1 ) aligned color camera (K2) second image information (I-K2) are detected using the first image information (I-K1) and the second image information (I-K2) at least one lane boundary (MW, MG, R) is determined. Computer program product for a driver assistance system for a vehicle ( 1 ) having an input for receiving first image information (I-K1), which is from a forward direction of the vehicle ( 1 a black and white camera (K1), an input for receiving second image information (I-K2), which is from a vehicle in the backward direction (K1). 1 ) detected color camera (K2), a determination component for determining at least one lane boundary (MW, MG, R) using the first image information (I-K1) and the second image information (I-K2).