WO2017157394A1

WO2017157394A1 - Method and device for a vehicle for detecting ground-based symbols

Info

Publication number: WO2017157394A1
Application number: PCT/DE2017/200020
Authority: WO
Inventors: Stefan Hegemann; Alice Natoli; Lijo Cherian Mathew
Original assignee: Conti Temic Microelectronic Gmbh
Priority date: 2016-03-16
Filing date: 2017-03-07
Publication date: 2017-09-21
Also published as: DE102016204319A1; DE112017000326A5

Abstract

The invention relates to a method and to a device for detecting ground-based symbols for vehicles and is used in particular in driver assistance functions and systems. An method according to the invention for detecting ground-based symbols (3) by means of an optical or imaging environment detection sensor (2) of a vehicle (1) comprises the following steps: a sequence of images is recorded by means of the imaging environment detection sensor (2) during the movement of the vehicle (1); a combined image of a ground area of the environment is assembled from scan areas (22; 22.1 to 22.6) of images from the image sequence; detection of the ground-based symbol (3) from the combined image of the ground area is carried out; and the ground-based symbol (3) that is detected or information regarding to the ground-based symbol (3) that is detected is output. By means of the invention, even under extremely difficult conditions (dense traffic, queue before a traffic light, low visual range) ground-based symbols (3) can be detected reliably.

Description

Method and apparatus for a vehicle for detecting ground symbols

The invention relates to a method and a device for detecting ground symbols for vehicles and finds application in particular in driver assistance functions and systems.

A camera-based recognition of traffic signs is basically state of the art. The camera-based detection of traffic signals and traffic lights and their signals is the subject of current research and development. For a more complete understanding of situations and Ver ^¬ traffic regulations, especially in the urban space, the detection of ground symbols is another challenge, not least on the way to automated driving.

It is an object of the present invention to improve the detection of soil symbols by environmental detection sensors or in some situations in the first place.

A basic idea on which the present invention is based is the observation that ground symbols are often used in cities, especially at intersections with traffic lights, but may be partially obscured there by other road users. A preceding vehicle, which partly stands or is currently driving over a ground symbol, makes it impossible for the driver, as well as for an environment detection sensor arranged in or on the vehicle, to determine the regulatory content of this soil symbol.

Under a floor symbol is understood in particular to a coating applied to the Ver ^¬ sweeping surface or road surface symbolization. Soil symbols are often used as a supplement to traffic signs and should lead to a correct and contribute to driving in road traffic. A typical representative of the soil symbols are directional arrows. For example, the direction of arrow a on a turn lane signaled ^¬ sierender Abbiegerpfeil or a direction arrow, which shows the correct direction of travel of the corresponding road or lane. Ground symbols are intended to help the driver to orient himself and to support safe driving. So z. For example, directional arrows marked 297 of the German Road Traffic Regulations (StVO) recommend that the driver be assigned to the right lane in good time.

Other soil symbols are for example applied to the roadway special characters or letter sequences, z. B. the inscription "STOP" or a "the right of way" road sign corresponding triangle, the top points down. Stop lines can also be considered as a special type of bottom symbol.

An inventive method for recognizing symbols ground by means of an optical or imaging Umgebungserfas ^¬ sungssensors of a vehicle comprising the steps of: a sequence of images is recorded by means of the imaging environment detecting sensor during movement of the vehicle;

of scanning areas of images from the sequence of images is a composite image of a bottom portion of the environment is to ^¬ sammengesetzt;

a detection of the bottom symbol is made from the quantity together ^¬ translated image of the bottom portion; and

the detected soil symbol or information regarding the detected soil symbol is output.

As an imaging environment detection sensor may be disposed inside the vehicle, looking through the windshield Mono or stereo camera, arranged at the front of the vehicle camera all-around sensor system (surround view or top view) or an imaging laser system can be used.

By means of the imaging environment detection sensor, an image of an environment and a floor area in the vicinity of the vehicle can be recorded. In particular, the surrounding environment ^¬ detection sensor may be directed substantially forward, the optical axis is substantially horizontal and / or less than 20 °, in particular less than 45 ° inclined to the horizontal. The imaging environment detection sensor may preferably be arranged in the region of the windshield, the radiator hood, the front bumper or the hood.

During a movement of the vehicle, in particular, images are taken of at least two different positions of the own vehicle and thus two different positions of the (vehicle-mounted) imaging environment detection sensor. Of course, this does not exclude that images are taken by means of the imaging environment detection sensor during standstill of the own vehicle. Advantageously the process is used only for partial occlusion of a ground symbol, so if the Umgebungserfas ^¬ sungssensor can not represent the complete floor symbol because it is partially obscured by an object. The scan area is a section (which may also be referred to as region of interest) of an image taken by the imaging environment detection sensor. This scan area can be advantageous in terms of size or extent and position. be determined in the detection range of the environment detection sensor in the context of the method and specified.

Size and / or position of the scanning area within the He ^¬ detection area of the environment detection sensor can be pre-defined preferably such that no occlusion in ^¬ nerhalb a scan area occurs.

A constant scan area can be used in pictures of the picture sequence. Alternatively, the scan area can be adjusted in terms of size and / or position in images of the image sequence.

Preferably, an adaptation can be made such that the size (and / or position) of the scan area depends on the speed of the own vehicle. The individual scan areas are preferably transformed into a stationary coordinate system or into a vehicle-fixed coordinate system, and the composite image is composed of the scan areas in this coordinate system. As a result, advantageously a perspective distortion of images of the environmental detection sensor can be eliminated.

Advantageously, images with scan regions of the image sequence are selected such that a gap-free image of the bottom region can be assembled with the soil symbol from the scan regions. The selection takes into account advantageously one or more of the following parameters: the size and / or position of the scanning areas in the images of the image sequence, the recording ^¬ times of the images of the image sequence, the movement of the order ^¬ gebungserfassungssensor between the receiving timings and the relative movement of the occluding object.

An output of the detected soil symbol or information with regard to the detected soil symbol can be effected, in particular, by virtue of the fact that the control content of the soil symbol is output, eg "straight arrow" or "STOP". The output can be made to a driver assistance system, such as intersection assistants, foreperson assistants, traffic light assistants; to a route guidance support or navigation system; to a telematics device (Vehicle to X) or to a control device for an autonomously driving vehicle.

In a particular embodiment lane boundary markings (eg, solid, dashed or formed from Bott's dots lines) not considered to be recognized as a ground ^¬ symbols within the meaning of the process. In other words, in this embodiment lane ^boundary markings are not floor symbols. The method may be limited to ground areas within lanes. In this embodiment, the detection of punctually regulating soil symbols is the focus.

Advantageously, the imaging environment detection sensor is or comprises a monocamera. This may be a front camera behind the windshield, looking in the direction of travel through the windshield. In this case, the area of the ground immediately in front of the own vehicle is not abge ^¬ forms, since the hood of the vehicle covered this. With such a camera, data can advantageously be provided for further driver assistance functions.

Preferably, the imaging environment detection sensor is or comprises a stereo camera. This offers the particular advantage that the intrinsic speed, the distance to a concealing object whose speed can be precisely determined by the stereo camera by the 3D detection.

According to an advantageous embodiment, the imaging environment detection sensor is or comprises at least one Camera of a surround view sensor system (Surround View System or Top View System). Due to the installation position, the camera of the all-round sensor system enables the image of the area of the ground immediately before (beside or behind) the own vehicle.

Preferably, the imaging environment detection sensor may be an imaging laser sensor system. This also offers the advantage of a precise 3D detection of the vehicle environment. Optionally, another environment detection sensor, in particular a radar / lidar sensor, can measure the distance and optionally also the relative speed to a concealing object. Thus, the size and / or position of the scanning area within the detection range of the imaging environmental detection sensor can be adapted to the distance and, if necessary, also the relative movement of the obscuring object.

Another object of the invention is a device for detecting soil symbols by means of an imaging environment detection sensor of a vehicle, which includes a correspondingly formed control and evaluation unit.

In the following, embodiments of the invention and FIG. Are described and explained.

Showing:

1 shows a situation in which a preceding ground symbol is partly hidden by a preceding vehicle, and FIG. 2 schematically shows the composition of a composite image of a ground area with a ground symbol of a plurality of scan areas taken from a sequence of images.

Fig. 1 shows an example of a situation that occurs frequently, especially in urban areas. The driver of a vehicle (1) can not recognize a forward ground symbol (3) because it is largely obscured (30) by a preceding vehicle (4). In particular, on one of several lanes in front of a traffic light system, it happens that the preceding vehicle (4) is partially over the ground symbol (3). Accordingly, a can in the direction of the environment of the (own) vehicle (1) detecting the imaging environment detecting sensor (2), eg a camera, the floor symbol is not completely he ^¬ grasp, or the ground symbol from a camera image are not recognized. The knowledge of the complete ground symbol (3) could be crucial for the driver of the vehicle (1) to know on a multi-lane road in front of an intersection whether he has located in the right lane, depending on whether the ground symbol (3) Left, right and / or straight arrow is.

The camera shown in FIG. 1 as an imaging Conversely ^¬ environment detection sensor (2) has the reverse side illustrated detection area (20). Since the camera is in the range of the windshield inside the own vehicle (1) is arranged reasonable, it can be ground until a line (21; ge ^¬ dashed lines shown with short dashes) capture. The bottom left of this line (21) in Fig. 1 covers the hood of the own vehicle (1). An embodiment of a method according to the invention can proceed as follows:

With a camera as an imaging environment detection sensor (2) pictures of the vehicle environment are taken.

If it is determined (3 without the ge ^¬ scores shown part 30) in an image by an image data processing and object detection systems a potential ground symbol but can not be detected, and a complete ground symbol possibly obscuring object or vehicle (4) is determined, a scanning process is started.

Thus, if the camera image data processing recognizes, as in FIG. 1, that there is a symbol (3) on the roadway that is partially hidden (30), the procedure may be as follows:

A scan area (22, shown in dashed lines with long dashes) is defined in the detection area (20) of the camera (2), which images a ground area in front of the own vehicle (1) which is not from an object or vehicle (4) located above it. is covered.

The size of the scanning area (22) and / or the position of the scanning area within the detection area (20) can be defined from the image data processing or object recognition such that the distance to the obscuring object (4), the ground detection line of the camera (21) expected width of the soil symbol (3) and / or other circumstances are taken into account.

The contents of the scan area (22; 22.1), ie the image data of the corresponding image area of the captured image of the vehicle environment, are temporarily stored.

A sequence of images is taken during the onward journey of the own vehicle (1) with the camera (2). From subsequently recorded images, the content of further scan areas (22, 22.2 to 22.6) is temporarily stored.

Advantageously, the further scan areas (22, 22.2 to 22.6) can then be examined as to whether an end piece of a potential floor symbol (3) is already contained therein, eg an arrowhead or a closing edge. If so, it is possible to start assembling a composite image from cached scan areas (22; 22.1 to 22.6). Fig. 2 shows diagrammatically the assembly of an image of the ground symbol, which was obscured in Fig. 1, six Zvi ^¬ rule stored scan regions (1.22 to 6.22).

From the composite image, the soil symbol (3) can be recognized or its regulatory content determined (e.g., classified).

Information regarding the detected ground symbol (3) is output. For example, the information can be output to a traffic light assistant that the own vehicle (1) is located on a straight-ahead track.

The scan or area (s) (22; 01/22 to 06/22) can be defined fol ^¬ gender extent:

a) A scan area (22; 22.1) defined from the first image can be kept constant in terms of size and position within the detection area (20) of the camera (2);

b) The size of the scan area (22; 22.2 to 22.6) can be varied in subsequent images;

c) Alternatively or cumulatively, the position of the scanning region (22; 22.2 to 22.6) within the detection region (20) of the camera (2) can be adapted in subsequent images.

In subsequently recorded images so the scan range (22; 22.6 to 22.2) are adapted to the optional Ge ^¬ speed and / or movement data (yaw / pitch / roll angle and rate) of the own vehicle (1). Advantageously, a Movement of the obscuring object (4) are taken into account, so that the scan area (22) contains an uncovered image of the ground area. Another aspect concerns the selection of images from the sequence of images whose scan area (22, 22.2 to 22.6) is to be used in the composition. The selection should advantageously take into account temporal intervals between the recording times of pictures of the picture sequence and the own speed of the own vehicle (1).

One possibility is that the scan areas (22, 22.1 to 22.6) are defined to be the same size in all images and are selected from images of the image sequence such that they have an exactly or approximately equal distance apart from each other that is smaller than the size of the image Scan area (22, 22.1 to 22.6) in this direction. An example is intended to illustrate this: if a picture is taken every 40 ms and the own vehicle's own speed (1) is constant at 36 km / h, the scan area (22, 22.1 to 22.6) shifts from one picture to the next in the direction of travel around 0.4m. If the scan area (22; 22.1 to 22.6) extends in the direction of travel of one meter, it would therefore be sufficient to buffer a scan area or its contents from every other picture in order to ensure an overlap of the scan areas of 0.2 m each.

Thus, it is not absolutely necessary to evaluate a scan area (22, 22.1 to 22.6) for each recorded image of the image sequence, this can be adapted as a function of the speed or intrinsic movement of the own vehicle (1) and the image acquisition rate of the camera (2) ,

This allows even in case of different time intervals between the recording of successive images and the changing own speed of the own vehicle (1) an adaptation of the selection of the (subsequent) images or scan areas (20; 20.2 to 20.6) from the image sequence. Taking into account the proper motion of the own vehicle (1), the size of the scan area (22), and the image acquisition rate, it can be predicted, for example, from which subsequently captured image of the image sequence the next scan area is evaluated.

Advantageously, the scan areas give (1.22 to 6.22) together ^¬ quantity sets or a complete seamless image of the un- covered floor area in which the floor symbol (3) is located. This is the case with the composite image of FIG. 2.

From the overlapping scanning regions (22, 22.1 to 22.6) of individual images of the image sequence, a composite image of the bottom region of the environment can thus be created. This is similar to the principle of Zei ^¬ steering using the camera.

The six images (22.1 to 22.6) of the scanning region (22) of the camera (2) shown in FIG. 2 could be recorded at a constant speed, for example when the vehicle is traveling further. By composing these images (1.22 to 6.22) taking into account the distance traveled by the own-vehicle distance between the images of the six images with these scanning ranges (1.22 to 6.22), a to-sammengesetztes image is determined, which contains the complete Bo ^¬ densymbol (3). Based on this composite image, the bottom symbol (3) can be recognized or interpreted as a straight arrow by means of an image processing, although the straight arrow was at no time detectable for the camera (2) as a whole. So even in the most difficult conditions (heavy traffic, low visibility) soil symbols can be detected. When defining or defining the size of the scan area (22), the speed of the own vehicle (2) can advantageously be taken into account, since otherwise higher velocities could possibly result in gaps in the composite image. A coupling with an object recognition or with a radar sensor could also be advantageous at this point in order to be able to measure the distance to a preceding vehicle (4) and to be able to adapt the scan area to it. One option for defining the scan area is thus to vary its size as a function of the speed and distance to preceding vehicles. Finally, the scanning area (22) in each image should preferably be free of obscurations by the preceding vehicle (4). Alternatively, in the case of a (partially) autonomously driving own vehicle (1), the speed can be limited such that the own vehicle (1) does not move faster than a preceding vehicle (4). This is similar to a Stop & Go distance control (ACC) already feasible today.

The accumulation of the images or of the scan areas (22.1 to 22.6) can be carried out in particular in a world-wide map or in a vehicle-mounted map in order to take into account the imaging properties of the camera (2) or to counteract the perspective distortion.

Claims

Claims / Patent Claims

A method of detecting ground symbols (3) by means of an imaging environment detection sensor (2) of a vehicle (1), comprising the steps of:

a sequence of images is taken during a movement of the vehicle (1);

from scanning areas (22; 22.1 to 22.6) of pictures from the sequence of pictures, a composite picture of a floor area of the surroundings is composed;

a recognition symbol of the bottom (3) is carried out from the quantitative together ^¬ translated image of the bottom portion is effected; and

Information regarding the detected ground symbol (3) is output.

2. The method of claim 1, wherein the method is used only in partial occlusion of a soil symbol (3).

Be ^¬ defined sungsbereichs (20) of the environment detection sensor (2) in such a way within the Erfas that no occlusion within a scanning area (22; 3. The method according to claim 1 or 2, wherein size and / or position of the scan range (1.22 to 6.22 22); 22.1 to 22.6) occurs.

4. The method according to any one of the preceding claims, wherein the size of the scan area (22; 22.1 to 22.6) depends on the Geschwin ^¬ speed of the own vehicle (1).

5. The method according to any one of the preceding claims, wherein a further sensor detecting surroundings sensor, in particular a radar / lidar sensor, the distance and, optionally, the Real ^¬ tivgeschwindigkeit measures a concealing object.

6. The method according to any one of the preceding claims, wherein images with scanning fields (22; 01/22 to 06/22) of the image sequence will be selected such that from the scan areas (22; 01/22 to 06/22), a seamless image of the bottom portion with the Bo ^¬ densymbol (3) can be composed.

7. The method according to any one of the preceding claims, wherein the imaging environment detection sensor (2) is a monocamera.

8. The method of claim 1, wherein the imaging environment detection sensor is a stereo camera.

9. The method according to any one of the preceding claims, wherein the imaging environment detection sensor (2) is at least one camera of a panoramic sensor system.

10. The method according to any one of the preceding claims, wherein the imaging environment detection sensor (2) is an imaging laser sensor system.

11. A device for detecting ground symbols (3) by means of an imaging environment detection sensor (2) of a vehicle (1), comprising a control and evaluation unit which is adapted to carry out the following steps:

a sequence of images is taken during a movement of the vehicle (1);

a recognition of the floor symbol (3) is made from the quantity together ^¬ translated image of the bottom portion is effected; and

the detected ground symbol (3) is output.