EP3276588B1

EP3276588B1 - Cross traffic detection system for a vehicle

Info

Publication number: EP3276588B1
Application number: EP16181582.4A
Authority: EP
Inventors: Bhanu Prakash Padiri
Original assignee: Continental Autonomous Mobility Germany GmbH
Current assignee: Continental Autonomous Mobility Germany GmbH
Priority date: 2016-07-28
Filing date: 2016-07-28
Publication date: 2024-01-10
Anticipated expiration: 2036-07-28
Also published as: EP3276588A1

Description

Field of the invention

The invention generally relates to assistance systems for vehicles. Particularly, the invention relates to a cross traffic detection system for a vehicle, a vehicle with such system and a method for detecting cross traffic.

Background of the invention

Cross traffic detection systems and/or cross traffic alert systems, which are based on optically detecting crossing road users, such as vehicles, motorbikes or pedestrians, usually comprise a camera and/or a radar device. Therein, the road users are passively or directly illuminated in order to detect them. Accordingly, during night time, limited visibility and/or if a view to the road user is blocked by an obstacle, an efficiency of such cross traffic systems may be reduced and/or limited. This may particularly apply to urban areas, cross junctions and/or intersections, where a driver of a vehicle may be surrounded by buildings and thus may be unaware of any cross traffic until he closely approaches the road user.
US2014/0009617A1 refers to an environment recognition system able to recognize vehicles running behind one own's vehicle independent on weather or light conditions based on the detection of light spots. Similar systems for vehicle control are disclosed in EP2023307A2 , US20070221822A1 and EP2523174A1 . Furthermore EP1334871A2 refers to a vehicle safety system utilizing a set of beam radiators and imaging devices for determining whether a subject vehicle is in a state where attention is required during running. This system is adapted to detect at an intersection the approach of a target vehicle moving in a transverse direction with respect to a moving direction of the subject vehicle. The target vehicle radiates, with a beam radiator, a visible or invisible light beam on a road surface. An imaging device is provided on the host or subject vehicle, with an effective imaging region for detecting the reflection on the road surface of the light beam radiated by the target vehicle. When the subject vehicle is itself equipped with a beam radiator, the configuration may be such that both the subject pattern formed by the visible light beam from the subject vehicle and the other pattern formed by the visible light or the invisible light beam from the target vehicle are extracted; and then, on the basis of the relation between the subject pattern and the other pattern, it is determined whether the subject vehicle is in a state where attention is required. When the subject vehicle itself does not have the beam radiators for radiating the light beams, the other patterns from the target vehicle or the like are obtained using the imaging device, and a state where attention is required in the subject vehicle is determined by comparing the obtained other patterns with patterns determined based on the presumed running trajectory of the subject vehicle or the like.

Description of the invention

It is an objective of the present invention to provide a comprehensive cross traffic detection system, which allows to reliably determine and/or detect other road users.
This objective is achieved by the subject matter of the independent claims. Further exemplary embodiments are evident from the dependent claims and the following description.
A first aspect of the invention relates to a cross traffic detection system for a vehicle. The system comprises a camera arrangement with at least one camera, a data storage device and a controller. The system is configured for capturing, by means of the camera arrangement, an image sequence comprising consecutive images of an environment and/or of surroundings of the vehicle. The system is further configured for storing the images of the image sequence in the data storage device. Therein, the controller is configured for determining in at least two of the images an image section, in which a light cone emitted by an illumination device of a road user moving in a transverse direction with respect to a moving direction of the vehicle is captured. The road user is a traffic participant and the illumination device is a spotlight, a head lamp, a head light or a reflector. Moreover, the controller is configured to determine whether the vehicle and the road user moving in the transverse direction with respect to a moving direction of the vehicle approach each other by comparing the determined image sections.
The cross traffic detection system may generally be configured for being employed in a vehicle, wherein the vehicle may be a motor vehicle like a car, a bus or a truck, or a bicycle like a bike or a motor cycle.
The camera arrangement may comprise one or more cameras. Particularly, the camera arrangement may comprise a front camera arranged in a front part of a vehicle. The camera arrangement may also comprise a plurality of cameras arranged around a vehicle and providing a surround view, e.g. 360° view, of the vehicle. The camera arrangement may be configured to capture images in the optical wavelength range and/or in the infrared range.
The controller may be or comprise a processing unit, a processor, a control module or a control unit configured for processing data, particularly image data. Accordingly, the controller may be configured for image processing and/or image data processing.
The road user is any kind of traffic participant, such as e.g. a vehicle and/or a pedestrian. Particularly, the road user moves in a transverse direction with respect to a moving direction of a vehicle, which comprises the cross traffic detection system.
The data storage device may be any type of data storage configured for storing and/or reading data.
Rephrasing the first aspect, the cross traffic detection system may be configured to capture and/or acquire an image sequence of timely consecutive images of the environment and/or of surroundings of the vehicle within a field of view of the camera. The image sequence may comprise at least two images, which may be captured in an arbitrary time difference with respect to each other. These images and/or corresponding image data may then be stored in the data storage device. By analyzing and/or processing at least a part of the images and/or corresponding image data, the controller may determine and/or identify the image section, in which the light cone of the illumination device of the road user is captured.
According to the present invention, an image section refers to a subset of image data of an image, in which a representation and/or a projection of the light cone is comprised. Further, , the light cone refers to a region of the camera's field of view (or a two-dimensional representation thereof), which region is illuminated by the illumination device. The illumination device in turn refers to a spot light, a head lamp, a head light, or a reflector. The controller may then compare the image sections, which contain the light cones, of consecutive images with each other and may determine whether the vehicle and the road user approach each other based on that comparison.
Accordingly, the cross traffic detection system may advantageously provide a system to detect cross traffic in scenarios of limited visibility and especially during night times. By means of the system, accidents may be avoided, a decision making time of a driver of the vehicle may be reduced and hence safety for the vehicle, the driver and the road user may be increased.
Summarizing, the cross traffic detection system may be capable of reliably and comprehensively determine and/or detect the road user in situations and/or scenarios, in which state-of-the-art Radar based and/or camera based systems may not find the road user as there may be obstacles, such as buildings, trees, cars or the like, blocking a direct view to the road user. By detecting the road user indirectly via the light cone emitted by the illumination device, the cross traffic detection system may reliably detect the road user. Thus, a driver of the vehicle, which is equipped with the cross traffic detection system, may be made aware when approaching the road user, although there may be no direct visibility of the road user. Thus, the cross traffic detection system may be particularly applicable at night conditions when illumination devices are used by road users. However, the cross traffic detection system may also be advantageously used during day time.
Further, the cross traffic detection system may be utilized with any front facing camera. According to a preferred embodiment, while reversing the vehicle, a rear camera may also be utilized to detect the road user in opposite direction. In that way the cross traffic detection system may be assisting the driver while reversing out of a parking space, i.e. in a situation where crossing traffic is hard to see for the driver of the vehicle. Moreover, the camera arrangement may comprise cameras arranged on all sides of the vehicle, which may provide a surround view of about 360°. Generally, the cross traffic detection system may be retrofit to vehicles already comprising at least one camera.
According to an embodiment, the controller is configured to determine the image section in a part, e.g. a lower part of the image, in which part a road and/or a roadway is captured.
According to the invention, the controller is configured for determining a reflection of the light cone on a road surface in the at least two image sections, wherein the controller is configured to determine whether the vehicle and the road user approach each other taking into account the determined reflections. By taking reflections of the light cone into account, an accuracy and robustness of the system may be enhanced. Particularly during rainy conditions, e.g. rainy season, a performance of the system may thus be enhanced. Further, by taking the reflections into account the system may facilitate differentiation of reflections on the road surface caused by static or fixed lights, such as street lamps or signboards, from the light cone of the road user, because there may be no movement of the fixed lights in the reflections.
According to an embodiment, the controller is configured for determining a movement of the light cone and/or a color change by comparing the at least two images and/or by comparing the determined image sections of the at least two images. This may further allow to differentiate more accurately and more reliably between fixed lights and the light cone of the road user. For this purpose, various classification techniques and/or optical flow methods may be utilized. Moreover, by detecting the movement of the light cone, e.g. caused by front lights focused on the road by the road user, the cross traffic detection system may differentiate more reliably between the light cone and ambient light because the light cone may be in a lower part of the image whereas ambient light may be everywhere.
Further, the cross traffic detection system may detect and/or determine a change of red light of a vehicle at a cross road and/or a junction, while this vehicle may not be directly visible. This may allow the system to determine and/or detect that another vehicle just stopped. Apart from that, as the other vehicle may be moving, the system may detect and/or determine small variations of light and/or a velocity of the vehicle.
According to an embodiment, the system further comprises a warning device for outputting a warning signal, wherein the controller is configured for triggering the warning device upon determining that the vehicle and the road user approach each other. The warning signal may be an acoustic, an optical and/or a haptic warning signal.
The warning signal may be generated and/or output e.g. on a display screen as a symbol comprising a set of visualized images and/or a video, optionally along with an overlay. A surround view camera may generate a view with 360 degree images and/or a real time video around the vehicle. Additionally to a video, the controller may generate an overlay with a further symbol, such as e.g. an arrow indicating a direction, in which the cross traffic is moving, along with a color based symbol, e.g. indicating a distance and/or a speed of the road user, the cross traffic and/or the vehicle. For instance, a red symbol may indicate that the vehicle is nearby and/or moving rather fast. Further, an orange and/or green symbol may indicate that the vehicle is far away and/or moving rather slowly.
By means of the warning signal it may be brought to the attention of the driver of the vehicle that he is approaching the road user. Thus, this may allow to reduce a speed of the vehicle and/or to actuate a brake system of the vehicle. Thus, accidents may be reduced and safety may be increased.
According to an embodiment, the warning signal is indicative of a type of the road user. By way of example there may be a specific color of the warning signal indicating that the road user is e.g. a vehicle, a pedestrian or a bicycle. Also a size of an optical warning signal may indicate the type of road user. Further, different acoustic signals may be used for different types of road users.
A second aspect of the invention relates to a vehicle comprising a cross traffic detection system as described above and in the following.
It has to be understood that features of the cross traffic detection system may be features of the vehicle and vice versa.
A third aspect of the invention relates to a method for detecting and/or determining cross traffic. The method comprises the following steps:

capturing, by means of a camera arrangement of a vehicle, an image sequence comprising consecutive images of an environment of the vehicle;
storing the images of the image sequence in a data storage device;
determining in at least two of the images an image section, in which a light cone emitted by an illumination device of a road user moving in a transverse direction with respect to a moving direction of the vehicle is captured; wherein the road user is a traffic participant; and wherein the illumination device is a spotlight, a head lamp, a head light or a reflector;

wherein the image section refers to a subset of image data of the image, in which a representation and/or projection of the light cone is comprised;
wherein the light cone refers to a region of the camera's field of view, which region is illuminated by the illumination device; and
determining a reflection of the light cone on a road surface in the at least two image sections, and determining whether the vehicle and the road user approach each other by comparing the determined image sections.

The present invention is as set out in the appended claims.

Brief description of the drawings

Fig. 1 shows schematically a cross traffic detection system according to embodiment.
Fig. 2 shows schematically a vehicle with the cross traffic detection system of Fig. 1.
Fig. 3 shows a flowchart illustrating steps of a method for detecting cross traffic according to an embodiment.

In principle, identical parts are provided with the same reference symbols in the figures.

Detailed description of exemplary embodiments

Fig. 1 shows schematically a cross traffic detection system 10 for a vehicle 100 according to an embodiment. Fig. 2 shows schematically a vehicle 100 with the cross traffic detection system 10 of Fig. 1.
The cross traffic detection system 10 comprises a camera arrangement 12 with at least one camera 14, a data storage device 16 and a controller 18.
The camera 14 may be an optical and/or an infrared camera. The camera 14 may be a front camera of the vehicle 100 as shown in Fig. 2. Alternatively, the camera 14 may be e.g. a rear camera or any other camera. The camera arrangement 12 may also comprise a plurality of cameras 14 arranged on vehicle 100. This way, the camera arrangement 12 may provide surround view.
The cross traffic detection system 10 is configured to capture and/or acquire, by means of the camera arrangement 12 and/or by means of the camera 14, an image sequence comprising consecutive images of an environment of the vehicle 100. The image sequence may comprise at least two images. Further, the cross traffic detection system 10 is configured to store the images of the image sequence and/or respective image data in the data storage device 16. Therein, the environment and/or the surroundings of the vehicle 100 is captured within a field of view 102 of the camera 14. As illustrated in Fig. 2 this field of view 102 may be limited and/or a part of the field of view 102 may be blocked by an obstacle 101, such as e.g. a building, a vehicle, trees, or the like. This may particularly apply to urban areas and/or a cross junction 104 and/or an intersection 104 as illustrated in Fig. 2. A road user 106 such as the further vehicle 106 shown in Fig. 2 may thus not be visible to a driver of the vehicle 100 and/or to the camera 14.
However, the road user 106 and/or the vehicle 106 comprises an illumination device 108, which generates a light cone 110 extending into the field of view 102 of the camera 14.
The controller 18 of the cross traffic detection system 10 is configured for determining in at least two of the images of the image sequence an image section, in which the light cone 110 emitted by the illumination device 108 of the road user 106 is captured. Based on a comparison of these determined image sections in the consecutive images, the controller 18 is capable to determine whether the vehicle 100 and the road user 106 approach each other.
The controller 18 is configured to determine the image sections only in a lower part of the captured images, in which lower part a road 103 is captured. This may accelerate the process of detecting the road user 106. Further, the controller 18 according to the present invention is configured for determining a reflection of the light cone 110 on a road surface 105. Generally, this may allow to distinct and/or differentiate the light cone 110 from other static and/or fixed lights, such as street lamps.
Apart from that, the controller 18 is configured for determining a movement of the light cone 110 and/or a color change by comparing the at least two images and/or by comparing the determined image sections of the at least two images.
The cross traffic detection system 10 further comprises a warning device 20 for outputting a warning signal. The warning signal may be an optical, an acoustic and/or a haptic signal, which may alert the driver of the vehicle 100 and/or which may bring to the attention of the driver that he is approaching the road user 106. The warning signal and/or the warning device 20 is triggered by the controller 18, if the controller 18 has determined that the vehicle 100 and the road user 106 approach each other.
Apart from that, the controller 18 may be configured to actuate a breaking system 112 of the vehicle 100 in order to reduce a speed of the vehicle 100 and to avoid a potential accident with the road user 106.
Summarizing, the cross traffic detection system 10 may alert the driver and hence may prevent accidents, especially at night time. Also, collisions and accidents may be avoided even during blocking scenarios, in which a field of view 102 is limited and/or partly blocked by an obstacle 101. The cross traffic detection system 10 may be easily and cost efficiently retrofit to the vehicle 100 without a need for extra sensors. Due to a significant color of the front light of the cross vehicle 106 and/or the road user 106 and its reflection on the street surface 105 and/or road surface 105 it may easily be distinguished from ambient light, which in turn may increase reliability and accuracy of the cross traffic detection system 10.
Fig. 3 shows a flowchart illustrating steps of a method for detecting cross traffic according to an embodiment.
In a step S1 an image sequence comprising consecutive images of an environment of a vehicle 100 is captured by means of a camera arrangement 12 of the vehicle 100. In a further step S2 the images of the image sequence are stored in a data storage device 16. In a further step S3 in at least two of the images an image section is determined, in which image section a light cone 110 emitted by an illumination device 108 of a road user 106 is captured. Finally, in a step S4 the determined image sections are compared with each other by means of a controller 18 of the cross traffic detection system 10 and it is determined whether the vehicle 10 and the road user 106 approach each other.
While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary. Any reference signs in the claims should not be construed as limiting the scope.

Claims

A cross traffic detection system (10) for a vehicle (100), the system (10) comprising:
a camera arrangement (12) with at least one camera (14);

a data storage device (16); and

a controller (18);

wherein the system (10) is configured for capturing, by means of the camera arrangement (12), an image sequence comprising consecutive images of an environment of the vehicle (100);

wherein the system (10) is configured for storing the images of the image sequence in the data storage device (16);

wherein the controller (18) is configured for determining in at least two of the images an image section, in which a light cone (110) emitted by an illumination device (108) of a road user (106) moving in a transverse direction with respect to a moving direction of the vehicle is captured; wherein the road user is a traffic participant; and wherein the illumination device is a spotlight, a head lamp, a head light or a reflector;

wherein the image section refers to a subset of image data of the image, in which a representation and/or projection of the light cone is comprised;

wherein the light cone refers to a region of the camera's field of view, which region is illuminated by the illumination device; and

wherein the controller (18) is configured to determine a reflection of the light cone on a road surface in the at least two image sections, and wherein the controller is configured to determine whether the vehicle (100) and the road user (106) moving in the transverse direction with respect to a moving direction of the vehicle approach each other by comparing the determined image sections in the consecutive images and taking into account the determined reflections.
The system (10) according to one of the preceding claims,
wherein the controller (18) is configured to determine the image section in a part of the image, in which part a road (103) is captured.
The system (10) according to one of the preceding claims,
wherein the controller (18) is configured for determining a movement of the light cone (110) and/or a color change by comparing the at least two images.
The system (10) according to one of the preceding claims, further comprising:
a warning device (20) for outputting a warning signal;

wherein the controller (18) is configured for triggering the warning device (20) upon determining that the vehicle (100) and the road user (106) approach each other.
The system (10) according to claim 4,
wherein the warning signal is indicative of a type of the road user (106).
A vehicle (100) comprising a cross traffic detection system (10) according to one of the preceding claims.
A method for detecting cross traffic, the method comprising
the steps of:
capturing, by means of a camera arrangement (14) of a vehicle (100), an image sequence comprising consecutive images of an environment of the vehicle (100);

storing the images of the image sequence in a data storage device (16);

determining in at least two of the images an image section, in which a light cone (110) emitted by an illumination device (108) of a road user (106) moving in a transverse direction with respect to a moving direction of the vehicle is captured; wherein the road user is a traffic participant; and wherein the illumination device is a spotlight, a head lamp, a head light or a reflector;

wherein the image section refers to a subset of image data of the image, in which a representation and/or projection of the light cone is comprised;

wherein the light cone refers to a region of the camera's field of view, which region is illuminated by the illumination device; and

determining a reflection of the light cone on a road surface in the at least two image sections, and determining whether the vehicle (100) and the road user (106) moving in the transverse direction with respect to a moving direction of the vehicle approach each other by comparing the determined image sections in the consecutive images and taking into account the determined reflections.