EP3276588A1

EP3276588A1 - Cross traffic detection system for a vehicle

Info

Publication number: EP3276588A1
Application number: EP16181582.4A
Authority: EP
Inventors: Bhanu Prakash Padiri
Original assignee: Continental Automotive GmbH
Current assignee: Continental Autonomous Mobility Germany GmbH
Priority date: 2016-07-28
Filing date: 2016-07-28
Publication date: 2018-01-31
Anticipated expiration: 2036-07-28
Also published as: EP3276588B1

Abstract

A Cross traffic detection system (10) for a vehicle (100) is provided, which comprises a camera arrangement (12) with at least one camera (14), a data storage device (16) and a controller (18). The system (10) is configured for capturing an image sequence with consecutive images of an environment of the vehicle (100) and for storing the images of the image sequence in the data storage device (16). The controller (18) is configured for determining in at least two of the images an image section, in which a light cone (110) emitted by an illumination device (108) of a road user (106) is captured, wherein the controller (18) is configured to determine whether the vehicle (100) and the road user (106) approach each other by comparing the determined image sections.

Description

Field of the invention

The invention generally relates to assistance systems for vehicles. Particularly, the invention relates to a cross traffic detection system for a vehicle, a vehicle with such system and a method for detecting cross traffic.

Background of the invention

Cross traffic detection systems and/or cross traffic alert systems, which are based on optically detecting crossing road users, such as vehicles, motorbikes or pedestrians, usually comprise a camera and/or a radar device. Therein, the road users are passively or directly illuminated in order to detect them. Accordingly, during night time, limited visibility and/or if a view to the road user is blocked by an obstacle, an efficiency of such cross traffic systems may be reduced and/or limited. This may particularly apply to urban areas, cross junctions and/or intersections, where a driver of a vehicle may be surrounded by buildings and thus may be unaware of any cross traffic until he closely approaches the road user.

Description of the invention

It is an objective of the present invention to provide a comprehensive cross traffic detection system, which allows to reliably determine and/or detect other road users.
This objective is achieved by the subject matter of the independent claims. Further exemplary embodiments are evident from the dependent claims and the following description.
A first aspect of the invention relates to a cross traffic detection system for a vehicle. The system comprises a camera arrangement with at least one camera, a data storage device and a controller. The system is configured for capturing, by means of the camera arrangement, an image sequence comprising consecutive images of an environment and/or of surroundings of the vehicle. The system is further configured for storing the images of the image sequence in the data storage device. Therein, the controller is configured for determining in at least two of the images an image section, in which a light cone emitted by an illumination device of a road user is captured. Moreover, the controller is configured to determine whether the vehicle and the road user approach each other by comparing the determined image sections.
The cross traffic detection system may generally be configured for being employed in a vehicle, wherein the vehicle may be a motor vehicle like a car, a bus or a truck, or a bicycle like a bike or a motor cycle.
The camera arrangement may comprise one or more cameras. Particularly, the camera arrangement may comprise a front camera arranged in a front part of a vehicle. The camera arrangement may also comprise a plurality of cameras arranged around a vehicle and providing a surround view, e.g. 360° view, of the vehicle. The camera arrangement may be configured to capture images in the optical wavelength range and/or in the infrared range.
The controller may be or comprise a processing unit, a processor, a control module or a control unit configured for processing data, particularly image data. Accordingly, the controller may be configured for image processing and/or image data processing. The road user may be any kind of traffic participant, such as e.g. a vehicle and/or a pedestrian. Particularly, the road user may move in a transverse direction with respect to a moving direction of a vehicle, which comprises the cross traffic detection system.
The data storage device may be any type of data storage configured for storing and/or reading data.
Rephrasing the first aspect, the cross traffic detection system may be configured to capture and/or acquire an image sequence of timely consecutive images of the environment and/or of surroundings of the vehicle within a field of view of the camera. The image sequence may comprise at least two images, which may be captured in an arbitrary time difference with respect to each other. These images and/or corresponding image data may then be stored in the data storage device. By analyzing and/or processing at least a part of the images and/or corresponding image data, the controller may determine and/or identify the image section, in which the light cone of the illumination device of the road user is captured. Accordingly, an image section may refer to a subset of image data of an image, in which a representation and/or a projection of the light cone is comprised. Generally, the light cone may refer to a region of the camera' s field of view (or a two-dimensional representation thereof), which region is illuminated by the illumination device, wherein the illumination device may refer to a spot light, a head lamp, a head light, a reflector or the like. The controller may then compare the image sections, which contain the light cones, of consecutive images with each other and may determine whether the vehicle and the road user approach each other based on that comparison.
Accordingly, the cross traffic detection system may advantageously provide a system to detect cross traffic in scenarios of limited visibility and especially during night times. By means of the system, accidents may be avoided, a decision making time of a driver of the vehicle may be reduced and hence safety for the vehicle, the driver and the road user may be increased.
Summarizing, the cross traffic detection system may be capable of reliably and comprehensively determine and/or detect the road user in situations and/or scenarios, in which state-of-the-art Radar based and/or camera based systems may not find the road user as there may be obstacles, such as buildings, trees, cars or the like, blocking a direct view to the road user. By detecting the road user indirectly via the light cone emitted by the illumination device, the cross traffic detection system may reliably detect the road user. Thus, a driver of the vehicle, which is equipped with the cross traffic detection system, may be made aware when approaching the road user, although there may be no direct visibility of the road user. Thus, the cross traffic detection system may be particularly applicable at night conditions when illumination devices are used by road users. However, the cross traffic detection system may also be advantageously used during day time.
Further, the cross traffic detection system may be utilized with any front facing camera. According to a preferred embodiment, while reversing the vehicle, a rear camera may also be utilized to detect the road user in opposite direction. In that way the cross traffic detection system may be assisting the driver while reversing out of a parking space, i.e. in a situation where crossing traffic is hard to see for the driver of the vehicle. Moreover, the camera arrangement may comprise cameras arranged on all sides of the vehicle, which may provide a surround view of about 360°. Generally, the cross traffic detection system may be retrofit to vehicles already comprising at least one camera.
According to an embodiment, the controller is configured for determining a light intensity of the light cones in each of the determined image sections, wherein the controller is configured to determine whether the vehicle and the road user approach each other based on a comparison of the determined light intensities of the light cones. For instance, if the light intensity is higher in an image with respect to an image captured earlier, the cross traffic detection system may determine with high accuracy that the vehicle and the road user approach each other.
According to an embodiment, the controller is configured for determining a size of the light cones in each of the determined image sections, wherein the controller is configured to determine whether the vehicle and the road user approach each other based on a comparison of the determined sizes of the light cones. For instance, if the size is larger in an image with respect to an image captured earlier, the cross traffic detection system may determine with high accuracy that the vehicle and the road user approach each other. The size of the light cones may refer to a real size of the light cones, which may be estimated by means of the controller. Alternatively or additionally, the size of the light cone may be determined relative to a size of the respective image and/or the image section.
According to an embodiment, the controller is configured to determine the image section in a part, e.g. a lower part of the image, in which part a road and/or a roadway is captured. The controller may be configured to determine the image section exclusively in the part of the image, in which the road user is expected. The rest of the image and/or the remaining image data may be disregarded. This may accelerate the detection of the road user.
According to an embodiment, the controller is configured for determining a reflection of the light cone on a road surface in the at least two image sections, wherein the controller is configured to determine whether the vehicle and the road user approach each other taking into account the determined reflections. By taking reflections of the light cone into account, an accuracy and robustness of the system may be enhanced. Particularly during rainy conditions, e.g. rainy season, a performance of the system may thus be enhanced. Further, by taking the reflections into account the system may facilitate differentiation of reflections on the road surface caused by static or fixed lights, such as street lamps or signboards, from the light cone of the road user, because there may be no movement of the fixed lights in the reflections.
According to an embodiment, the controller is configured for determining a movement of the light cone and/or a color change by comparing the at least two images and/or by comparing the determined image sections of the at least two images. This may further allow to differentiate more accurately and more reliably between fixed lights and the light cone of the road user. For this purpose, various classification techniques and/or optical flow methods may be utilized. Moreover, by detecting the movement of the light cone, e.g. caused by front lights focused on the road by the road user, the cross traffic detection system may differentiate more reliably between the light cone and ambient light because the light cone may be in a lower part of the image whereas ambient light may be everywhere.
Further, the cross traffic detection system may detect and/or determine a change of red light of a vehicle at a cross road and/or a junction, while this vehicle may not be directly visible. This may allow the system to determine and/or detect that another vehicle just stopped. Apart from that, as the other vehicle may be moving, the system may detect and/or determine small variations of light and/or a velocity of the vehicle.
According to an embodiment, the system further comprises a warning device for outputting a warning signal, wherein the controller is configured for triggering the warning device upon determining that the vehicle and the road user approach each other. The warning signal may be an acoustic, an optical and/or a haptic warning signal.
The warning signal may be generated and/or output e.g. on a display screen as a symbol comprising a set of visualized images and/or a video, optionally along with an overlay. A surround view camera may generate a view with 360 degree images and/or a real time video around the vehicle. Additionally to a video, the controller may generate an overlay with a further symbol, such as e.g. an arrow indicating a direction, in which the cross traffic is moving, along with a color based symbol, e.g. indicating a distance and/or a speed of the road user, the cross traffic and/or the vehicle. For instance, a red symbol may indicate that the vehicle is nearby and/or moving rather fast. Further, an orange and/or green symbol may indicate that the vehicle is far away and/or moving rather slowly.
By means of the warning signal it may be brought to the attention of the driver of the vehicle that he is approaching the road user. Thus, this may allow to reduce a speed of the vehicle and/or to actuate a brake system of the vehicle. Thus, accidents may be reduced and safety may be increased.
According to an embodiment, the warning signal is indicative of a type of the road user. By way of example there may be a specific color of the warning signal indicating that the road user is e.g. a vehicle, a pedestrian or a bicycle. Also a size of an optical warning signal may indicate the type of road user. Further, different acoustic signals may be used for different types of road users.
A second aspect of the invention relates to a vehicle comprising a cross traffic detection system as described above and in the following.
It has to be understood that features of the cross traffic detection system may be features of the vehicle and vice versa.
A third aspect of the invention relates to a method for detecting and/or determining cross traffic. The method comprises the following steps:

capturing, by means of a camera arrangement of a vehicle, an image sequence comprising consecutive images of an environment of the vehicle;
storing the images of the image sequence in a data storage device;
determining in at least two of the images an image section, in which a light cone emitted by an illumination device of a road user is captured; and
determining whether the vehicle and the road user approach each other by comparing the determined image sections.

It has to be understood that features of the cross traffic detection system and the vehicle may be features of the method and vice versa.
These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

Brief description of the drawings

Fig. 1 shows schematically a cross traffic detection system according to embodiment.
Fig. 2 shows schematically a vehicle with the cross traffic detection system of Fig. 1.
Fig. 3 shows a flowchart illustrating steps of a method for detecting cross traffic according to an embodiment.

In principle, identical parts are provided with the same reference symbols in the figures.

Detailed description of exemplary embodiments

Fig. 1 shows schematically a cross traffic detection system 10 for a vehicle 100 according to an embodiment. Fig. 2 shows schematically a vehicle 100 with the cross traffic detection system 10 of Fig. 1.
The cross traffic detection system 10 comprises a camera arrangement 12 with at least one camera 14, a data storage device 16 and a controller 18.
The camera 14 may be an optical and/or an infrared camera. The camera 14 may be a front camera of the vehicle 100 as shown in Fig. 2. Alternatively, the camera 14 may be e.g. a rear camera or any other camera. The camera arrangement 12 may also comprise a plurality of cameras 14 arranged on vehicle 100. This way, the camera arrangement 12 may provide surround view.
The cross traffic detection system 10 is configured to capture and/or acquire, by means of the camera arrangement 12 and/or by means of the camera 14, an image sequence comprising consecutive images of an environment of the vehicle 100. The image sequence may comprise at least two images. Further, the cross traffic detection system 10 is configured to store the images of the image sequence and/or respective image data in the data storage device 16. Therein, the environment and/or the surroundings of the vehicle 100 is captured within a field of view 102 of the camera 14. As illustrated in Fig. 2 this field of view 102 may be limited and/or a part of the field of view 102 may be blocked by an obstacle 101, such as e.g. a building, a vehicle, trees, or the like. This may particularly apply to urban areas and/or a cross junction 104 and/or an intersection 104 as illustrated in Fig. 2. A road user 106 such as the further vehicle 106 shown in Fig. 2 may thus not be visible to a driver of the vehicle 100 and/or to the camera 14.
However, the road user 106 and/or the vehicle 106 comprises an illumination device 108, which generates a light cone 110 extending into the field of view 102 of the camera 14.
The controller 18 of the cross traffic detection system 10 is configured for determining in at least two of the images of the image sequence an image section, in which the light cone 110 emitted by the illumination device 108 of the road user 106 is captured. Based on a comparison of these determined image sections in the consecutive images, the controller 18 is capable to determine whether the vehicle 100 and the road user 106 approach each other.
For this purpose, the controller 18 is configured to determine a light intensity of the light cones 110 in each of the determined image sections and to compare the light intensities in order to determine whether the vehicle 100 and the road user 106 approach each other. If the intensity of the light cone 110 increases in an image section of an image captured later than another one, the vehicle 100 and the road user 106 approach each other.
Further, the controller 18 is configured for determining a size of the light cones 110 in each of the determined image sections and to compare the sizes of the light cones 110 in order to determine whether the vehicle and the road user 106 approach each other.
Moreover, the controller 18 is configured to determine the image sections only in a lower part of the captured images, in which lower part a road 103 is captured. This may accelerate the process of detecting the road user 106. Further, the controller 18 is configured for determining a reflection of the light cone 110 on a road surface 105. Generally, this may allow to distinct and/or differentiate the light cone 110 from other static and/or fixed lights, such as street lamps.
Apart from that, the controller 18 is configured for determining a movement of the light cone 110 and/or a color change by comparing the at least two images and/or by comparing the determined image sections of the at least two images.
The cross traffic detection system 10 further comprises a warning device 20 for outputting a warning signal. The warning signal may be an optical, an acoustic and/or a haptic signal, which may alert the driver of the vehicle 100 and/or which may bring to the attention of the driver that he is approaching the road user 106. The warning signal and/or the warning device 20 is triggered by the controller 18, if the controller 18 has determined that the vehicle 100 and the road user 106 approach each other.
Apart from that, the controller 18 may be configured to actuate a breaking system 112 of the vehicle 100 in order to reduce a speed of the vehicle 100 and to avoid a potential accident with the road user 106.
Summarizing, the cross traffic detection system 10 may alert the driver and hence may prevent accidents, especially at night time. Also, collisions and accidents may be avoided even during blocking scenarios, in which a field of view 102 is limited and/or partly blocked by an obstacle 101. The cross traffic detection system 10 may be easily and cost efficiently retrofit to the vehicle 100 without a need for extra sensors. Due to a significant color of the front light of the cross vehicle 106 and/or the road user 106 and its reflection on the street surface 105 and/or road surface 105 it may easily be distinguished from ambient light, which in turn may increase reliability and accuracy of the cross traffic detection system 10.
Fig. 3 shows a flowchart illustrating steps of a method for detecting cross traffic according to an embodiment.
In a step S1 an image sequence comprising consecutive images of an environment of a vehicle 100 is captured by means of a camera arrangement 12 of the vehicle 100. In a further step S2 the images of the image sequence are stored in a data storage device 16. In a further step S3 in at least two of the images an image section is determined, in which image section a light cone 110 emitted by an illumination device 108 of a road user 106 is captured. Finally, in a step S4 the determined image sections are compared with each other by means of a controller 18 of the cross traffic detection system 10 and it is determined whether the vehicle 10 and the road user 106 approach each other.
While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art and practising the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.

Claims

A Cross traffic detection system (10) for a vehicle (100), the system (10) comprising:
a camera arrangement (12) with at least one camera (14);

a data storage device (16); and

a controller (18);

wherein the system (10) is configured for capturing, by means of the camera arrangement (12), an image sequence comprising consecutive images of an environment of the vehicle (100) ;

wherein the system (10) is configured for storing the images of the image sequence in the data storage device (16);

wherein the controller (18) is configured for determining in at least two of the images an image section, in which a light cone (110) emitted by an illumination device (108) of a road user (106) is captured; and

wherein the controller (18) is configured to determine whether the vehicle (100) and the road user (106) approach each other by comparing the determined image sections.
The system10v according to claim 1,
wherein the controller (18) is configured for determining a light intensity of the light cones (110) in each of the determined image sections; and
wherein the controller (18) is configured to determine whether the vehicle (100) and the road user (106) approach each other based on a comparison of the determined light intensities of the light cones (110).
The system (10) according to claim 1 or 2,
wherein the controller (18) is configured for determining a size of the light cones (110) in each of the determined image sections; and
wherein the controller (18) is configured to determine whether the vehicle (100) and the road user (106) approach each other based on a comparison of the determined sizes of the light cones (110).
The system (10) according to one of the preceding claims,
wherein the controller (18) is configured to determine the image section in a part of the image, in which part a road (103) is captured.
The system (10) according to one of the preceding claims,
wherein the controller (18) is configured for determining a reflection of the light cone (110) on a road surface (105) in the at least two image sections; and
wherein the controller (18) is configured to determine whether the vehicle (100) and the road user (106) approach each other taking into account the determined reflections.
The system (10) according to one of the preceding claims,
wherein the controller (18) is configured for determining a movement of the light cone (110) and/or a color change by comparing the at least two images.
The system (10) according to one of the preceding claims, further comprising:
a warning device (20) for outputting a warning signal;

wherein the controller (18) is configured for triggering the warning device (20) upon determining that the vehicle (100) and the road user (106) approach each other.
The system (10) according to claim 7,
wherein the warning signal is indicative of a type of the road user (106).
A vehicle (100) comprising a cross traffic detection system (10) according to one of the preceding claims.
A method for detecting cross traffic, the method comprising the steps of:
capturing, by means of a camera arrangement (14) of a vehicle (100), an image sequence comprising consecutive images of an environment of the vehicle (100);

storing the images of the image sequence in a data storage device (16);

determining in at least two of the images an image section, in which a light cone (110) emitted by an illumination device (108) of a road user (106) is captured; and

determining whether the vehicle (100) and the road user (106) approach each other by comparing the determined image sections.