WO2023094370A1

WO2023094370A1 - Lateral boundary warning method in assisted driving

Info

Publication number: WO2023094370A1
Application number: PCT/EP2022/082776
Authority: WO
Inventors: Ashwin Dayal GEORGE
Original assignee: Valeo Schalter Und Sensoren Gmbh
Priority date: 2021-11-29
Filing date: 2022-11-22
Publication date: 2023-06-01
Also published as: DE102021131232A1

Abstract

The present invention refers to a method for generating a lateral boundary warning (54, 56) to a human driver of a vehicle (10) when approaching or crossing a lateral driving boundary (38, 40, 42), comprising the steps of receiving sensor information covering the environment (20) of the vehicle (10), identifying at least one lateral driving boundary (38, 40, 42) in the environment (20) of the vehicle (10) based on the received sensor information, determining a position of the at least one lateral driving boundary (38, 40, 42) relative to the vehicle (10) based on the received sensor information, and generating a lateral boundary warning (54, 56) based on the position of the at least one lateral driving boundary (38, 40, 42) relative to the vehicle (10) in at least one mirror device (26, 28, 30) of the vehicle (10). The present invention also refers to a driving support system (12) for a vehicle (10) for generating a lateral boundary warning (54, 56) to a human driver of the vehicle (10) when approaching or crossing a lateral driving boundary (38, 40, 42), which is adapted to perform the above method.

Description

Lateral boundary warning method in assisted driving

The present invention refers to a method for generating a lateral boundary warning to a human driver of a vehicle when approaching or crossing a lateral driving boundary.

The present invention also refers to a driving support system for a vehicle for generating a lateral boundary warning to a human driver of the vehicle when approaching or crossing a lateral driving boundary, wherein the driving support system is adapted to perform the above method.

Keeping a vehicle within lateral driving boundaries is an important task. This is already challenging with normal visibility conditions and even more challenging when driving at night on poorly lit roads or under poor weather conditions with limited visibility. In all these cases it is difficult for the driver of the vehicle to judge on the position of the vehicle in the driving lane. A high level of concentration is required, which causes stress for the driver when driving under such conditions. A lapse in concentration could lead to unintentional lane departures.

In this context, document JP 6769073 B2 relates to a vehicle display method and a vehicle display device, which display a simulated vehicle that moves in a direction when it is determined that the vehicle has a danger based on the surrounding information of the vehicle. This comprises displaying an ambient situation image and a control direction image on an image display means provided in the own vehicle.

Furthermore, document TW 2014 38943 A refers to a driving visual aids marking method and a system thereof. Different distances are marked on a horizontal line, which is provided with markings of vertical line intersecting angles at different distances. The different distances of the horizontal line and the vertical line intersecting angles define angles included therebetween. The included angles are inversely proportional to the distances on the horizontal line. Different distances are marked on a straight line, wherein the distances of the straight line appear visually in the brain being inversely proportional to the marking distances on the straight line. This is applied on the same visual plane to achieve an effect of correcting visual errors. The method assists a driver in establishing the overall front, back, left and right senses of distance.

Still further, document US 2021 024000 A1 refers to a vehicular vision system including a camera disposed at an exterior portion of a body side of the equipped vehicle and viewing sideward and rearward of the equipped vehicle. A video display screen is disposed in an interior cabin of the equipped vehicle and operable to display video images. When no other vehicle is detected in another traffic lane, the video display screen displays video images representative of a first portion of the exterior scene viewed by the camera. Responsive to determination that another vehicle is in the other traffic lane and approaching from rearward of the equipped vehicle, the video display screen displays video images representative of a second portion of the exterior scene that includes the determined other vehicle. Responsive to detection of the other vehicle, the video display screen displays a graphic overlay overlaying the displayed video images representative of the second portion of the exterior scene.

This overlay is complicated, requires high computational effort and is therefore too expensive.

It is an object of the present invention to provide a method for generating a lateral boundary warning to a human driver of a vehicle when approaching or crossing a lateral driving boundary and a driving support system for performing the method, which enable a generation of lateral boundary warnings to a human driver of the vehicle when approaching or crossing a lateral driving boundary, which can be generated in a simple and efficient way and which can be easily and reliably perceived by the human driver.

This object is achieved by the independent claims. Advantageous embodiments are given in the dependent claims.

In particular, the present invention provides a method for generating a lateral boundary warning to a human driver of a vehicle when approaching or crossing a lateral driving boundary, comprising the steps of receiving sensor information covering the environment of the vehicle, identifying at least one lateral driving boundary in the environment of the vehicle based on the received sensor information, determining a position of the at least one lateral driving boundary relative to the vehicle based on the received sensor information, and generating a lateral boundary warning based on the position of the at least one lateral driving boundary relative to the vehicle in at least one mirror device of the vehicle.

The present invention also provides a driving support system for a vehicle for generating a lateral boundary warning to a human driver of the vehicle when approaching or crossing a lateral driving boundary, comprising at least one environment sensor for providing sensor information covering the environment of the vehicle, a processing unit, which is connected via a data connection to the at least one environment sensor to receive the sensor information from the at least one environment sensor, and at least one mirror device for generating a lateral boundary warning, wherein the driving support system is adapted to perform the above method.

The basic idea of the invention is to provide a simple way of generating lateral boundary warning to a human driver of a vehicle when approaching or crossing a lateral driving boundary, which can easily and reliably be perceived in the at least one mirror device of the vehicle. The driver of the vehicle is used to frequently checking mirror devices to obtain a good overview with respect to the surrounding of his vehicle towards its rear. Hence, a lateral boundary warning displayed in at least one of the mirror devices is likely to be perceived immediately. Furthermore, drivers of current vehicles are already used to a display of warnings in different mirror devices of the vehicle. For example, blind spot detection systems are already in place, which use the wing mirrors of the vehicle in order to warn with respect to overtaking or overtaken third party vehicles on neighboring driving lanes. Hence, human drivers are already accustomed to information provided on mirror devices of a vehicle, so that they will perceive also different information displayed on the at least one mirror device in a natural way. Furthermore, display of information like warning messages on mirror devices is based on a known display technique, which incurs low costs. The lateral boundary warning provides passive assistance in the form of a visual warning on the at least one mirror device. Highly complicated autonomous driving systems are not required. As a further advantage, when providing a passive method and driving support system for warning the driver, such a system does not require a highly accurate detection of the lateral driving boundaries. They also work with simple and cheap environment sensors, which further reduces costs for generating lateral boundary warnings. Still further, the lateral boundary warnings can be generated even in poor visibility conditions, when the environment sensors provide a limited recognition of the environment of the vehicle. Hence, without the need for an accurate determination of the position of the vehicle relative to lateral driving boundaries, a reliable warning can be generated for the driver of the vehicle in a very economical way.

The lateral boundary warnings are generated based on the position of the at least one lateral driving boundary relative to the vehicle. Accordingly, a lateral distance between the position of the at least one lateral driving boundary and the vehicle is the basis for generating the lateral boundary warning. The warnings can be generated at any time when approaching or crossing a lateral driving boundary. Hence, the warnings can be generated when approaching the lateral driving boundary, i.e. prior to crossing the lateral driving boundary, and/or when crossing the lateral driving boundary, and/or after having crossed the lateral driving boundary.

The vehicle can be any kind of vehicle for driving on street and/or roads including cars, trucks, or even motorbikes or others.

The driving support system assists the human driver of the vehicle. It can be a driving support system, which provides additional kinds of driving support, or which only generates the lateral boundary warning. Such driving support systems are also known as driver assistance systems, which are frequently referred to as ADAS (Advanced Driver Assistance Systems).

The lateral driving boundaries can be identified based on line markings provided on a street or road as well as by structural differences between a street or road and a surrounding, e.g. a curb or a switch from tar to grass, as well as a guard rail delimiting a road.

The sensor information covering the environment of the vehicle is received from the at least one environment sensor of the vehicle at the processing unit. The sensor information can be pre-processed or raw sensor data.

The processing unit receives the sensor information and performs the further processing to identify the at least one lateral driving boundary in the environment of the vehicle and to determine the position of the at least one lateral driving boundary relative to the vehicle. Identifying the at least one lateral driving boundary in the environment of the vehicle based on the received sensor information can be performed e.g. based on image recognition techniques or others. The identification may vary depending on the type of lateral driving boundary and/or depending on the kind of received sensor information.

Determining the position of the at least one lateral driving boundary relative to the vehicle can be performed in a similar way.

The lateral boundary warning is a visual warning generated on the mirror device. This lateral boundary warning can be generated on a mirror surface of the at least one mirror device. However, it is also possible to generate the lateral boundary warning on a frame of the respective mirror device encompassing the mirror surface or on a housing of the respective mirror device. The lateral boundary warning is preferably generated by illuminating a respective warning symbol on the mirror device.

According to a modified embodiment of the invention, receiving sensor information covering the environment of the vehicle comprises receiving sensor information from at least one environment sensor of the vehicle out of an optical camera, a LiDAR-based environment sensor, a radar sensor, an ultrasonic sensor or an infrared sensor. In general, any kind of environment sensor can be suitable. E.g. ultrasonic sensors, radar sensors and LiDAR-based environment sensors can detect structural differences between a street or road and a surrounding, e.g. a curb, a guide rail, a switch from tar to grass, walls or other objects delimiting a street or road. Optical cameras can reliably detect line markings provided on a street or road as well as structural differences between a street or road and a surrounding. Furthermore, radar sensors and LiDAR- based environment sensors can also detect line markings provided on a street or road, when the line markings contain particles responsive to radar irradiation or light, e.g. metallic particles or highly reflective particles added to the line markings. Preferably, sensor information provided from multiple environment sensors of the same kind or in any combination of different kinds of environment sensors can be fused in order to provide a single set of environment information, in particular to provide an environment map. According to a modified embodiment of the invention, identifying at least one lateral driving boundary in the environment of the vehicle based on the received sensor information comprises identifying at least one boundary of a street or road currently used by the vehicle and/or at least one driving lane boundary of a driving lane of a street or road currently used by the vehicle. Each of the driving lane boundaries can be relevant for the vehicle. Crossing a boundary of a street or road can result in a collision with objects besides the street or road. Crossing a driving lane boundary can result in a collision with other vehicles, in particular when the neighboring driving lane belongs to vehicles moving in an opposite direction. The method can be performed based on the detected lateral driving boundary without knowledge of the type of boundary.

According to a modified embodiment of the invention, identifying at least one lateral driving boundary in the environment of the vehicle based on the received sensor information comprises determining a type of the at least one lateral driving boundary based on the received sensor information and/or determining a marking type of the at least one lateral driving boundary based on the received sensor information. In this case, the type of the line boundary is provided together with the identified at least one lateral driving boundary. Different types of lateral driving boundaries refer e.g. to boundaries of streets or roads, driving lane boundaries or others. Different marking types of lateral driving boundaries refer e.g. to discontinuous line markings, which are typically allowed to be crossed, continuous line markings, which are typically not allowed to be crossed, double continuous line markings, which can be used to separate driving lanes for opposite driving directions and which are also not allowed to be crossed, temporary line markings as used e.g. in construction areas, preferably with a different color, or others.

According to a modified embodiment of the invention, generating a lateral boundary warning based on the position of the at least one lateral driving boundary relative to the vehicle in at least one mirror device of the vehicle comprises comparing the position of the at least one lateral driving boundary relative to the vehicle with at least one threshold value and generating the lateral boundary warning when the position of the at least one lateral driving boundary relative to the vehicle reaches or passes the at least one threshold value. The threshold value defines a lateral distance between the vehicle and the respective lateral driving boundary for generating the lateral boundary warning. Preferably, multiple threshold values can be defined e.g. to generate (different) warnings depending e.g. on a remaining distance between the vehicle and the respective lateral driving boundary. Hence, when the vehicle approaches the lateral driving boundary, first a light warning is generated, and when the vehicle further approaches the lateral driving boundary, a stronger warning is generated. The multiple threshold values can be chosen with a regular, fixed difference between subsequent threshold values. Alternatively, the threshold values can be closer to each other the closer the vehicle has approached the lateral driving boundary. The threshold can be an upper or lower threshold. Hence, the warning can be generated when reaching or passing the respective threshold from above or below. Typically, the threshold value defines a lower threshold, and the lateral boundary warning will be generated when the vehicle comes closer to the at least one lateral driving boundary than the threshold value. However, in particular in case lateral driving boundaries are identified only at one side of the vehicle, the lateral boundary warning can be generated when the vehicle moves away from the identified lateral driving boundaries, i.e. based on an upper threshold. This can prevent the vehicle from leaving a street or road in case the boundaries of the street or road have not been detected. The threshold values can be fix values or variable values, e.g. depending on driving conditions like a velocity of the vehicle, depending on street or road conditions, or depending on the identified at least one lateral boundary.

According to a modified embodiment of the invention, identifying at least one lateral driving boundary in the environment of the vehicle based on the received sensor information comprises identifying a lateral driving boundary at each lateral side of the vehicle, determining a position of the at least one lateral driving boundary relative to the vehicle comprises determining the positions of the lateral driving boundaries at each lateral side of the vehicle relative to the vehicle, and generating a lateral boundary warning based on the position of the at least one lateral driving boundary relative to the vehicle in at least one mirror device of the vehicle comprises generating the lateral boundary warning under additional consideration of the positions of the lateral driving boundaries at each lateral side of the vehicle. In other words, the lateral boundary warning is generated upon a deviation of the vehicle from a center position between the lateral driving boundaries at both lateral sides of the vehicle. This helps the driver to keep the vehicle centered between the lateral driving boundaries at both sides. Accordingly, the lateral boundary warning can already be generated when the vehicle leaves the center position, instead of waiting until the vehicle approaches the respective lateral driving boundary before generating the lateral boundary warning. Furthermore, the lateral boundary warning can be generated in particular only for one lateral side of the vehicle to avoid confusion, which might occur when the lateral boundary warning is generated at the same time for both lateral sides of the vehicle, e.g. when driving in a narrow driving lane. Hence, the lateral boundary warning can be generated only for the lateral side, which is closer to the respective lateral driving boundary.

According to a modified embodiment of the invention, identifying at least one lateral driving boundary in the environment of the vehicle based on the received sensor information comprises identifying a lateral driving boundary at each lateral side of the vehicle, determining a position of the at least one lateral driving boundary relative to the vehicle comprises determining the positions of the lateral driving boundaries at each lateral side of the vehicle relative to the vehicle, and generating a lateral boundary warning based on the position of the at least one lateral driving boundary relative to the vehicle in at least one mirror device of the vehicle comprises generating the lateral boundary warning based on a distance between the lateral driving boundaries and the distance of the respective lateral driving boundary to the vehicle. In other words, the lateral boundary warning can be generated based on a relative value for approaching the lateral driving boundaries. Accordingly, the lateral boundary warning can be a relative value based on the distance between the lateral boundaries. For example, when the vehicle is driving on a driving lane, which is narrow, there is only little margin between the vehicle and the lateral driving boundaries. Hence, the lateral boundary warning will only be generated when the distance between the vehicle and the respective driving lane is relatively small. However, when the vehicle is driving on a driving lane, which is broad, there is a bigger margin between the vehicle and the lateral driving boundaries. Hence, the lateral boundary warning will already be generated when the distance between the vehicle and the respective driving lane is relatively bigger compared to driving on the narrow driving lane. Summarizing, a particular distance between the vehicle and the lateral driving boundary can result in a lateral boundary warning on a broad road, but not on a narrow road. On the narrow road, the lateral boundary warning will only be generated when the distance between the vehicle and the driving boundary is smaller.

According to a modified embodiment of the invention, generating a lateral boundary warning based on the position of the at least one lateral driving boundary relative to the vehicle in at least one mirror device of the vehicle comprises generating different lateral boundary warnings in the at least one mirror device of the vehicle depending on different positions of the at least one lateral driving boundary relative to the vehicle, generating different lateral boundary warnings in the at least one mirror device depending on different kinds of the at least one lateral driving boundary, and/or generating different lateral boundary warnings in the at least one mirror device based on different marking types of the at least one lateral driving boundary. Hence, the different lateral boundary warnings can indicate a different impact of the vehicle being closer to a driving lane boundary. Furthermore, the different lateral boundary warnings can indicate a different importance of the type of lane marking, e.g. crossing a boundary of a street or road can be considered as more dangerous and therefore more important than crossing a driving lane boundary, while the vehicle is still on the street or road. Still further, the different lateral boundary warnings can indicate a different legal impact of the type of the different marking types, e.g. crossing a continuous line is typically prohibited.

According to a modified embodiment of the invention, generating different lateral boundary warnings in the at least one mirror device comprises displaying a lateral boundary warning indication in the at least one mirror device and illuminating the lateral boundary warning indication with different intensities, displaying a lateral boundary warning indication in the at least one mirror device and illuminating the lateral boundary warning indication with different colors, displaying a lateral boundary warning indication in the at least one mirror device and illuminating the lateral boundary warning indication with different illumination modes, e.g. continuous illumination or blinking illumination, displaying differently sized lateral boundary warning indications in the at least one mirror device, displaying the lateral boundary warning indication at different positions in the at least one mirror device, displaying differently bold lateral boundary warning indications in the at least one mirror device, displaying the lateral boundary warning indication in different mirror devices, and/or displaying a different numbers of lateral boundary warning indications in the at least one mirror device. Hence, multiple ways exist to generate different lateral boundary warnings. The above specified alternatives can be applied individually or in any suitable combination. The lateral boundary warning indication can be a boundary symbol like a boundary line, which preferably provides a perspective display of the boundary line on the respective mirror device.

According to a modified embodiment of the invention, generating a lateral boundary warning based on the position of the at least one lateral driving boundary relative to the vehicle in at least one mirror device of the vehicle comprises generating the lateral boundary warning in a wing mirror or an interior rear-view mirror of the vehicle. The wing mirror is mounted laterally on the vehicle, typically in the area of an A-column of the vehicle. The rear-view mirror is typically located centrally within a passenger cabin of the vehicle. The lateral boundary warning can be generated in just one wing mirror, similarly in both wing mirrors, or alternatively in the two wing mirrors. The lateral boundary warning can also be provided in the interior rear-view mirror. Still further, both possibilities for generating the lateral boundary warning can be combined.

The at least one mirror device can comprise an optical mirror or a display, which shows a mirror view based on a camera, which is provided to substitute optical mirrors.

According to a modified embodiment of the invention, generating the lateral boundary warning in a wing mirror or an interior rear-view mirror of the vehicle comprises generating the lateral boundary warning in the wing mirror corresponding to the at least one lateral driving boundary causing the lateral boundary warning, and/or generating the lateral boundary warning laterally at a respective side of the mirror device corresponding to the at least one lateral driving boundary causing the lateral boundary warning. Hence, an intuitive warning is generated, which can be understood without understanding of a meaning of symbolic lateral boundary warning indications used as warning symbols.

According to a modified embodiment of the invention, generating a lateral boundary warning based on the position of the at least one lateral driving boundary relative to the vehicle in at least one mirror device of the vehicle comprises generating an additional acoustic warning, in particular located at a respective side of passenger cabin of the vehicle corresponding to the at least one lateral driving boundary causing the lateral boundary warning. The acoustic warning can support the driver to focus on the lateral boundary warning as generated in the at least one mirror device.

According to a modified embodiment of the invention, the method comprises the additional steps of identifying at least one third party vehicle in the environment of the vehicle based on the received sensor information, determining a position of the at least one identified third party vehicle relative to the vehicle based on the received sensor information, and generating a blind spot warning based on the position of the at least one identified third party vehicle relative to the vehicle in the at least one mirror device of the vehicle together with the lateral boundary warning. Hence, the generation of the lateral boundary warnings can be combined with warnings generated by a blind spot detection system (BSD), which are together provided in the mirror device. Such blind spot detection systems may warn the driver of the vehicle in respect to other vehicles that may be approaching his/her vehicle on a neighboring driving lane. This is in particular important in case the other vehicle has a much higher speed.

Features and advantages described above with reference to the inventive method apply equally to the inventive driving support system and vice versa.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. Individual features disclosed in the embodiments can constitute alone or in combination an aspect of the present invention. Features of the different embodiments can be carried over from one embodiment to another embodiment.

In the drawings:

Fig. 1 shows a schematic view of an ego vehicle with a driving support system comprising multiple environment sensors and a processing unit, which are connected via a data connection, according to a first, preferred embodiment,

Fig. 2 shows a first driving situation of the vehicle of Fig. 1 on a road with three driving lanes with a detailed zoom view of the wing mirrors and the lateral boundary warnings displayed in the wing mirrors according to the first embodiment,

Fig. 3 shows a second driving situation of the vehicle of Fig. 1 on a road with three driving lanes with a detailed zoom view of the wing mirrors and the lateral boundary warnings displayed in the wing mirrors according to the first embodiment,

Fig. 4 shows a third driving situation of the vehicle of Fig. 1 on a road with three driving lanes with a detailed zoom view of the wing mirrors and the lateral boundary warnings displayed in the wing mirrors according to the first embodiment,

Fig. 5 shows a fourth driving situation of the vehicle of Fig. 1 on a road with three driving lanes with a detailed zoom view of the wing mirrors and the lateral boundary warnings displayed in the wing mirrors according to the first embodiment,

Fig. 6 shows a fifth driving situation of the vehicle of Fig. 1 on a road with three driving lanes with a detailed zoom view of a rear-view mirror and the lateral boundary warnings displayed in the rear-view mirror according to the first embodiment,

Fig. 7 shows a flow chart of a method for generating a lateral boundary warning to a human driver of a vehicle when approaching or crossing a lateral driving boundary according to the first embodiment.

Fig. 8 shows a sixth driving situation of with a vehicle of a second embodiment on a road with three driving lanes with a detailed zoom view of the wing mirrors and the lateral boundary warnings displayed in the wing mirrors together with a display for blind spot detection according to the second embodiment,

Fig. 9 shows a seventh driving situation of with the vehicle of the second embodiment on a road with three driving lanes with a detailed zoom view of the wing mirrors and the lateral boundary warnings displayed in the wing mirrors together with a display for blind spot detection according to the second embodiment,

Fig. 10 shows an eighth driving situation of with the vehicle of the second embodiment on a road with three driving lanes with a detailed zoom view of the wing mirrors and the lateral boundary warnings displayed in the wing mirrors together with a display for blind spot detection according to the second embodiment, Fig. 11 shows a ninth driving situation of with the vehicle of the second embodiment on a road with three driving lanes with a detailed zoom view of the wing mirrors and the lateral boundary warnings displayed in the wing mirrors together with a display for blind spot detection warnings according to the second embodiment,

Fig. 12 shows a tenth driving situation of with the vehicle of the second embodiment on a road with three driving lanes with a detailed zoom view of the wing mirrors and the lateral boundary warnings displayed in the wing mirrors together with a display for blind spot detection warnings according to the second embodiment,

Fig. 13 shows an eleventh driving situation of with the vehicle of the second embodiment on a road with three driving lanes with a detailed zoom view of the wing mirrors and the lateral boundary warnings displayed in the wing mirrors together with a display for blind spot detection warnings according to the second embodiment, and

Fig. 14 shows a flow chart of a method for generating a lateral boundary warning to a human driver of a vehicle when approaching or crossing a lateral driving boundary according to the second embodiment.

Figure 1 shows a vehicle 10 comprising a driving support system 12, which provides driving support for a human driver, according to a first, preferred embodiment. The vehicle 10 can be any kind of vehicle 10, e.g. a passenger car, a truck or a motor bike, which is driven by the human driver.

The driving support system 12 assists the human driver of the vehicle 10. It can be a driving support system 12, which provides additional kinds of driving support, or which only generates the lateral boundary warnings 54, 56 as discussed below in detail. Such driving support systems 12 are also known as driver assistance systems, which are frequently referred to as ADAS (Advanced Driver Assistance Systems). The driving support system 12 comprises in this embodiment a set of environment sensors 14, 16, 18 for monitoring an environment 20 of the vehicle 10. The environment sensors 14, 16, 18 comprise a LiDAR-based environment sensor 14, an optical camera 16 and multiple ultrasonic sensors 18. The environment sensors 14, 16, 18 recognize the environment 20 of the vehicle 10. The environment sensors 14, 16, 18 generate sensor information, which can comprise raw data or pre-processed data.

The vehicle 10 of the first embodiment further comprises a processing unit 22 and a data connection 24, which interconnects the environment sensors 14, 16, 18 and the processing unit 22. The processing unit 22 can be any kind of processing unit 22 suitable for the use in the system under test 10. Such processing units 22 are typically known as ECU (electronic control unit) in the automotive area. The processing unit 22 can be shared for performing multiple tasks or applications. The processing unit 22 receives and processes the sensor information provided from the environment sensors 14, 16, 18.

The data connection 24 can be a dedicated connection between the environment sensors 14, 16, 18 and the processing unit 22 or a data bus. Furthermore, the data connection 24 can be a shared data connection 24 used by different kinds of devices of the vehicle 10, e.g. a multi-purpose data bus. The data connection 24 can be implemented e.g. as CAN-bus, LIN-bus, or others.

Although a single data connection 24 is depicted in figure 1 , multiple connections or data busses can be provided in parallel for connecting the environment sensors 14, 16, 18 to the processing unit 22, and which are considered together as data connection 24. The sensor information from the environment sensors 14, 16, 18 is transferred to the processing unit 22 via the data connection 24. Similarly, although a single processing unit 22 is depicted in figure 1 , multiple processing units 22 can be provided in parallel for processing the sensor information from the environment sensors 14, 16, 18 in any way.

The vehicle 10 further comprises a left wing mirror 26 and a right wing mirror 28, as can be seen in figure 1 as well as throughout figure 2 to 5. Furthermore, as indicated in figure 6, the vehicle 10 comprises an interior rear-view mirror 30, which is located inside a passenger cabin of the vehicle 10. The left and right wing mirrors 26, 28 and the rearview mirror 30 are mirror devices 26, 28, 30 in the sense of the present disclosure. Subsequently will be described a method for generating a lateral boundary warning 54, 56 to a human driver of the vehicle 10 when approaching or crossing a lateral driving boundary 38, 40, 42 according to the first embodiment. A flow chart of the method is shown in figure 7. Figures 2 to 6 show examples how the lateral boundary warnings 54, 56 are generated according to the described method.

The method starts with step S100, which refers to receiving sensor information covering the environment 20 of the vehicle 10 from the environment sensors 14, 16, 18 of the vehicle 10 at the processing unit 22. The sensor information can be pre-processed or raw sensor data. The processing unit 22 receives the sensor information and performs the further processing, as described below.

In this embodiment, the vehicle 10 is equipped with the LiDAR-based environment sensor 14, the optical camera 16 and multiple ultrasonic sensors 18 provided at all sides of the vehicle 10. The sensor information provided from the environment sensors 14, 16, 18 is fused by the processing unit 22 to provide a single set of environment information, in particular to provide an environment map.

Step S110 refers to identifying at least one lateral driving boundary 38, 40, 42 in the environment 20 of the vehicle 10 based on the received sensor information.

In this respect, figures 2 to 6 show a road 32 with three driving lanes 34, 36 and different kinds of lateral driving boundaries 38, 40, 42. As can be seen in figures 2 to 6, the road 32 is provided with the three driving lanes 34, 36 for only one driving direction. The vehicle 10 is driving on an ego driving lane 36, which is located between two neighboring driving lanes 34. Hence, the road 32 is delimited at its right and left side by road boundaries 38 as lateral driving boundaries 38, 40, 42. In the examples of figures 2 to 6, the three driving lanes 34, 36 are separated by driving lane boundaries 40, 42, which are a dashed line 40, as shown on a right side of the vehicle 10, or a continuous line 42, as shown on a left side of the vehicle 10.

The driving lane boundaries 40, 42 are provided as line markings provided on the road 32. The road boundaries 38 are provided as a combination of line markings together with a curb or a guard rail delimiting the road 32. The identification of the lateral driving boundaries 38, 40, 42 in the environment 20 of the vehicle 10 is performed based on the received sensor information using image recognition techniques or others. When the lateral driving boundaries 38, 40, 42 are identified, a type of the respective lateral driving boundary 38, 40, 42 is determined based on the received sensor information. In addition, a marking type of the respective lateral driving boundary 38, 40, 42 is determined based on the received sensor information. Hence, each lateral driving boundary 38, 40, 42 is identified as road boundary 38 or as lane boundary 40, 42. Furthermore, the different marking types of the lateral driving boundaries 38, 40, 42 are determined as discontinuous line markings, i.e. dashed lines 40, and continuous line markings, i.e. continuous lines 42.

The lateral driving boundaries 38, 04, 42 are identified at each lateral side of the vehicle 10.

Step S120 refers to determining a position of the at least one lateral driving boundary 38, 40, 42 relative to the vehicle 10 based on the received sensor information.

Accordingly, a lateral distance between the position of the respective lateral driving boundary 38, 40, 42 and the vehicle 10 is determined as basis for a subsequent generation of the lateral boundary warning 54, 56.

This includes that the position of the lateral driving boundaries 38, 40, 42 are determined at both lateral sides of the vehicle 10 relative to the vehicle 10.

Step S130 refers to generating a lateral boundary warning 54, 56 based on the position of the at least one lateral driving boundary 38, 40, 42 relative to the vehicle 10 in at least one mirror device 26, 28, 30 of the vehicle 10.

The lateral boundary warning 54, 56 can be generated when approaching the respective lateral driving boundary 38, 40, 42, i.e. prior to crossing the lateral driving boundary 38, 40, 42, and/or when crossing the lateral driving boundary 38, 40, 42, and/or after having crossed the lateral driving boundary 38, 40, 42. The lateral boundary warning 54, 56 is generated based on a comparison of the position of the at least one lateral driving boundary 38, 40, 42 relative to the vehicle 10 with at least one threshold value and generating the lateral boundary warning 54, 56 when the position of the at least one lateral driving boundary 38, 40, 42 relative to the vehicle 10 reaches or passes the at least one threshold value. The threshold value defines a lateral distance between the vehicle 10 and the respective lateral driving boundary 38, 40, 42 for generating the lateral boundary warning. Multiple threshold values are defined e.g. to generate (different) warnings depending e.g. on a remaining distance between the vehicle 10 and the respective lateral driving boundary 38, 40, 42, as discussed below. Hence, when the vehicle 10 approaches the lateral driving boundary 38, 40, 42, first a first level lateral boundary warning 54 is generated, and when the vehicle 10 further approaches the lateral driving boundary 38, 40, 42, a second level lateral boundary warning 56 is generated. This can be seen in figures 2 to 6.

As can be seen in figure 2, the vehicle is located in a center of the ego driving lane 36. The left and right wing mirrors 26, 28 indicate the continuous line 42 at the left side of the vehicle 10 by a respective continuous line symbol 50 in the left wing mirror 26 and a respective dashed line symbol 52 in the right wing mirror 28. No lateral boundary warning 54, 56 is generated. The same is shown in figure 6 for the interior rear-view mirror 30 where the continuous line symbol 50 is shown at a left side and the respective dashed line symbol 52 is shown at a right side of the interior rear-view mirror 30 in accordance with the position of the continuous line 42 at the left side of the vehicle 10 and the dashed line 40 at the right side of the vehicle 10.

As shown in figure 3, the vehicle 10 has approached the continuous line 42 at its left side and passed a first distance d1 , which is a first threshold value. Accordingly, a first level lateral boundary warning 54 is shown in the left wing mirror 26. The display of the right wing mirror 28 has not changed compared to figure 2.

As shown in figure 4, the vehicle 10 has further approached the continuous line 42 at its left side and passed a second distance d2, which is a second threshold value. Accordingly, a second level lateral boundary warning 56 is shown in the left wing mirror 26. The display of the right wing mirror 28 has not changed compared to figure 2. Figure 5 shows a display of the left wing mirror 26 and the right wing mirror 28, when the continuous line 42 is not properly identified by the processing unit 22. Hence, the left wing mirror 26 shows a not illuminated symbol 58 indicating that no lateral driving boundary 38, 40, 42 has been identified at the left side of the vehicle 10. The display of the right wing mirror 28 has not changed compared to figure 2, since the dashed line 40 has been properly identified by the processing unit 22.

Overall, as can be seen from figure 2 to 6, the lateral boundary warning 54, 56 is a visual warning generated in the mirror device 26, 28, 30, which is generated on a mirror surface of the respective mirror device 26, 28, 30. Although not shown in the figures, the different lateral boundary warnings 54, 56 are based on a display of a line indicating the respective lateral driving boundary 38, 40, 42 as lateral boundary warning indication. The differences between the different lateral boundary warnings 54, 56 can be generated by illuminating the lateral boundary warning indication with different intensities, illuminating the lateral boundary warning indication with different colors, illuminating the lateral boundary warning indication with different illumination modes, e.g. continuous illumination or blinking illumination, displaying differently sized lateral boundary warning indications in the respective mirror device 26, 28, 30, displaying the lateral boundary warning indication at different positions in the respective mirror device 26, 28, 30, displaying differently bold lateral boundary warning indications in the respective mirror device 26, 28, 30, and/or displaying a different numbers of lateral boundary warning indications in the respective mirror device 26, 28, 30. Furthermore, as already described above, the lateral boundary warnings 54, 56 in respect to a lateral driving boundary 38, 40, 42 at the left side of the vehicle 10 are indicated by displaying the lateral boundary warning indication in the left wing mirror 26 as well as at the left side of the interior rear-view mirror 30. The same way, the lateral boundary warnings 54, 56 in respect to a lateral driving boundary 38, 40, 42 at the right side of the vehicle 10 are indicated by displaying the lateral boundary warning indication in the right wing mirror 28 as well as at the right side of the interior rear-view mirror 30.

For each of the lateral boundary warnings 54, 56, an additional acoustic boundary warning is generated, in particular located at a respective side of passenger cabin of the vehicle 10 corresponding to the respective lateral driving boundary 38, 40, 42 causing the lateral boundary warning 54, 56. In an alternative embodiment, the lateral boundary warning 54, 56 is generated under additional consideration of the positions of the lateral driving boundaries 38, 40, 42 at each lateral side of the vehicle 10. In other words, the lateral boundary warning 54, 56 is generated upon a deviation of the vehicle 10 from a center position between the lateral driving boundaries 38, 40, 42 at both lateral sides of the vehicle 10, so that the lateral boundary warning 38, 40, 42 can already be generated when the vehicle 10 leaves the center position within the ego driving lane 36, instead of waiting until the vehicle 10 approximates the respective lateral driving boundary 38, 40, 42.

In a further alternative embodiment, the lateral boundary warning 54, 56 is generated based on a distance between the lateral driving boundaries 38, 40, 42 and the distance of the respective lateral driving boundary 38, 40, 42 to the vehicle 10. In other words, the lateral boundary warning 54, 56 is generated based on a relative value for approaching the lateral driving boundaries 38, 40, 42 depending on a distance between the lateral driving boundaries 38, 40 42.

Subsequently will be described a method for generating a lateral boundary warning to a human driver of the vehicle 10 when approaching or crossing a lateral driving boundary 38, 40, 42 according to a second embodiment. A flow chart of the method is shown in figure 14. Figures 8 to 13 show indicate by way of example how the lateral boundary warnings 38, 40, 42 are generated according to the described method.

The method is performed with a vehicle 10 of a second embodiment, which is identical to the vehicle 10 of the first embodiment except for the display of warnings in the respective mirror devices 26, 28, 30, as discussed below. Hence, the same reference numerals are used and a detailed description of the vehicle 10 of the second embodiment is omitted.

Furthermore, the method of the second embodiment comprises method steps S100 to S120, as described above in respect to the method of the first embodiment, without changes.

In step S130, the method of the second embodiment comprises identifying at least one third party vehicle 60 in the environment 20 of the vehicle 10 based on the received sensor information. Step S130 is performed by the processing unit 22. Step S140 refers to determining a position of the at least one identified third party vehicle 60 relative to the vehicle 10 based on the received sensor information. Also step S140 is performed by the processing unit 22.

Step S150 refers to generating a blind spot warning 64, 66 based on the position of the at least one identified third party vehicle 60 relative to the vehicle 10 in the at least one mirror device 26, 28, 30 of the vehicle 10 together with the lateral boundary warning 54, 56. Hence, the lateral boundary warnings 54, 56 are generated in the mirror devices 26, 28, 30 as described in respect to step S130 of the method of the first embodiment together with the blind spot warnings 64, 66.

In addition to the lateral boundary warnings 54, 56, the blind spot warning 64, 66 is generated and shown together with the lateral boundary warnings 54, 56 in the respective mirror device 26, 28, 30.

Accordingly, as can be seen in figures 8 to 10, no third party vehicles 60 are present in the environment 20 of the vehicle 10. Hence, the lateral boundary warnings 54, 56 are generated as described with regard to figures 2 to 4 of the method of the first embodiment. Merely an additional blind spot detection symbol 62 is shown in the wing mirrors 26, 28, indicating that the blind spot detection is active.

Figure 11 to 13 show an example with third party vehicles 60 present in the environment 20 of the vehicle 10. As can be seen in figure 11 , the vehicle 10 is driving in the ego driving lane 36, which is in this example delimited at both sides by a dashed line 40. In figure 11 , the third party vehicles 60 have not yet reached the environment 20 of the vehicle 10. The wing mirrors 26, 28 display the dashed line symbol 52 together with the blind spot detection symbol 62.

According to figure 12, two third party vehicles 60 have approached the vehicle 10, one on each neighboring driving lane 34. Accordingly, a first level blind spot warning 64 is generated in both wing mirrors 26, 28. Additionally, the two third party vehicles 60 increase a possible consequence of crossing the driving lane boundaries 40, 42, so that also the first level lateral boundary warning 54 is generated in both wing mirrors 26, 28. Figure 13 is based on figure 12. The two third party vehicles 60 have approached the vehicle 10. Furthermore, according to figure 13, the vehicle 10 has approached the dashed line 40 at its left side and passed a third distance d3, which is a third threshold value applicable only in case third party vehicles 60 are present. Accordingly, the second level lateral boundary warning 56 is shown in the left wing mirror 26. Furthermore, a second level blind spot warning 66 is generated in the left wing mirror 26. The display of the right wing mirror 28 has not changed compared to figure 12.

Reference signs list

10 vehicle

12 driving support system

14 LiDAR-based environment sensor, environment sensor

16 optical camera, environment sensor

18 ultrasonic sensor, environment sensor

20 environment

22 processing unit

24 data connection

26 left wing mirror, mirror device

28 right wing mirror, mirror device

30 interior rear-view mirror, mirror device

32 road

34 neighboring driving lane, driving lane

36 ego driving lane, driving lane

38 road boundary, lateral driving boundary

40 dashed line, driving lane boundary, lateral driving boundary

42 continuous line, driving lane boundary, lateral driving boundary

50 continuous line symbol

52 dashed line symbol

54 first level lateral boundary warning

56 second level lateral boundary warning

60 third party vehicle

62 blind spot detection symbol

64 first level blind spot warning

66 second level blind spot warning

Claims

23

Patent claims Method for generating a lateral boundary warning (54, 56) to a human driver of a vehicle (10) when approaching or crossing a lateral driving boundary (38, 40, 42), comprising the steps of receiving sensor information covering the environment (20) of the vehicle (10), identifying at least one lateral driving boundary (38, 40, 42) in the environment (20) of the vehicle (10) based on the received sensor information, determining a position of the at least one lateral driving boundary (38, 40, 42) relative to the vehicle (10) based on the received sensor information, and generating a lateral boundary warning (54, 56) based on the position of the at least one lateral driving boundary (38, 40, 42) relative to the vehicle (10) in at least one mirror device (26, 28, 30) of the vehicle (10). Method according to claim 1 , characterized in that receiving sensor information covering the environment (20) of the vehicle (10) comprises receiving sensor information from at least one environment sensor (14, 16, 18) of the vehicle out (10) of an optical camera (16), a LiDAR-based environment sensor (14), a radar sensor, an ultrasonic sensor (18), or an infrared sensor. Method according to any of preceding claims 1 or 2, characterized in that identifying at least one lateral driving boundary (38, 40, 42) in the environment (20) of the vehicle (10) based on the received sensor information comprises identifying at least one boundary (38) of a street or road (32) currently used by the vehicle (10) and/or at least one driving lane boundary (40, 42) of a driving lane (34, 36) of a street or road (32) currently used by the vehicle (10). Method according to any preceding claim, characterized in that identifying at least one lateral driving boundary (38, 40, 42) in the environment (20) of the vehicle (10) based on the received sensor information comprises determining a type of the at least one lateral driving boundary (38, 40, 42) based on the received sensor information and/or determining a marking type of the at least one lateral driving boundary (38, 40, 42) based on the received sensor information. Method according to any of preceding claims 1 to 3, characterized in that generating a lateral boundary warning (54, 56) based on the position of the at least one lateral driving boundary (38, 40, 42) relative to the vehicle (10) in at least one mirror device (26, 28, 30) of the vehicle (10) comprises comparing the position of the at least one lateral driving boundary (38, 40, 42) relative to the vehicle (10) with at least one threshold value and generating the lateral boundary warning (5, 56) when the position of the at least one lateral driving boundary (38, 40, 42) relative to the vehicle (10) reaches or passes the at least one threshold value. Method according to any preceding claim, characterized in that identifying at least one lateral driving boundary (38, 40, 42) in the environment (20) of the vehicle (10) based on the received sensor information comprises identifying a lateral driving boundary (38, 40, 42) at each lateral side of the vehicle (10), determining a position of the at least one lateral driving boundary (38, 40, 42) relative to the vehicle (10) comprises determining the positions of the lateral driving boundaries (38, 40, 42) at each lateral side of the vehicle (10) relative to the vehicle (10), and generating a lateral boundary warning (54, 56) based on the position of the at least one lateral driving boundary (38, 40, 42) relative to the vehicle (10) in at least one mirror device (26, 28, 30) of the vehicle (10) comprises generating the lateral boundary warning (54, 56) under additional consideration of the positions of the lateral driving boundaries (38, 40, 42) at each lateral side of the vehicle (10). Method according to any preceding claim, characterized in that identifying at least one lateral driving boundary (38, 40, 42) in the environment (20) of the vehicle (10) based on the received sensor information comprises identifying a lateral driving boundary (38, 40, 42) at each lateral side of the vehicle (10), determining a position of the at least one lateral driving boundary (38, 40, 42) relative to the vehicle (10) comprises determining the positions of the lateral driving boundaries (38, 40, 42) at each lateral side of the vehicle (10) relative to the vehicle (10), and generating a lateral boundary warning (54, 56) based on the position of the at least one lateral driving boundary (38, 40, 42) relative to the vehicle (10) in at least one mirror device (26, 28, 30) of the vehicle (10) comprises generating the lateral boundary warning (54, 56) based on a distance between the lateral driving boundaries (38, 40, 42) and the distance of the respective lateral driving boundary (38, 40, 42) to the vehicle (10). Method according to any preceding claim, characterized in that generating a lateral boundary warning (54, 56) based on the position of the at least one lateral driving boundary (38, 40, 42) relative to the vehicle (10) in at least one mirror device (26, 28, 30) of the vehicle (10) comprises

- generating different lateral boundary warnings (54, 56) in the at least one mirror device (26, 28, 30) of the vehicle (10) depending on different positions of the at least one lateral driving boundary (38, 40, 42) relative to the vehicle (10),

- generating different lateral boundary warnings (54, 56) in the at least one mirror device (26, 28, 30) depending on different kinds of the at least one lateral driving boundary (38, 40, 42), and/or

- generating different lateral boundary warnings (54, 56) in the at least one mirror device (26, 28, 30) based on different marking types of the at least one lateral driving boundary (38, 40, 42). Method according to claim 8, characterized in that generating different lateral boundary warnings (54, 56) in the at least one mirror device (26, 28, 30) comprises displaying a lateral boundary warning indication in the at least one mirror device (26, 28, 30) and illuminating the lateral boundary warning indication with different intensities, 26 displaying a lateral boundary warning indication in the at least one mirror device (26, 28, 30) and illuminating the lateral boundary warning indication with different colors, displaying a lateral boundary warning indication in the at least one mirror device (26, 28, 30) and illuminating the lateral boundary warning indication with different illumination modes, e.g. continuous illumination or blinking illumination, displaying differently sized lateral boundary warning indications in the at least one mirror device (26, 28, 30), displaying the lateral boundary warning indication at different positions in the at least one mirror device (26, 28, 30), displaying differently bold lateral boundary warning indications in the at least one mirror device (26, 28, 30), displaying the lateral boundary warning indication in different mirror devices (26, 28, 30), and/or displaying a different numbers of lateral boundary warning indications in the at least one mirror device (26, 28, 30). Method according to any preceding claim, characterized in that generating a lateral boundary warning (54, 56) based on the position of the at least one lateral driving boundary (38, 40, 42) relative to the vehicle (10) in at least one mirror device (26, 28, 30) of the vehicle (10) comprises generating the lateral boundary warning (54, 56) in a wing mirror (26, 28) or an interior rear-view mirror (30) of the vehicle (10). Method according to preceding claim 10, characterized in that generating the lateral boundary warning (54, 56) in a wing mirror (26, 28) or an interior rear-view mirror (30) of the vehicle (10) comprises

- generating the lateral boundary warning (54, 56) the wing mirror (26,

28) corresponding to the at least one lateral driving boundary (38, 40, 42) causing the lateral boundary warning (54, 56), and/or

- generating the lateral boundary warning (54, 56) laterally at a respective side of the mirror device (26, 28, 30) corresponding to the at least one lateral driving boundary (38, 40, 42) causing the lateral boundary warning (54, 56). 27 Method according to any preceding claim, characterized in that generating a lateral boundary warning (54, 56) based on the position of the at least one lateral driving boundary (38, 40, 42) relative to the vehicle (10) in at least one mirror device (26, 28, 30) of the vehicle (10) comprises generating an additional acoustic warning, in particular located at a respective side of passenger cabin of the vehicle (10) corresponding to the at least one lateral driving boundary (38, 40, 42) causing the lateral boundary warning (54, 56). Method according to any preceding claim, characterized in that the method comprises the additional steps of identifying at least one third party vehicle (60) in the environment (20) of the vehicle (10) based on the received sensor information, determining a position of the at least one identified third party vehicle (60) relative to the vehicle (10) based on the received sensor information, and generating a blind spot warning (64, 66) based on the position of the at least one identified third party vehicle (60) relative to the vehicle (10) in the at least one mirror device (26, 28, 30) of the vehicle (10) together with the lateral boundary warning (54, 56). Driving support system (12) for a vehicle (10) for generating a lateral boundary warning (54, 56) to a human driver of the vehicle (10) when approaching or crossing a lateral driving boundary (38, 40, 42), comprising at least one environment sensor (14, 16, 18) for providing sensor information covering the environment of the vehicle (10), a processing unit (22), which is connected via a data connection (24) to the at least one environment sensor (14, 16, 18) to receive the sensor information from the at least one environment sensor (14, 16, 18), and at least one mirror device (26, 28, 30) for generating a lateral boundary warning (54, 56), wherein the driving support system (12) is adapted to perform the method of any of claims 1 to 13.