WO2024132829A1

WO2024132829A1 - Determining applicable speed limit for road work areas based on visual cues

Info

Publication number: WO2024132829A1
Application number: PCT/EP2023/085763
Authority: WO
Inventors: Alexandre GARNAULT; Benazouz Bradai
Original assignee: Valeo Schalter und Sensoren GmbH
Current assignee: Valeo Schalter und Sensoren GmbH
Priority date: 2022-12-21
Filing date: 2023-12-14
Publication date: 2024-06-27
Anticipated expiration: 2025-06-21
Also published as: DE102022134372A1

Abstract

The present invention refers to a method for determining an applicable speed limit for a vehicle (10) based on a perception of the environment (16) of the vehicle (10) with at least one environment sensor (14) covering the environment (16) of the vehicle (10), comprising the steps of receiving sensor information (30) from the at least one environment sensor (14) at least partially covering the environment (16) of the vehicle (10), identifying visual cues (44, 48, 50, 54, 56, 58, 60) indicative for presence of a road work area (24) based on the received sensor information (30), calculating a confidence level for presence of a road work area (24) based on the identified visual cues (44, 48, 50, 54, 56, 58, 60), identifying presence of a road work area (24) based on the confidence level being above a threshold value indicative for driving in a road work area (24), and determining the applicable speed limit based on the identified presence of the road work area (24). The present invention also refers to an intelligent speed assistance system (12) for determining an applicable speed limit for a vehicle (10), which is adapted to perform the above method.

Description

Determining applicable speed limit for road work areas based on visual cues

The present invention refers to a method for determining an applicable speed limit for a vehicle based on a perception of the environment of the vehicle with at least one environment sensor covering the environment of the vehicle.

The present invention also refers to an intelligent speed assistance system for determining an applicable speed limit for a vehicle, wherein the intelligent speed assistance system is adapted to perform the above method for determining an applicable speed limit for a vehicle based on a perception of the environment of the vehicle with at least one environment sensor covering the environment of the vehicle.

New general safety regulations require vehicles to be equipped with Intelligent Speed Assistance (ISA) systems, which determine a currently applicable speed limit. These systems shall assist a human driver of the vehicle by informing in respect to the currently applicable speed limit, or they shall apply the applicable speed limit by themselves, i.e. the Intelligent Speed Assistance systems limit the velocity of the vehicle based on the currently applicable speed limit.

In one approach, such Intelligent Speed Assistant systems can comprise a front optical camera, which is used to detect traffic signs with applicable speed limits and possibly other features such driving lanes. The currently applicable speed limit is determined based on the speed limit indicated by the respected traffic signs. This approach depends on the performance of the system in detecting the respective traffic signs and identifying the indicated speed limits. In particular in conditions with no light, low light and twilight, a reliable detection of the traffic signs is challenging. Furthermore, uncertainty exists or can be added in case the traffic signs are partially or fully covered, e.g. by other traffic participants or by vegetation, or in case the traffic signs are located outside a detection window of the optical camera.

Another approach is based on a map data service containing speed limit information. The map data with the speed limit information may either be stored in the vehicle, i.e. a navigation system, or in a remote cloud server, which provides the map data including the speed limit information for a current location of the vehicle. The current location of the vehicle is determined based on reception of signals from a global navigation satellite system (GNSS). Several GNSS are known and can be used alone or in combination. This approach only works with continuously updated map data and speed limit information. If the information is stored in the vehicle, huge amounts of data have to be transferred frequently to update the information in the vehicle. If the information is transferred to the vehicle on demand from the remote cloud server, a continuous data connection between vehicle and server is required. Furthermore, also in this case, huge amounts of data have to be transferred.

In order to overcome some of these drawbacks, a third approach consists in a combination of the two above approaches. However, this combination also combines the drawbacks of each approaches up to a certain degree.

In each case, a major problem of the above approaches is the handling of temporary speed limit changes on road segments. Such temporary speed limit changes may be difficult to detect with the front optical camera, for which the infrastructure may be failing to provide explicit speed limits, or the camera fails to detect the respective traffic signs, and the temporary speed limit changes are not contained in the map data, when the speed limit information of the map data is not up to date due to the temporary character of the respective speed limits. Such temporary speed limit changes are frequently based on road work areas.

In several countries, the speed limit is reduced near a road work area, which is indicated by a traffic sign. This temporarily reduced speed limit applies on the road section corresponding to the length of the road work area. There are usually traffic signs indicating the reduced speed limit at the beginning of the road work area, which can be detected by the front camera, but which is most probably not registered in the map data with the speed limit information. Apart from the possibility that the traffic signs indicating the temporarily applicable speed limit are not correctly recognized, the indication of a different speed limit based on the map data increases the risk that the overall output of the Intelligent Speed Assistant system is not correct, since two different velocities are identified based on the different approaches, and the system has to choose one of them. Still further, even when the system has correctly identified the temporarily applicable speed limit in the road work area, a further challenge consists in determining an end of the temporarily reduced speed limit, i.e. when the temporarily reduced speed limit is not applicably anymore. This is the case when the vehicle has passed the road work area and the currently applicable speed limit equals the speed limit before entering the road work area. The difficulty is that there are frequently no traffic signs indicating the end of the road work area. Additionally, there is a risk that traffic signs indicating the end of the road work area are not correctly recognized. Therefore, the system does not know when to switch to back the previously applicable speed limit of the section before entering the road work area. Hence, the system can remain stuck with an incorrect speed limit, which lowers the performance of the system and reduces a user experience of the system. Accordingly, acceptance of such driving support systems will be reduced, and people will not rely on the indications provided by the Intelligent Speed Assistance systems.

It is an object of the present invention to provide a method for determining an applicable speed limit for a vehicle based on a perception of the environment of the vehicle and an intelligent speed assistance system adapted to perform the above method, which are able to determine applicable speed limits under consideration of road work areas, in particular temporarily changed speed limits due to road work areas.

This object is achieved by the independent claims. Advantageous embodiments are given in the dependent claims.

In particular, the present invention provides a method for determining an applicable speed limit for a vehicle based on a perception of the environment of the vehicle with at least one environment sensor covering the environment of the vehicle, comprising the steps of receiving sensor information from the at least one environment sensor at least partially covering the environment of the vehicle, identifying visual cues indicative for presence of a road work area based on the received sensor information, calculating a confidence level for presence of a road work area based on the identified visual cues, identifying presence of a road work area based on the confidence level being above a threshold value indicative for driving in a road work area, and determining the applicable speed limit based on the identified presence of the road work area. The present invention also provides an intelligent speed assistance system for determining an applicable speed limit for a vehicle, wherein the intelligent speed assistance system is adapted to perform the above method for determining an applicable speed limit for a vehicle based on a perception of the environment of the vehicle with at least one environment sensor covering the environment of the vehicle.

The basic idea of the invention is to adapt the currently applicable speed limit based on presence of road work areas and temporarily applicable speed limits, which are applicable when driving in such road work areas. The road work area is determined based on visual cues, which are detected in the environment of the vehicle, and which together indicate presence of a road work area. The identification is based on a confidence level, which is calculated based on the visual cues, so that the road work area can be reliably identified based on different visual cues. Typically, the visual cues indicative for presence of a road work area are present in the surrounding of the vehicle, the higher the confidence level for presence of a road work area. According to the presence or absence of a road work area, the applicable speed limit can be reliably determined. In particular, the identification of the road work area provides means to detect the end of the road work area independently from traffic signs. Such traffic signs are frequently omitted or not correctly located, so that in particular the end of the road word area and the respective speed limit is not correctly detected. However, the visual cues indicative for the road work area disappear after having passed the road work area, so that the vehicle can return to normal driving after having passed the road work area.

The applicable speed limit is a current speed limit, which is applicable for the vehicle. It can be based on explicit rules/traffic signs as well as on implicit rules/traffic signs. Furthermore, the current speed limit can be derived from traffic law, which is typically national law, and which may specify e.g. a speed limit for driving on different kinds of roads and/or for driving in a town. The presence of a road work area overrules such speed limits temporarily, so that a different speed limit becomes applicable at least while driving through the road work area.

The vehicle can be any kind of vehicle, which performs the method. Is merely required that the vehicle is equipped with at least one environment sensor covering the environment of the vehicle. Hence, the at least one environment sensor provides sensor information, which cover the environment of the vehicle at least partially. The coverage of the environment can depend on the kind of the at least one environment sensor, the detailed design of the at least one environment sensor, the number of environment sensors and the positioning of the at least one environment sensor at the vehicle. Also a combination of different kinds of environment sensors can be used.

The environment is at least partially covered by the at least one environment sensor in order to identify the visual cues. The environment can be fully covered by the at least one environment sensor. In this context, multiple environment sensors can together provide the sensor information, which at least partially cover the environment.

The method can be performed entirely by a single control unit. However, the method can be performed by a processing unit, which performs further tasks, e.g. a control unit of a further driving support system. Alternatively, the method can be performed commonly using different control units.

Receiving the sensor information from the at least one environment sensor refers to a transfer of the sensor information from the at least one environment sensor to a control unit or a general processing unit for processing the received sensor information.

Identifying visual cues indicative for presence of a road work area based on the received sensor information refers to a processing of the received sensor information in order to identify the visual cues. For example, image processing techniques can be used to identify the visual cues. This can include identifying visual cues, which are counterindicative for presence of a road work area, e.g. the visual cues can provide a positive or a negative indication for presence of a road work area. Visual cues indicative for presence of a road work area can be separated or marked for further processing according to the described method.

Calculating a confidence level for presence of a road work area based on the identified visual cues refers to a processing of information in respect to the identified visual cues. Any kind of information in respect to the visual cues can be processed. A most essential information is that the cues are somehow indicative for presence of a road work area. Further details are given below. A common processing is performed for all aboveidentified visual cues, i.e. common rules are applied for processing the above-identified visual cues. However, different cues can be processed differently and have a different impact on the calculation of the confidence level.

Identifying presence of a road work area based on the confidence level being above a threshold value indicative for driving in a road work area comprises a suitable definition of the threshold value. The threshold value can be fix or it can be modified e.g. for different driving conditions or based on environment conditions like lighting conditions.

The applicable speed limit can be determined based on the identified presence of the road work area. This includes application of the applicable speed throughout the road work area as well as after having passed the road work area. The applicable speed limit can be based on a detection of a respective traffic sign, on general driving rules for driving in road work areas, on a pre-defined value for driving in road work areas, or others.

The intelligent speed assistance system is a driving support system for determining an applicable speed limit for a vehicle. The intelligent speed assistance system assists a human driver of the vehicle by informing in respect to the currently applicable speed limit, or it applies the applicable speed limit by itself, i.e. the intelligent speed assistance system limits the velocity of the vehicle based on the currently applicable speed limit.

The general operation of the intelligent speed assistance system is based on a front environment sensor, which is used to detect traffic signs with applicable speed limits and possibly other features such driving lanes ahead of the vehicle. The currently applicable speed limit is determined based on the speed limit indicated by the respected traffic signs. Additionally, the intelligent speed assistance system comprises a map data service containing speed limit information. The map data with the speed limit information may either be stored in the vehicle, i.e. a navigation system, or in a remote cloud server, which provides the map data including the speed limit information for a current location of the vehicle. The current location of the vehicle is determined based on reception of signals from a global navigation satellite system (GNSS). Several GNSS are known and can be used alone or in combination. Hence, the intelligent speed assistance system receives position information from a receiver for GNSS signals located in the vehicle.

The receiver for GNSS signals can be part of the intelligent speed assistance system, or it can be a separate component of the vehicle. In general, the above method does not require a full intelligent speed assistance system, since the method focuses on identification of road work areas. Hence, it can be implemented as an individual system or as a subsystem of the full intelligent speed assistance system.

The intelligent speed assistance system comprises a control unit, which receives the sensor information from the at least one environment sensor. The control unit and the at least one environment sensor are connected via a data connection, and the control unit receives the sensor information from the at least one environment sensor via the data connection. The data connection can be a point-to-point connection between the control unit and the at least one environment sensor. However, the data connection between the control unit and the at least one environment sensor can be implemented as a data bus, which can be used uniquely by the control unit and the at least one environment sensor, or the data bus can be used by different devices of the vehicle. Different types of data busses like CAN, LIN, LON, and FlexRay are known and used in the automotive area.

According to a modified embodiment of the invention, receiving sensor information from the at least one environment sensor at least partially covering the environment of the vehicle comprises receiving sensor information from at least one optical camera. Optical cameras are very suitable for determining the environment of the vehicle, since they provide a high level of detail for reliably identifying objects in the surrounding of the vehicle, thereby enabling reliable identification of the visual cues indicative for the presence of the road work area. Pictures can be provided from the optical cameras, either as part of a video frame or as individual pictures, as sensor information. Processing of such pictures is widely known in the automotive area. Furthermore, optical cameras are widely available for use in automotive applications like driving support systems.

Alternatively or additionally, other kinds of environment sensors can be used to provide the sensor information. For example, in some cases, LiDAR-based environment sensors can provide a sufficient level of detail for identifying the visual cues, even though the sensor information is provided in a different way, i.e. as a point cloud compared to a picture. According to a modified embodiment of the invention, identifying visual cues indicative for presence of a road work area comprises identifying traffic signs explicitly indicative for presence of a road work area, and/or identifying visual cues indicative for presence of a road work area comprises identifying traffic signs implicitly indicative for presence of a road work area including traffic signs indicating to perform a lane change, indicating a narrowing road, indicating a preference for oncoming traffic, indicating a preference for ongoing traffic, indicating a deviation, and indicating a reduction of a driving lane width, and/or identifying visual cues indicative for presence of a road work area comprises identifying items implicitly indicative for presence of a road work area including traffic cones, road work poles, direction boards, and road work equipment. Each of the types of visual cues, i.e. the traffic signs explicitly indicative for presence of a road work area, the traffic signs implicitly indicative for presence of a road work area, and the items implicitly indicative for presence of a road work area, can indicate presence of a road work area and together provide a reliable identification of the presence of the road work area. However, further types of visual cues as well as further visual cues of each of the types of visual cues can be identified and considered for identifying presence of a road work area.

According to a modified embodiment of the invention, identifying visual cues indicative for presence of a road work area based on the received sensor information comprises identifying positions of the visual cues indicative for presence of the road work area, and calculating a confidence level for presence of a road work area based on the identified visual cues comprises calculating the confidence level for presence of a road work area under additional consideration of the positions of the visual cues. Based on the positions of the visual cues, it can be determined if the visual cues are e.g. too far away and can be discarded. In particular, a lateral distance of the visual cues can be determined to identify, if the visual cues belong to the own driving direction or to the opposite driving direction. A longitudinal distance, i.e. a distance ahead of or behind the vehicle, can indicate if the cues are to far away to identify a current road work area. Furthermore, importance of visual cues can be weighted based on their positions, e.g. based on their distance ahead of the vehicle/behind the vehicle. This includes that the visual cues can be weighted based on their lateral positions, e.g. based on their distance besides the vehicle. According to a modified embodiment of the invention, calculating a confidence level for presence of a road work area based on the identified visual cues comprises increasing the confidence level based on newly identified visual cues and/or decreasing the confidence level based on visual cues no longer identified. Hence, the visual cues are accumulated over time and their identification increases the confidence level. Furthermore, the visual cues expire when they are no longer identified, so that the confidence level decreases. Overall, only those visual cues are considered, that are currently visible.

According to a modified embodiment of the invention, decreasing the confidence level based on visual cues no longer identified comprises decreasing the confidence level based on visual cues no longer identified with a delay based on a time and/or a distance travelled after identification of the visual cues and/or after the visual cues are no longer identified. The delay as well as the distance add a margin for expiry of cues, which are no longer visible for the at least one environment sensor. This can in particular be helpful, when approaching visual cues ahead of the vehicle are identified, e.g. based on a front environment sensor like a front camera of the vehicle with a view field in a direction ahead of the vehicle. In this case, the vehicle can still be in a road work area even when the visual cues indicative for presence of a road work area are no longer identified in the sensor information from the front environment sensor. Hence, no additional rear environment sensor is required to monitor, when the visual cues are left behind the vehicle.

According to a modified embodiment of the invention, calculating a confidence level for presence of a road work area based on the identified visual cues comprises assigning a weight to each of the identified visual cues, in particular depending on a type of each of the identified visual cues, and calculating the confidence level for presence of a road work area under consideration of the weights assigned to the identified visual cues. Some of the visual cues are considered as more indicative in respect to presence of a road work area than others. Hence, such more indicative visual cues can have a higher weight than other visual cues, which are considered as less indicative in this respect. The weights can be assigned individually to the different visual cues. However, a simpler approach consists in forming different weight groups and assigning the different visual cues to the different weight groups. Apart from the type of the identified visual cues, also positions of the identified visual cues can be considered for calculating the weight. Hence, objects further away can have assigned lower weights than objects closer by. This can refer to a lateral distance as well as a longitudinal distance. The positions of the identified visual cues can be considered for calculating the weight instead of the type of the identified visual cues or together with the type of the identified visual cues.

According to a modified embodiment of the invention, calculating a confidence level for presence of a road work area based on the identified visual cues comprises calculating a number of the identified visual cues. The higher the number of visual cues indicative for presence of a road work area, the higher the confidence in respect to presence of a road work area. Hence, objects unintentionally remaining from a former road work area, do not affect correct identification of presence of a road work area. Also, in cases some objects like traffic cones are used e.g. to mark a potholes or other damages to the road. However, since the calculation of the confidence level for presence of a road work area depends on identification of multiple visual cues indicative for presence of a road work area, also such a usage of the traffic cones does not affect correct identification of presence of a road work area.

According to a modified embodiment of the invention, calculating a confidence level for presence of a road work area based on the identified visual cues comprises increasing the confidence level to a maximum value upon detection of an explicit visual cue indicative for a start of a road work area, and/or calculating a confidence level for presence of a road work area based on the identified visual cues comprises decreasing the confidence level to a minimum value upon detection of an explicit visual cue indicative for end of a road work area. Hence, some explicit visual cues can be so strongly and highly indicative for presence of a road work area, that the detection of these explicit visual cues indicative for presence and/or end of a road work area can lead to an identification of presence and/or end of a road work area.

According to a modified embodiment of the invention, identifying presence of a road work area based on the confidence level being above a threshold value indicative for driving in a road work area comprises applying a hysteresis for the threshold value. The hysteresis refers to a different threshold for an increasing and a decreasing confidence level in order to avoid unstable conditions, where the confidence level is close to the threshold value. Hence, identification of presence of a road work area requires the confidence level being above the threshold value indicative for driving in a road work area, and based on this positive identification of presence of a road work area, switching back to no road work area present requires the confidence level falling below the threshold value minus a hysteresis delta. Accordingly, frequent toggling between positive and negative identification of presence of a road work area can be avoided.

According to a modified embodiment of the invention, the method comprises additional steps of storing a value of a previously applicable speed limit ahead of the identified road work area, and applying the stored value of the previously applicable speed limit after leaving the road work area. According to most national laws, speed limits applicable in road work areas are not applicable anymore after having passed the road work areas. In case a new applicable speed limit is identified, this new speed limit can be applied. However, in case no new applicable speed limit can be identified, the previously applicable speed limit ahead of the identified road work area can be used and applied after leaving the road work area. Hence, general speed limit settings, which are temporarily overturned by the road work area, can be applied. The value of the previously applicable speed limit ahead of the identified road work area can be stored upon identification of presence of a road work area. However, it is also possible to store the previously applicable speed limit ahead of the identified road work area continuously, and to stop storing temporarily applicable speed limits due to the road work area.

In an alternative embodiment, when the vehicle leaves the road work area, the current speed limit can be applied based a map data service containing the speed limit information. In a further alternative embodiment, when the vehicle leaves the road work area, the intelligent speed assistance system can turn back to normal operation, in particular decreasing a confidence given to the speed limit detected by the front camera of the vehicle and increasing the confidence given to the map data service containing the speed limit information.

According to a modified embodiment of the invention, the intelligent speed assistance system comprises at least one environment sensor at least partially covering the environment of the vehicle. Hence, the at least one environment sensor can be part of the intelligent speed assistance system and provide the sensor information at least primarily to a control unit of the intelligent speed assistance system. However, the intelligent speed assistance system or the at least one environment sensor itself can provide the sensor information to further driving support systems of the vehicle. In each case, the intelligent speed assistance system can perform the above method without technical differences, independently from the at least one environment sensor being part of it or not. In case the vehicle is equipped with multiple environment sensor, at least one of the environment sensors can be part of the intelligent speed assistance system, while at least another one of the environment sensors is not part of the intelligent speed assistance system.

According to a modified embodiment of the invention, the at least one environment sensor is provided as at least one optical camera. Optical cameras are very suitable for determining the environment of the vehicle, since they provide a high level of detail for reliably identifying objects in the surrounding of the vehicle, thereby enabling reliable identification of the visual cues indicative for the presence of the road work area. Optical cameras are widely available for use in automotive applications like driving support systems.

Alternatively, also other kinds of environment sensors can be used to provide the sensor information. For example, in some cases, LiDAR-based environment sensors can provide a sufficient level of detail for identifying the visual cues.

Feature and advantages described above with reference to the inventive method apply equally to the inventive driving support system and vice versa.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. Individual features disclosed in the embodiments can constitute alone or in combination an aspect of the present invention. Features of the different embodiments can be carried over from one embodiment to another embodiment.

In the drawings:

Fig. 1 shows a schematic view of a vehicle with an intelligent speed assistance system, an optical camera and a receiver for receiving signals from a global navigation satellite system, which are connected to each other via a data connection according to a first, preferred embodiment, Fig. 2 a schematic view showing a picture provided from the optical camera with a first driving scene on a road with a road work area ahead of the vehicle in accordance with the first embodiment,

Fig. 3 a schematic view showing a picture provided from the optical camera with a second driving scene on a road with a road work area ahead of the vehicle in accordance with the first embodiment,

Fig. 4 a schematic view showing a picture provided from the optical camera with a third driving scene with road work poles as visual cues indicative for presence of a road work area,

Fig. 5 a schematic view of a traffic cone as visual cue indicative for presence of a road work area,

Fig. 6 a schematic view showing a picture provided from the optical camera with a fourth driving scene with a trailer used in road work areas carrying a traffic sign indicating to perform a lane change as visual cue indicative for presence of a road work area,

Fig. 7 a schematic view of a traffic sign indicating road work ahead as visual cue indicative for presence of a road work area,

Fig. 8 a schematic view of mobile driving barrier with a traffic sign indicating road work as visual cue indicative for presence of a road work area,

Fig. 9 a schematic view of a traffic sign indicating road work as visual cue indicative for presence of a road work area,

Fig. 10 a schematic view of a traffic sign indicating an end of road work area as visual cue indicative for end of a road work area, Fig. 11 a schematic view of a traffic sign indicating an end of road work area as visual cue indicative for end of a road work area,

Fig. 12 a schematic view of a traffic sign indicating to perform a lane change as visual cue indicative for a road work area,

Fig. 13 a schematic view of a traffic sign indicating a narrowing road as visual cue indicative for a road work area,

Fig. 14 a schematic view of a traffic sign indicating an end of all road bans as visual cue for a negative indication for a road work area,

Fig. 15 a schematic view of two different curves for weighing visual cues as indications for a road work area based on their distance to the vehicle, and

Fig. 16 a flow chart of a method for determining an applicable speed limit for the vehicle of figure 1 based on a perception of the environment of the vehicle with the optical camera, which partially covers the environment of the vehicle, in accordance with the first embodiment.

Figure 1 shows a vehicle 10 with an intelligent speed assistance system 12 according to a first, preferred embodiment. The vehicle 10 can be any kind of vehicle 10.

The intelligent speed assistance system 12 is a driving support system for determining an applicable speed limit for the vehicle 10. The intelligent speed assistance system 12 assists a human driver of the vehicle 10 by informing in respect to the currently applicable speed limit, or it applies the applicable speed limit by itself, i.e. the intelligent speed assistance system 12 limits the velocity of the vehicle 10 based on the currently applicable speed limit.

The vehicle 10 further comprises an environment sensor 14, which is an optical camera 14 in this embodiment. The optical camera 14 partially covers an environment 16 of the vehicle 10. The optical camera 14 is provided as front camera behind a windscreen of the vehicle 10. The optical camera 14 provides sensor information covering the environment, i.e. the optical camera 14 provides images, which show a part of the environment 16 of the vehicle 10.

According to the first embodiment, the intelligent speed assistance system 12 comprises a control unit 18. The control unit 18 is connected via a data connection 20 to the optical camera 14. The data connection 20 is implemented in this embodiment as a data bus 20, which can be used by different devices of the vehicle 10. Different types of data busses 20 like CAN, LIN, LON, and FlexRay are known and used in the automotive area. The optical camera 14 provides the sensor information covering the environment 16, i.e. the images, which show a part of the environment 16 of the vehicle 10, via the data bus 20.

In this embodiment, the optical camera 14 is provided independently from the intelligent speed assistance system 12. In an alternative embodiment, the optical camera 14 can be part of the intelligent speed assistance system 12.

The intelligent speed assistance system 12 implements a general operation for determining a currently applicable speed limit for the vehicle 10. The operation of the intelligent speed assistance system 12 is based on the optical camera 14 provided at the front of the vehicle 10, which is used to detect traffic signs with applicable speed limits and possibly other features such driving lanes ahead of the vehicle 10. Operation of such intelligent speed assistance systems 12 is generally known. Additionally, the intelligent speed assistance system 12 comprises a map data service containing speed limit information. The map data with the speed limit information of this embodiment is stored in the vehicle 10, i.e. in a navigation system of the vehicle 10. In order to determine a current location of the vehicle 10, the vehicle 10 comprises a receiver 22 for signals from a global navigation satellite system (GNSS). The receiver 22 for the GNSS signals can be part of the intelligent speed assistance system 12, or it can be a separate component of the vehicle 10, like in the present embodiment.

The intelligent speed assistance system 12 performs a method for determining an applicable speed limit for the vehicle 10 based on a perception of the environment 16 of the vehicle 10 with the optical camera 14, which partially covers the environment 16 of the vehicle 10. A flow chart of the method is shown in figure 16. The method will be described below with additional reference to figures 2 to 15.

The applicable speed limit is a current speed limit, which is currently applicable for the vehicle 10. It can be based on explicit rules/traffic signs as well as on implicit rules/traffic signs. Furthermore, the current speed limit can be derived from traffic law, which is typically national law, and which may specify e.g. a speed limit for driving on different kinds of roads and/or for driving in a town. The presence of a road work area 24 may overrule such speed limits at least temporarily, so that a different speed limit becomes applicable while driving through the road work area 24.

The method starts with step S100, which refers to receiving sensor information 30 from the optical camera 14, which partially covers the environment 16 of the vehicle 10 in driving direction 26 ahead of the vehicle 10.

In this embodiment, with the optical camera 14 as environment sensor 14, the sensor information 30 is provided as visual pictures 30 taken by the optical camera 14. Different pictures 30 are shown by way of example in figures 2, 3,4 and 6. Hence, receiving the sensor information 30 from the optical camera 14 refers to a transfer of the picture(s) 30 from the optical camera 14 to the control unit 18 for processing the received picture(s) 30. The pictures 30 can be provided from the optical camera 14 either as part of a video frame or as individual pictures 30.

By way of example, the picture of figure 2 shows a driving scene with the vehicle 10 approaching a road work area 24. The vehicle 10 is driving on a road 32 with two driving lanes 34 for each driving direction 26. An emergency lane 36 is located at each side of the road 32 next to the outer driving lane 34. The outer driving lane 34 is separated from the emergency lane 36 by a dashed line 38. The road 32 is laterally delimited by a guard rail 40, which is provided next to the emergency lane 36. The driving lanes 34 for the two driving directions 26 are separated by a median strip 42.

The road work area 24 results in a limitation of available driving lanes 34 and a deviation of traffic to a single driving lane 34 originally provided for oncoming traffic, i.e. traffic in a direction opposite to the driving direction 26 of the vehicle 10. The road work area 24 is marked by traffic cones 44 and an additional line marking 46 on the road 32. Additional traffic signs 48 are provided in the context of the road work area 24 including a traffic sign 48a indicating to perform a lane change and a traffic sign 48b indicating a speed limit applicable in the road work area 24. The scene of figure 2 additionally shows a mobile street light 50 for illuminating the road work area 24. A third party vehicle 52 indicates traffic in the road work area 24.

By way of example, the picture 30 of figure 3 shows a driving scene similar to that of figure 2 with the vehicle 10 approaching a road work area 24. Also in this case, the road work area 24 results in a limitation of available driving lanes 34 and a deviation of traffic to a single driving lane 34, which in this case is originally designated to driving in the driving direction 26 of the vehicle 10. The road work area 24 is marked in figure 3 additionally by road work poles 54 and traffic signs 48c indicating a deviation.

By way of example, the picture 30 of figure 4 shows a driving scene with multiple road work poles 54 marking the road work area 24.

By way of example, the picture 30 of figure 6 shows a driving scene with a road work area 24, which is marked in accordance with the driving scenes of figures 2 und 3 with multiple traffic cones 44. Additionally, a trailer 56 used in road work areas 24 carrying a traffic sign 48a indicating to perform a lane change is located in the road work area 24.

Step S110 refers to identifying visual cues 44, 48, 50, 54, 56, 58, 60 indicative for presence of a road work area 24 based on the received sensor information 30, i.e. the received picture 30.

In detail, the control unit 18 processes the pictures 30 received from the optical camera 14 with an image processing algorithm to identify the visual cues 44, 48, 50, 54, 56, 58, 60 indicative for presence of a road work area 24. This can include identifying visual cues 44, 48, 50, 54, 56, 58, 60, which are counter-indicative for presence of a road work area 24, e.g. the visual cues 44, 48, 50, 54, 56, 58, 60 can provide a positive or a negative indication for presence of a road work area 24. The visual cues 44, 48, 50, 54, 56, 58, 60 indicative for presence of a road work area 24 are identified and marked for further processing within all objects identified in the respective picture(s) 30. Identifying the visual cues 44, 48, 50, 54, 56, 58, 60 indicative for presence of a road work area 24 comprises identifying traffic signs 48 explicitly indicative for presence of a road work area 24, i.e. traffic signs 48c indicating a road work area 24, which are shown in figures 7, 8 and 9. Identifying the visual cues 44, 48, 50, 54, 56, 58, 60 indicative for presence of a road work area 24 further comprises identifying traffic signs 48 implicitly indicative for presence of a road work area 24. This includes traffic signs 48a indicating to perform a lane change as shown in Fig. 12, traffic signs 48d indicating a narrowing road as shown in figure 13, traffic signs 48 indicating a preference for oncoming traffic, traffic signs 48 indicating a preference for ongoing traffic, traffic signs 48 indicating a deviation, and traffic signs 48 indicating a reduction of a driving lane width. Identifying the visual cues 44, 48, 50, 54, 56, 58, 60 indicative for presence of a road work area 24 still further comprises identifying items implicitly indicative for presence of a road work area 24 including traffic cones 44, one of which is shown alone by way of example in figure 5, road work poles 54, which can be seen by way of example in figure 4, direction boards 58, which can be seen in figure 3, driving barriers 58 like the one carrying the traffic signs 48c indicating a road work area 24 in figure 8, and road work equipment like the light post 50 of figure 2 or the trailer 56 of figure 6.

Similarly, also the traffic signs 48e indicating an end of road work area 24 in figures 10 and 11 as well as the traffic sign 48f indicating an end of all road bans area in figure 14 can be identified as visual cues for indicating a road work area 24.

However, further types of visual cues 44, 48, 50, 54, 56, 58, 60 as well as further visual cues 44, 48, 50, 54, 56, 58, 60 of each of the types of visual cues 44, 48, 50, 54, 56, 58, 60 can be identified and considered for identifying presence of a road work area 24.

In this embodiment, identifying the visual cues 44, 48, 50, 54, 56, 58, 60 indicative for presence of a road work area 24 comprises identifying positions of the visual cues 44, 48, 50, 54, 56, 58, 60 indicative for presence of the road work area 24.

Step S120 refers to calculating a confidence level for presence of a road work area 24 based on the identified visual cues 44, 48, 50, 54, 56, 58, 60.

The confidence level for presence of a road work area 24 is calculated based on the identified visual cues 44, 48, 50, 54, 56, 58, 60, so that the confidence level for presence of a road work area 24 is based on a calculation depending on a number of identified visual cues 44, 48, 50, 54, 56, 58, 60. In addition, the visual cues 44, 48, 50, 54, 56, 58, 60 have assigned particular weights depending on whether they are highly indicative or less indicative for presence of a road work area 24. The weights are typically defined by an expert. By way of example, some weights can be defined as traffic sign 48a indicating to perform a lane change having a weight of 0.2, a traffic sign 48c indicating a road work area 24 having a weight of 0.5, a traffic cone 44 having a weight of 0.05 and road work poles 54 having a weight 0.05.

Furthermore, the calculation of the confidence level for presence of a road work area 24 based on the identified visual cues 44, 48, 50, 54, 56, 58, 60 is performed under additional consideration of the positions of the visual cues 44, 48, 50, 54, 56, 58, 60. Figure 15 shows two curves 62, 64 for adapting the weights of the visual cues 44, 48, 50, 54, 56, 58, 60 depending on their distances from the vehicle 10. As can be seen in figure 14, visual cues 44, 48, 50, 54, 56, 58, 60 further away than a distance threshold are discarded based on each of the two curves 62, 64. A linear adaption curve 62 linearly increases until a confidence factor of 1 has been reached at a minimum distance. A signoid adaption curve 64 also increases from the distance threshold until the confidence factor of 1 has been reached at a minimum distance.

Furthermore, as the visual cues 44, 48, 50, 54, 56, 58, 60 are identified based on the visual camera 14, which is directed to a front side of the vehicle 10, the visual cues 44, 48, 50, 54, 56, 58, 60 can already be out of a field of view of the visual camera 14 before the vehicle has passed the road work area 24. Hence, when the visual cues 44, 48, 50, 54, 56, 58, 60 have been identified, they are added to the calculation of the confidence level for presence of a road work area 24, so that the confidence level increases based on the identified visual cues 44, 48, 50, 54, 56, 58, 60. To compensate for the field of view of the visual camera 24 not covering the visual cues 44, 48, 50, 54, 56, 58, 60 besides the vehicle, the visual cues 44, 48, 50, 54, 56, 58, 60 are discarded from the calculation of the confidence level with a delay, which can be by way of example three seconds, after identification of the visual cues 44, 48, 50, 54, 56, 58, 60 and/or after the visual cues 44, 48, 50, 54, 56, 58, 60 are no longer identified in the pictures 30 provided from the optical camera 14. Hence, the confidence level decreases based on visual cues 44, 48, 50, 54, 56, 58, 60 no longer identified. Some of the visual cues 44, 48, 50, 54, 56, 58, 60 can be highly indicative for presence of a road work area 24, i.e. for the beginning and/or the end of the road work area 24. Accordingly, in an alternative embodiment, in case such a highly indicative visual cue 44, 48, 50, 54, 56, 58, 60 has been identified based on the picture 30, the calculation of the confidence level for presence of a road work area 24 is increased to a maximum value upon detection such a visual cue indicative for a beginning of a road work area 24. The same applies to visual cues 44, 48, 50, 54, 56, 58, 60 highly indicative for an end of a road work area 24.

Step S130 refers to identifying presence of a road work area 24 based on the confidence level being above a threshold value indicative for driving in a road work area 24. The identification provides as identification result, if a road work area 24 is present in the environment 16 of the vehicle 10 or not.

Identifying presence of a road work area 24 based on the confidence level being above the threshold value indicative for driving in a road work area 24. This comprises a suitable definition of the threshold value, typically together with the weights assigned to the visual cues 44, 48, 50, 54, 56, 58, 60. The threshold value can be fix or it can be modified e.g. for different driving conditions or based on environment conditions like lighting conditions.

Furthermore, a hysteresis is applied for the threshold value for identifying presence of a road work area 24 compared to the confidence level. The hysteresis refers to a different threshold for an increasing and a decreasing confidence level in order to avoid unstable conditions, where the confidence level is close to the threshold value. Hence, identification of presence of a road work area 24 requires the confidence level being above the threshold value indicative for driving in a road work area 24, and based on this positive identification of presence of a road work area 24, switching back to no road work area 24 present requires the confidence level falling below the threshold value minus a hysteresis delta.

Step S140 refers to storing a value of a previously applicable speed limit ahead of the identified road work area 24. Hence, the applicable speed limit before entering the road work area 24 is stored, e.g. in a storage of the control unit 18. The value of the previously applicable speed limit ahead of the identified road work area 24 is stored upon identification of presence of a road work area 24 in preceding step S130. In an alternative embodiment, the previously applicable speed limit ahead of the identified road work area 24 is continuously stored, until the road work area 24 has been identified and the value of the previously applicable speed limit ahead of the identified road work area 24 remains in the storage while passing the road work area 24.

Step S150 refers to determining the applicable speed limit based on the identified presence of the road work area 24.

The applicable speed limit is the speed limit applicable in the road work area 24. It can be based on a traffic sign 48 indicating a speed limit, on a general driving rule, i.e. a general speed limit for driving in a road work area 24.

Step S160 refers to applying the stored value of the previously applicable speed limit after leaving the road work area 24.

Hence, when the vehicle 10 leaves the road work area 24, as identified in step S140, and the intelligent speed assistance system 12 has not identified an applicable speed limit, the stored value of the previously applicable speed limit can be applied to the vehicle 10.

In an alternative embodiment, when the vehicle 10 leaves the road work area 24, as identified in step S140, the current speed limit can be applied based on the map data service containing the speed limit information. In a further alternative embodiment, when the vehicle 10 leaves the road work area 24, the intelligent speed assistance system 12 turns back to normal operation, in particular decreasing the confidence given to the speed limit detected by the optical camera 14 of the vehicle and increasing the confidence given to the map data service containing the speed limit information. Reference signs list

10 vehicle

12 intelligent speed assistance system

14 environment sensor, optical camera

16 environment

18 control unit

20 data connection, data bus

22 receiver

24 road work area

26 driving direction

30 sensor information, picture

32 road

34 driving lane

36 emergency lane

38 dashed line

40 guard rail

42 median strip

44 traffic cone, visual cue

46 line marking

48 traffic sign, visual cue

48a a traffic sign indicating to perform a lane change, visual cue

48b traffic sign indicating a speed limit, visual cue

48c traffic sign indicating a road work area, visual cue

48d traffic sign indicating a narrowing road, visual cue

48e traffic sign indicating an end of road work area, visual cue

48f traffic sign indicating an end of all road bans, visual cue

50 light post, visual cue

52 third party vehicle

54 road work pole, visual cue

56 trailer, visual cue

58 direction board, visual cue

60 driving barrier, visual cue

62 linear adaption curve

64 signoid adaption curve

Claims

Patent claims

1 . Method for determining an applicable speed limit for a vehicle (10) based on a perception of the environment (16) of the vehicle (10) with at least one environment sensor (14) covering the environment (16) of the vehicle (10), comprising the steps of receiving sensor information (30) from the at least one environment sensor (14) at least partially covering the environment (16) of the vehicle (10), identifying visual cues (44, 48, 50, 54, 56, 58, 60) indicative for presence of a road work area (24) based on the received sensor information (30), calculating a confidence level for presence of a road work area (24) based on the identified visual cues (44, 48, 50, 54, 56, 58, 60), identifying presence of a road work area (24) based on the confidence level being above a threshold value indicative for driving in a road work area (24), and determining the applicable speed limit based on the identified presence of the road work area (24).

2. Method according to claim 1 , characterized in that receiving sensor information (30) from the at least one environment sensor (14) at least partially covering the environment (16) of the vehicle (10) comprises receiving sensor information (30) from at least one optical camera (14).

3. Method according to any of claims 1 or 2, characterized in that identifying visual cues (44, 48, 50, 54, 56, 58, 60) indicative for presence of a road work area (24) comprises identifying traffic signs (48) explicitly indicative for presence of a road work area (24), and/or identifying visual cues (44, 48, 50, 54, 56, 58, 60) indicative for presence of a road work area (24) comprises identifying traffic signs (48) implicitly indicative for presence of a road work area (24) including traffic signs (48) indicating to perform a lane change, indicating a narrowing road, indicating a preference for oncoming traffic, indicating a preference for ongoing traffic, indicating a deviation, and indicating a reduction of a driving lane width, and/or identifying visual cues (44, 48, 50, 54, 56, 58, 60) indicative for presence of a road work area (24) comprises identifying items implicitly indicative for presence of a road work area (24) including traffic cones (44), road work poles (54), direction boards (58), and road work equipment.

4. Method according to any preceding claim, characterized in that identifying visual cues (44, 48, 50, 54, 56, 58, 60) indicative for presence of a road work area (24) based on the received sensor information (30) comprises identifying positions of the visual cues (44, 48, 50, 54, 56, 58, 60) indicative for presence of the road work area (24), and calculating a confidence level for presence of a road work area (24) based on the identified visual cues (44, 48, 50, 54, 56, 58, 60) comprises calculating the confidence level for presence of a road work area (24) under additional consideration of the positions of the visual cues (44, 48, 50, 54, 56, 58, 60).

5. Method according to any preceding claim, characterized in that calculating a confidence level for presence of a road work area based on the identified visual cues (44, 48, 50, 54, 56, 58, 60) comprises increasing the confidence level based on newly identified visual cues (44, 48, 50, 54, 56, 58, 60) and/or decreasing the confidence level based on visual cues (44, 48, 50, 54, 56, 58, 60) no longer identified.

6. Method according to claim 5, characterized in that decreasing the confidence level based on visual cues (44, 48, 50, 54, 56, 58, 60) no longer identified comprises decreasing the confidence level based on visual cues (44, 48, 50, 54, 56, 58, 60) no longer identified with a delay based on a time and/or a distance travelled after identification of the visual cues (44, 48, 50, 54, 56, 58, 60) and/or after the visual cues (44, 48, 50, 54, 56, 58, 60) are no longer identified.

7. Method according to any preceding claim, characterized in that calculating a confidence level for presence of a road work area (24) based on the identified visual cues (44, 48, 50, 54, 56, 58, 60) comprises assigning a weight to each of the identified visual cues (44, 48, 50, 54, 56, 58, 60), in particular depending on a type of each of the identified visual cues (44, 48, 50, 54, 56, 58, 60), and calculating the confidence level for presence of a road work area (24) under consideration of the weights assigned to the identified visual cues (44, 48, 50, 54, 56, 58, 60).

8. Method according to any preceding claim, characterized in that calculating a confidence level for presence of a road work area (24) based on the identified visual cues (44, 48, 50, 54, 56, 58, 60) comprises calculating a number of the identified visual cues (44, 48, 50, 54, 56, 58, 60).

9. Method according to any preceding claim, characterized in that calculating a confidence level for presence of a road work area (24) based on the identified visual cues (44, 48, 50, 54, 56, 58, 60) comprises increasing the confidence level to a maximum value upon detection of an explicit visual cue (44, 48, 50, 54, 56, 58, 60) indicative for a start of a road work area (24), and/or calculating a confidence level for presence of a road work area (24) based on the identified visual cues (44, 48, 50, 54, 56, 58, 60) comprises decreasing the confidence level to a minimum value upon detection of an explicit visual cue (44, 48, 50, 54, 56, 58, 60) indicative for end of a road work area (24).

10. Method according to any preceding claim, characterized in that identifying presence of a road work area (24) based on the confidence level being above a threshold value indicative for driving in a road work area (24) comprises applying a hysteresis for the threshold value.

11 . Method according to any preceding claim, characterized in that the method comprises additional steps of storing a value of a previously applicable speed limit ahead of the identified road work area (24), and applying the stored value of the previously applicable speed limit after leaving the road work area (24).

12. Intelligent speed assistance system (12) for determining an applicable speed limit for a vehicle (10), wherein the intelligent speed assistance system (12) is adapted to perform the method for determining an applicable speed limit for a vehicle (10) based on a perception of the environment (16) of the vehicle (10) with at least one environment sensor (14) covering the environment (16) of the vehicle (10) according to any of claims 1 to 11 .

13. Intelligent speed assistance system (12) according to claim 12, characterized in that the intelligent speed assistance system (12) comprises at least one environment sensor (14) at least partially covering the environment (16) of the vehicle (10).

14. Intelligent speed assistance system (12) according to claim 13, characterized in that the at least one environment sensor (14) is provided as at least one optical camera (14).