EP4132829A1

EP4132829A1 - Environmental sensor system

Info

Publication number: EP4132829A1
Application number: EP21716318.7A
Authority: EP
Inventors: Mena NAGIUB
Original assignee: Valeo Schalter und Sensoren GmbH
Current assignee: Valeo Schalter und Sensoren GmbH
Priority date: 2020-04-08
Filing date: 2021-03-30
Publication date: 2023-02-15
Also published as: DE102020109761A1; WO2021204586A1

Abstract

An environmental sensor system (1) comprises a sensor unit (2), which is configured to generate sets of sensor data (S1) depicting an environment for first frames and a second frame, as well as a computing unit (3), configured to apply a first evaluation algorithm to the respective set of sensor data (S1) and to generate a respective first intermediate output (S2). The computing unit (3) is configured to apply a second evaluation algorithm to the set of sensor data (S1) of the second frame, to generate a respective second intermediate output (S3) for the second frame, to examine, whether a predefined maximum level of computational effort has been exceeded for generating the respective first intermediate output (S2) and to generate a final output for the second frame based on the second intermediate output (S3) depending on a result of the examination.

Description

Environmental sensor system

The present invention relates to an environmental sensor system comprising an environmental sensor unit, which is configured to generate respective sets of sensor data depicting an environment of the environmental sensor unit for one or more consecutive first frames and for a second frame, the one or more first frames preceding the second frame, and a computing unit. The invention further relates to a corresponding method for operating an environmental sensor system, an electronic vehicle guidance system, a method for guiding a motor vehicle and a computer program product.

Evaluating and processing output data of active optical sensor systems, such as lidar systems, radar systems or camera systems, may require a large amount of computational resources such as computation time and memory space. Typically, the requirements increase with increasing complexity of the depicted scene in the environment.

On the other hand, in particular in the context of embedded systems as they are used for example in motor vehicles, for example for semi-automated or automated driving, computational resources may be limited. Consequently, it may happen that not all available sensor data may be processed within the required time. Thus, sensor data of individual frames or plurality of frames may be dropped. However, dropping the sensor data may result in a reduced reliability of the output of the environmental sensor system, For example, the reliability of object detection, object tracking, segmentation or classification tasks using the output of the environmental sensor system may be reduced. In consequence, this may lead to a reduced safety and/or comfort level for a user.

It is therefore an object of the present invention to provide an improved concept for an environmental sensor system and for operating such sensor system, which increases the reliability of output data generated by the environmental sensor system,

This object is achieved by the respective subject matter of the independent claims.

Further implementations and preferred embodiments are subject matter of the dependent claims. The improved concept is based on the idea to provide two different evaluation algorithms that can be applied to sensor data generated by an environmental sensor unit of the environmental sensor system. Depending on the actual computational effort, intermediate output of the first or of the second evaluation algorithm is used to generate a final output.

According to the improved concept, an environmental sensor system, in particular for a motor vehicle, comprising an environmental sensor unit is provided. The environmental sensor unit is configured to generate respective sets of sensor data depicting an environment of the environmental sensor unit for one or more consecutive first frames and for a second frame, wherein the one or more first frames precede the second frame. The environmental sensor system further comprises a computing unit, which is, in particular, coupled to the environmental sensor unit to receive the sets of sensor data for the first frames and the second frame. The computing unit is configured to apply, for each of the one or more first frames, a first evaluation algorithm to the respective set of sensor data and to generate a respective first intermediate output for the respective first frame of the one or more first frames. The computing unit is configured to examine, for each the one or more first frames, whether a predefined maximum level of computational effort has been exceeded for generating the respective first intermediate output. The computing unit is configured to apply a second evaluation algorithm to the set of sensor data of the second frame and to generate a respective second intermediate output for the second frame. The computing unit is configured to generate a final output for the second frame based on the second intermediate output of the second frame, depending on a result of the examination.

Here and in the following, an environmental sensor system can be understood as a sensor systems, which is able to generate sensor data or sensor signals, which depict, represent or image an environment of the sensor system, in the present case of the sensor unit or the motor vehicle. In particular, lidar systems, radar systems or camera systems may qualify as environment sensor systems.

Depending on the specific implementation of the environmental sensor system, also the implementation of the environmental sensor unit may differ. For example, in case the environmental sensor system is implemented as a camera system, the environmental sensor unit may comprise an imager chip and/or another array of light sensitive pixels or optical detectors. In case the environmental sensor system is implemented as a lidar system, the environmental sensor unit may for example comprise one or more light sources for emitting light and one or more optical detectors for detecting reflected portions of the emitted light.

In case the environmental sensor system is implemented as a radar system, the environmental sensor unit may for example comprise one or more transmission or receiving antennas and, for example, front end circuitry coupled for operating the antennas.

Since the one or more first frames precede the second frames, the second frame may for example follow directly after the one or more first frames.

The computing unit may for example comprise a memory unit storing the first and the second evaluation algorithm.

The computing unit may also comprise one or more processing modules to apply the first and the second algorithm to the respective sensor data.

Here and in the following, a module can be understood as a hardware module or as a software module. In particular, a module may also comprise a hardware and a software portion implemented on the hardware.

A software module may be understood as a portion of software code functionally connected and combined to a unit. A software module may comprise or implement several processing steps and/or data structures.

The first intermediate output of a given frame may be understood as a result of the first evaluation algorithm, when it is applied to the respective set of sensor data of that frame. The same holds analogously for the second intermediate output. The first and the second intermediate output may also be denoted as first and second intermediate output data, respectively.

The first and the second evaluation algorithm may for example have different performance properties, different functionalities and so forth. Therefore, the different algorithms may have different computational requirements, in particular regarding computation time and/or memory requirements. The environmental sensor system according to the improved concept uses the result of examining the level of computational effort to decide whether the second intermediate output is used to generate the final output for the second frame. In particular, it may be determined based on the result of the examination, whether the second intermediate output or a corresponding first intermediate output for the second frame is actually used to generate the final output. In this way, the computing unit may select the first or the second evaluation algorithm to be used for generating the final output in order to adapt the operation of the system better to the actual environmental scene. For example, in case of a particularly complex scene, typically giving rise to an increase in computational effort and, in particular, in computation time, a reduced performance or functionality may be accepted by selecting the second intermediate output for generating the final output. Instead, the system may benefit from a reduced amount of computational resources required by the second evaluation algorithm compared to the first evaluation algorithm.

In particular, the second intermediate output may be used to generate the final output in case the result of the examination indicates that the sensor data of the first frames corresponds to a particularly complex environmental scene. By using the second intermediate output to generate the final output depending on the result of the examination, it may be avoided that individual sets of sensor data, in particular the set of sensor data of the second frame, are dropped in case the first evaluation algorithm cannot appropriately handle the sensor data for a complex scene. Consequently, by means of the improved concept, the potential drop rate of sensor data may be reduced, which increases the reliability of the final output.

In particular, the improved concept allows for a dynamic and situation specific tradeoff between maximized performance and functionality by using the first intermediate output on the one hand and reduced risk of data drop by using the second intermediate output.

According to several implementations of the environmental sensor system, the second evaluation algorithm is designed to require less computational resources than the first evaluation algorithm. Therein, the computational resources may for example comprise computation time or processing time and/or memory space, in particular permanently available memory space or memory space for working data, for example RAM memory.

In other words, for the same set of input data or sensor data, respectively, the second evaluation algorithm requires less computational resources to generate the respective second intermediate output than the first evaluation algorithm requires for generating corresponding first intermediate output.

In particular, in view of a predefined set of key performance indicators, KPIs, the second evaluation algorithm may not fulfill as many KPIs as the first evaluation algorithm does for the same set of input data or sensor data.

In other words, the reduced performance or functionality of the second evaluation algorithm may be accepted in case the result of the examination indicates a high workload of the first evaluation algorithm, for example in case of a particularly complex scene.

For example, the second evaluation algorithm may operate at a lower accuracy than the first evaluation algorithm.

The second algorithm may for example make use of less accurate computing schemes or mathematical models to generate the second intermediate output compared to the first algorithm to generate the first intermediate output.

The second evaluation algorithm may also operate at a reduced resolution compared to the first evaluation algorithm.

For example, the second evaluation algorithm may filter the input sensor data in a different manner than the first evaluation algorithm does. In consequence, the second evaluation algorithm may effectively have to handle a reduced amount of input data.

In this way, it may be achieved that, in case the first evaluation algorithm is not able to meet requires time limits, the second evaluation algorithm may still be able to meet the required time limits. According to several implementations, the computing unit, in particular an evaluation module of the computing unit, is configured to determine, for each of the one or more first frames, whether a respective predefined timing requirement is met in order to examine whether the respective maximum level of computational effort has been exceeded.

Therein, the timing requirement can be understood to be met in case the first evaluation algorithm has finished to generate the first intermediate output before a corresponding individual time limit has ended.

In other words, in case the timing requirement is met, the respective maximum level of computational effort has not exceeded.

On the other hand, in some implementations, when the timing requirements have not been met, the respective level of computational effort may have been exceeded.

However, in other implementations, further conditions may apply for the maximum level of computational effort to be exceeded.

According to several implementations, the computing unit is configured to determine a predefined level of complexity of the depicted environment for each of the one or more first frames and/or for the second frame. The computing unit is configured to generate the final output for the second frame depending on the determined levels of complexity, in particular depending on all determined levels of complexity.

In particular, the computing unit is configured to generate the final output for the second frame based on the first intermediate output of the second frame or based on the second intermediate output of the second frame, depending on the determined levels of complexity.

The predefined level or complexity may for example be given by a number of identified segments in the depicted environment. Therein, the number of segments may correspond to a result of a segmentation algorithm or a semantic segmentation algorithm, which may for example by comprised by the first and/or the second evaluation algorithm.

For example, the level of complexity may correspond or may be correlated to a number of objects that are identified in the depicted environment. For example, the sensor data may comprise a respective object list. In this case, the predefined level of complexity may correspond to a number of objects in the object list.

For example, the sensor data may comprise a point cloud, for example a lidar point cloud. Then, a segment may correspond to a clusters of correlated points in the point cloud. The predefined level of complexity may correspond to the number of segments in this case.

For example, the sensor data may comprise camera data including respective individual pixel values. A segment may correspond to a clusters of correlated pixels. In this case, the number of segments may correspond to the predefined level of complexity.

In particular, the computing unit may be configured to generate the final output for the second frame depending on the determined levels of complexity and on the result of the examination.

The level of complexity may also be understood a part of the computational effort or may contribute to the level of computational effort. In other words, examining whether the maximum level of computational effort has been exceeded, may explicitly or implicitly comprise determining the level of complexity.

In particular, a higher level of complexity may be a significant contribution or the only significant contribution for an increased computational effort. Therefore, by taking into account the level of complexity for generating the final output, particularly reliably results may be achieved.

According to several implementations, the computing unit is configured to apply the first evaluation algorithm to the set of sensor data of the second frame and to generate a respective first intermediate output for the second frame. The computing unit is configured to generate the final output either based on the first intermediate output of the second frame or based on the second intermediate output of the second frame, depending on a result of the examination.

According to several implementations, the computing unit is configured to operate in a normal operation mode or in a restricted operation mode, depending on the result of the examination. The computing unit is configured to generate the final output based on the first intermediate output of the second frame, when the computing unit is operating in the normal operation mode and based on the second intermediate output of the second frame, when the computing unit is operating in the restricted operation mode.

In particular, the first and the second evaluation algorithm may be continuously applied to each frame of a sequence of frames comprising the one or more first frames and the second frame. The computing unit may determine, based on the result of the examination, which intermediate output is used to generate the final output.

Such implementations allow for a particularly fast reaction to a changing environment.

In particular, if the final output is generated based on the first intermediate output, it is not generated based on the second intermediate output or it is generated independent of the second intermediate output. On the other hand, in case the final output is generated based on the second intermediate output, it is not generated based on the first intermediate output or it is generated independent of the first intermediate output.

According to the actual requirements, the computing unit may switch from the first intermediate output to the second intermediate output to immediately reduce a risk of data drop.

In particular, the computing unit may be configured to operate in the normal operation mode or the restricted operation mode further depending on a result of a further examination, whether the maximum level of computation effort has been exceeded for generating the respective first intermediate output of the second frame.

Operating in the normal operation mode or in the restricted operation mode may be understood such that the respected operation mode is activated or stays activated.

According to several implementations, the computing unit is configured to determine for each of one or more third frames after the second frame, whether the predefined maximum level of computational effort has been exceeded for generating a respective second intermediate output by applying the second evaluation algorithm and switch from the restricted operation mode to the normal operation mode or to a suspended mode depending on a result of the further examination. In the suspended mode, the computing unit may for example generate the final output neither based on the first intermediate output nor based on the second intermediate output. In particular, the computing unit may then not generate the final output at all.

According to several implementations, the computing unit comprises an evaluation module, which is configured to carry out the examination, whether the predefined maximum level of computational effort has been exceeded. The evaluation module is configured to generate a trigger signal depending on the result of the examination. The computing unit comprises a multiplexer module, which is configured to select the second intermediate output of the second frame for generating the final output of the second frame, depending on the trigger signal.

According to several implementations, the computing unit is configured to generate the final output for the second frame based on the second intermediate output of the second frame, only if the result of the examination indicates that the respective maximum level of computational effort has been exceeded for at least one of the one or more first frames.

In other words, the result of the examination indicating that the maximum respective level of computational effort has been exceeded for at least one of the one or more first frames may be considered as a necessary condition for generating the final output based on the second intermediate output.

In case the maximum level of computational effort has been exceeded for at least one of the one or more first frames, this may be considered as an indication that the first algorithm cannot fully handle the complexity of the scene.

According to several implementations, the computing unit is configured to generate the final output for the second frame based on the second intermediate output of the second frame only if the result of the examination indicates that the respective maximum level of computational effort has been exceeded for at least a predefined threshold number of frames of the one or more first frame.

According to several implementations, the computing unit is configured to generate the finale output for the second frame based on the first intermediate output of the second frame, if the result of the examination indicates that the respective maximum level of computational effort has not been exceeded for any of the one or more first frames. In other words, the result of the examination indicating that the respective maximum level of computational effort has not been exceeded for any of the one or more first frames may be considered as a sufficient condition for generating the final output based on the first intermediate output.

In this way, it may be ensured that the maximum performance and functionality is kept as long as possible.

According to several implementations, the computing unit is configured to further examine, whether the maximum level of computational effort has been exceeded for generating the respective first intermediate output of the second frame and to generate the final output for the second frame based on the first intermediate output of the second frame or based on the second intermediate output of the second frame, depending on the result of the further examination.

In particular, the computing unit may be configured to examine whether a respective further timing requirement is met in order to further examine whether the maximum level for computational effort has been exceeded for generating the respective first intermediate output of the second frame.

In this way, a historic and an actual degree of complexity, in other words the complexity according to the first frames and according to the second frame, is taken into account. In this way, an unnecessary reduction of performance or functionality may be avoided.

According to several implementations, the computing unit is configured to generate an information signal, wherein the information signal indicates, whether the final output for the second frame is or has been generated based on the first intermediate output of the second frame or based on the second intermediate output of the second frame.

In other words, the information signal may indicate, based on which one of the first and the second intermediate output the final output for the second frame is generated.

The information signal may for example be used in subsequent processes or tasks using the final output data. In particular, functionalities may be adapted accordingly. For example, in case of an environmental sensor system of a motor vehicle, a level of automation may be adapted or reduced based on the information signal. In this way, an improved level of safety may be achieved. According to several implementations, generating the final output for the second frame based on the second intermediate output comprises using the second intermediate output as the final output for the second frame.

According to several implementations, generating the final output for the second frame based on the first intermediate output comprises using the first intermediate output of the second frame as the final output for the second frame. According to several implementations, the environmental sensor system is designed as an active optical sensor systems, in particular a lidar system.

According to several implementations, the environmental sensor system is designed as a camera system or a radar system.

According to the improved concept, also an electronic vehicle guidance system, in particular for a motor vehicle, is provided. The electronic vehicle guidance system comprises an environmental sensor system according to the improved concept and a control unit. The control unit is configured to generate at least one control signal to guide the motor vehicle at least in part automatically, depending on the final output for the second frame.

An electronic vehicle guidance system may be understood as an electronic system, configured to guide a vehicle in a fully automated or a fully autonomous manner and, in particular, without a manual intervention or control by a driver or user of the vehicle being necessary. The vehicle conducts required steering maneuvers, braking maneuvers and/or acceleration maneuvers and so forth automatically. In particular, the electronic vehicle guidance system may implement a fully automatic or fully autonomous driving mode according to level 5 of the SAE J3016 classification. An electronic vehicle guidance system may also be implemented as an advanced driver assistance system, ADAS, assisting a driver for partially automatic or partially autonomous driving. In particular, the electronic vehicle guidance system may implement a partly automatic or partly autonomous driving mode according to levels 1 to 4 of the SAE J3016 classification. Here and in the following, SAE J3016 refers to the respective standard dated June 2018.

Guiding the vehicle at least in part automatically may therefore comprise guiding the vehicle according to a fully automatic or fully autonomous driving mode according to level 5 of the SAE J3016 classification. Guiding the vehicle at least in part automatically may also comprise guiding the vehicle according to a partly automatic or partly autonomous driving mode according to levels 1 to 4 of the SAE J3016 classification.

In some implementations of electronic vehicle guidance system, the control unit may comprise the computing unit of the environmental sensor system or vice versa, completely or in part.

According to the improved concept, also a motor vehicle comprising an electronic vehicle guidance system according to the improved concept or comprising an environmental sensor system according to the improved concept is provided.

According to the improved concept, also a method for operating an environmental sensor system is provided. Therein, respective sets of sensor data depicting an environment of the environmental sensor system, in particular depicting an environmental sensor unit of the environmental sensor system, are generated by the environmental sensor system, in particular by the environmental sensor unit, for one or more consecutive first frames and for a second frame, the one or more first frames preceding the second frame. For each of the one or more first frames, a first evaluation algorithm is applied, in particular by a computing unit of the environmental sensor system, to the respective set of sensor data to generate a respective first intermediate output. For each the one or more first frames, it is examined, in particular by the computing unit, whether a predefined maximum level of computational effort has been exceeded for generating the respective first intermediate output. A second evaluation algorithm is applied, in particular by the computing unit, to the set of sensor data of the second frame to generate a respective second intermediate output for the second frame. A final output for the second frame is generated, in particular by the computing unit, based on the second intermediate output of the second frame, depending on a result of the examination.

Further implementations of the method for operating an environmental sensor system according to the improved concept follow directly from the various implementations of the environmental sensor system according to the improved concept and from the implementations of the electronic vehicle guidance system according to the improved concept and vice versa. In particular, an environmental sensor system according to the improved concept may be programmed to or configured to carry out a method for operating an environmental sensor system according to the improved concept or carries out such a method. According to the improved concept, also a method for guiding a motor vehicle at least in part automatically is provided. Therein, an environmental sensor system of the motor vehicle is operated according to a method for operating an environmental sensor system according to the improved concept. At least one control signal to guide the motor vehicle at least in part automatically is generated depending on the final output for the second frame.

The vehicle is for example guided depending on the control signals, for example by an electronic vehicle guidance system according to the improved concept.

Further implementations for the method for guiding a motor vehicle follow directly from the various implementations of the environmental sensor system according to the improved concept and the electronic vehicle guidance system according to the improved concept and vice versa. In particular, an electronic vehicle guidance system according to the improved concept may be configured or programmed to carry out a method for guiding a motor vehicle according to the improved concept or carries out such a method.

According to the improved concept, a first computer program product comprising first instructions is provided. When the first instructions are executed or the first computer program is executed by an environmental sensor system according to the improved concept, in particular by the computing unit of the environmental sensor system, the first instructions cause the environmental sensor system to carry out a method for operating an environmental sensor system according to the improved concept.

According to the improved concept, also a second computer program comprising second instructions is provided. When the second instructions are executed or the second computer program is executed by an electronic vehicle guidance system according to the improved concept, the second instructions cause the electronic vehicle guidance system to carry out a method for guiding a motor vehicle at least in part automatically according to the improved concept.

According to the improved concept, also a computer readable storage medium is provided, which stores an implementation of the first computer program and/or an implementation of the second computer program according to the improved concept. The first computer program, the second computer program and the computer readable storage medium may be considered as respective computer program products comprising the first instructions or the second instructions, respectively. Further features of the invention are apparent from the claims, the figures and the description of figures. The features and feature combinations mentioned above in the description as well as the features and feature combinations mentioned below in the description of figures and/or shown in the figures alone may not only be encompassed by the improved concept in the respectively specified combination, but also in other combinations. Thus, implementations of the improved concept are encompassed and disclosed, which may not explicitly be shown in the figures or explained, but arise from and can be generated by separated feature combinations from the explained implementations. Implementations and feature combinations, which do not have all features of an originally formulated claim, may be encompassed by the improved concept. Moreover, implementations and feature combinations, which extend beyond or deviate from the feature combinations set out in the relations of the claims, may be encompassed by the improved concept.

In the figures,

Fig. 1 shows schematically a vehicle comprising an exemplary implementation of an electronic vehicle guidance system according to the improved concept;

Fig. 2 shows schematically an exemplary implementation of an environmental sensor system according to the improved concept; and

Fig. 3 shows schematically a part of a further exemplary implementation of an environmental sensor system according to the improved concept. Fig. 1 shows schematically a motor vehicle 10 with an exemplary implementation of an electronic vehicle guidance system 17 according to the improved concept.

The electronic vehicle guidance system 17 comprises an exemplary implementation of an environmental sensor system 1 according to the improved concept and a control unit 4 coupled to the sensor system 1. In some implementations, the control unit 4 may carry out tasks described in the following with respect to the environmental sensor system 1 , in particular a computing unit 3 (see Fig. 2) of the environmental sensor system, and vice versa.

The environmental sensor system 1 has field of view 5 and may detect objects 6 located within the field of view 5.

In particular, during operation, the environmental sensor system 1 may continuously or repeatedly depict the field of view 5 during consecutive frames and generate respective sets of sensor data S1 (see Fig. 2) for each of the frames.

In particular, the environmental sensor system 1 may comprise an environmental sensor unit 2 (see Fig. 2) which is configured to generate respective sets of sensor data S1 depicting an environment of the environmental sensor system 2, in particular the field of view 5, for one or more consecutive first frames and for a second frame.

The computing unit 3 may generate a final output for the second frame based on the sensor data for the second frame and the control unit 4 may generate one or more control signals for guiding the vehicle 10 at least in part automatically based on the final output for the second frame.

In the following, the functionality of the environmental sensor system 1 is explained in more detail with respect to Fig. 2 and 3.

Fig. 2 shows a block diagram of an exemplary implementation of an environmental sensor system 1 according to the improved concept, which may for example be used in the electronic vehicle guidance system 17 of Fig. 1.

As described above, the sensor system 1 comprises an environmental sensor unit 2 and a computing unit 3 coupled to the environmental sensor unit 2.

The computing unit 3 may comprise processing module 7 with a first algorithm module 8 and a second algorithm module 9. Furthermore, the computing unit 3 comprises a scheduling module 11. Each of the first algorithm module 8, the second algorithm module 9 and the schedule module 11 may receive the sensor data S1 . The schedule module 11 may generate a first activation signal S6 and provide it to the first algorithm module 8 and may generate a second activation signal S7 and provide it to the second algorithm module 9.

The first algorithm module 8 may apply a first evaluation algorithm to the sensor data S1 and generate a first intermediate output S2 based on a respective result of the first algorithm.

The second algorithm module 9 may apply a second evaluation algorithm to the sensor data S1 and generate a second intermediate output S3 based on a respective result of the second algorithm.

In particular, the first algorithm is applied to the sensor data S1 , if the first activation signal S6 is logic 1 or a corresponding value. On the other hand, if the second activation signal S7 for the second algorithm module 9 is logic 1 or a corresponding value, the second algorithm is applied to the sensor data S1.

Furthermore, the first algorithm module 8 may generate a first timing signal S4 indicating the corresponding execution time for generating the first intermediate output S2 and the second algorithm module 9 may generate a second timing signal S5 indicating the respective execution time for generating the second intermediate output S3.

Apart from the sensor data S1 , the schedule module 11 may receive the intermediate outputs S2, S3 and the timing signals S4, S5.

On an output side of the schedule module 11 , the schedule module 11 may outputs either the first intermediate output S2 or the second intermediate output S3 or none of them.

Furthermore, the schedule module 11 may optionally output an information signal S9 and/or an active state signal S12.

Figure 3 shows a block diagram of an exemplary implementation of the schedule module 11 , which may for example be used in an environmental sensor system 1 as described with respect to Fig. 1 or Fig. 2. The schedule module 11 comprises for example an evaluation module 12 and a multiplexer module13. The evaluation module 12 may comprise a complexity estimation module 14, a deadline module 15 and an activator module 16.

The complexity estimation module 14 may receive the intermediate outputs S2, S3 and the sensor data S1 . In particular, the complexity estimation module 14 may receive the output of previous frames accumulatively for several frames.

The deadline module 15 may receive the timing signals S4, S5 and compare the timing signals S4, S5 to respective predefined time limits for executing the first algorithm or the second algorithm, respectively.

As an output, the deadline module 15 may generate a deadline signal S10 indicating whether the respective deadlines have been met by the first and/or the second algorithm. The deadline signal S10 may also be received by the complexity estimation module 14.

The complexity estimation module 14 may determine from the intermediate outputs S2, S3, the sensor data S1 and the deadline signal S10, whether the system 1 is in a complex scene scenario that the sensor is not designed to operate in or is likely going to be in such a complex scene scenario. If the system 1 has stayed for at least a predefined period of time in the complex scene scenario while the first algorithm module 8 is operating, the complexity estimation module 14 may generate a first trigger signal S11 and supply it to the activation module 16. The first trigger signal S11 may then indicate that the second algorithm shall be used for generating the final output or the processing module 7 should stop processing.

In case the processing is already stopped, the intermediate output data S2, S3 is not generated and the complexity estimation module 14 may depend only on the sensor data S1. In this case, the complexity estimation module 14 may use the sensor data S1 alone to detect the complexity of the scene and generate the first trigger signal S1 correspondingly. If the scene starts to get simpler for a sufficiently long time, the first trigger signal S11 may indicate that the first algorithm should be used again for generating the final output.

The activation module 16 may receive the first trigger signal S11 , which may for example have one of three values indicating normal processing, restricted processing or no processing. Therein, normal processing may correspond to using the first algorithm for generating the final output, restricted processing may correspond to using the second algorithm and no processing may correspond to use neither of them.

Based on the first trigger signal S11 , the activator module 16 may generate a second trigger signal S8 and provide it to the multiplexer module 13. The content of the second trigger signal S8 may for example be identical to or correspond to the content of the first trigger signal S11.

The activator module 16 may also generate the activation signals S6, S7, depending on the first trigger signal S11.

The multiplexer module 13 may receive the second trigger signal S8 as well as the intermediate outputs S2, S3.

Depending on the second trigger signal S8, the multiplexer module 13 selects either the first intermediate output S2 or the second intermediate output S3 or neither of them to forward it respectively as the final output.

The multiplexer module 13 may also generate the information signal S9, which may indicate an automotive safety integrity level, ASIL, depending on the intermediate output S2, S3, which actually is used as final output. For example, the information signal may indicate ASIL-B or QM.

As described, the improved concept allows to generate output data of an active optical sensor system with an improved reliability by reducing the risk or rate of data drop.

The improved concept may for example be particularly suitable in implementations where the active optical sensor system is implemented as a laser scanner. When the laser scanner views a very complex scene, many segments and objects are generated and scan drop may occur. According to implementation of the improved concept, the received scan is continuously analyzed and when it is detected that the laser scanner is viewing a particularly complex scene for a long time and deadlines for the algorithm to finish the processing are missed, the system may switch from a normal scene analysis algorithm to a version of the algorithm with a lower accuracy, which may, however, finish on time. The system may also keep analyzing the scene complexity and the capability of the lower accuracy algorithm to meet the deadline. In some implementations, if the sensor is running the low accuracy algorithm and the complexity of the scene increases or the low accuracy algorithm cannot finish on time for a predefined period of time, an adapted scheduling algorithm may turn off the scene processing algorithm completely and keep monitoring the scene complexity until it drops. During this time, the system may generate an indicator, for example to a vehicle bus system, that the sensor is not available.

In some implementations, if the sensor is running the low accuracy algorithm and the complexity of the scene drops and the low accuracy algorithm can finish on time for a predefined period of time, it may switch back to the normal accuracy algorithm and, optionally, send a corresponding indicator, which may for example correspond to an ASIL- B level, to the bus.

In some implementations, if the sensor is running in the normal accuracy algorithm and complexity of the scene increases again and the normal accuracy algorithm cannot finish on time for a predefined period of time, it will for example switch back to the low accuracy algorithm and send a respective indicator, which may for example correspond to an ASIL QM level, to the bus.

Claims

1. Environmental sensor system comprising an environmental sensor unit (2), which is configured to generate respective sets of sensor data (S1) depicting an environment of the environmental sensor unit (2) for one or more consecutive first frames and for a second frame, the one or more first frames preceding the second frame, and a computing unit (3), characterized in that the computing unit (3) is configured to apply, for each of the one or more first frames, a first evaluation algorithm to the respective set of sensor data (S1) and to generate a respective first intermediate output (S2); examine, for each the one or more first frames, whether a predefined maximum level of computational effort has been exceeded for generating the respective first intermediate output (S2); apply a second evaluation algorithm to the set of sensor data (S1 ) of the second frame and generate a respective second intermediate output (S3) for the second frame; and generate a final output for the second frame based on the second intermediate output (S3) of the second frame, depending on a result of the examination.

2. Environmental sensor system according to claim 1 , characterized in that the computing unit (3) is configured to generate the final output for the second frame based on the second intermediate output (S3) of the second frame, only if the result of the examination indicates that the respective maximum level of computational effort has been exceeded for at least one of the one or more first frames.

3. Environmental sensor system according to one of the preceding claims, characterized in that the computing unit (3) is configured to apply the first evaluation algorithm to the set of sensor data (S1) of the second frame and to generate a respective first intermediate output (S2) for the second frame; and to generate the final output based on the first intermediate output (S2) of the second frame or based on the second intermediate output (S3) of the second frame, depending on the result of the examination.

Environmental sensor system according to claim 3, characterized in that the computing unit (3) is configured to generate the final output for the second frame based on the first intermediate output (S2) of the second frame, if the result of the examination indicates that the respective maximum level of computational effort has not been exceeded for any of the one or more first frames.

Environmental sensor system according to one of claims 3 or 4, characterized in that the computing unit (3) is configured to further examine, whether the maximum level of computational effort has been exceeded for generating the respective first intermediate output (S2) of the second frame; and to generate the final output for the second frame based on the first intermediate output (S2) of the second frame or based on the second intermediate output (S3) of the second frame, depending on the result of the further examination.

Environmental sensor system according to one of claims 3 to 5, characterized in that the computing unit (3) is configured to generate an information signal (S9), wherein the information signal (S9) indicates, whether the final output for the second frame is generated based on the first intermediate output (S2) of the second frame or based on the second intermediate output (S3) of the second frame.

Environmental sensor system according to one of claims 3 to 6, characterized in that the computing unit (3) is configured to operate in a normal operation mode or in a restricted operation mode, depending on the result of the examination; and the computing unit (3) is configured to generate the final output based on the first intermediate output (S2) of the second frame, when operating in the normal operation mode and based on the second intermediate output (S3) of the second frame, when operating in the restricted operation mode.

8. Environmental sensor system according to one of the preceding claims, characterized in that the computing unit (3) is configured to determine a predefined level of complexity of the depicted environment for each of the one or more first frames and/or for the second frame; and generate the final output for the second frame depending on the determined levels of complexity.

9. Environmental sensor system according to one of the preceding claims, characterized in that the computing unit (3) comprises an evaluation module (12), which is configured to carry out the examination and generate a trigger signal (S8) depending on the result of the examination; and the computing unit (3) comprises a multiplexer module (13), which is configured to select the second intermediate output (S3) of the second frame for generating the final output of the second frame, depending on the trigger signal (S8).

10. Environmental sensor system according to one of the preceding claims, characterized in that, the second evaluation algorithm is designed to require less computational resources than the first evaluation algorithm.

11. Environmental sensor system according to one of the preceding claims, characterized in that the computing unit (3) is configured to determine, for each the one or more first frames, whether a respective predefined timing requirement is met in order to examine whether the respective maximum level of computational effort has been exceeded.

12. Electronic vehicle guidance system comprising an environmental sensor system (1 ) according to one of the preceding claims; and a control unit (4), which is configured to generate at least one control signal to guide a motor vehicle (10) at least in part automatically, depending on the final output for the second frame.

13. Method for operating an environmental sensor system (1), wherein respective sets of sensor data (S1) depicting an environment of the environmental sensor system (1) are generated by the environmental sensor system (1) for one or more consecutive first frames and for a second frame, the one or more first frames preceding the second frame, characterized in that for each of the one or more first frames,

- a first evaluation algorithm is applied to the respective set of sensor data (S1) to generate a respective first intermediate output (S2); and

- it is examined, whether a predefined maximum level of computational effort has been exceeded for generating the respective first intermediate output; a second evaluation algorithm is applied to the set of sensor data (S1) of the second frame to generate a respective second intermediate output (S3) for the second frame; and a final output for the second frame is generated based on the second intermediate output (S3) of the second frame, depending on a result of the examination.

14. Method for guiding a motor vehicle (10) at least in part automatically, wherein an environmental sensor system (1 ) of the motor vehicle (10) is operated according to claim 13; and at least one control signal to guide the motor vehicle (10) at least in part automatically is generated depending on the final output for the second frame.

15. Computer program product comprising instructions, which when executed by an environmental sensor system (1) according to one of claims 1 to 11 , cause the environmental sensor system (1 ) to carry out a method according to claim 13; and/or when executed by an electronic vehicle guidance system (17) according to claim 12, cause the electronic vehicle guidance system (17) to carry out a method according to claim 14.