DE102019219389A1 - Method, computer program and device for reducing expected restrictions of a sensor system of a means of locomotion due to environmental influences during the operation of the means of locomotion - Google Patents

Abstract

The present invention relates to a method, a computer program with instructions and a device for reducing expected restrictions of a sensor system of a means of transport due to environmental influences during the operation of the means of transport. The invention further relates to a means of locomotion in which a method according to the invention or a device according to the invention is used. In a first step, information on the surroundings of the means of locomotion is received (10). On the basis of the information on the surroundings of the means of transport, it is determined (11) that restrictions of the sensor system due to environmental influences are to be expected. A warning signal about expected limitations of the sensor system is output to at least one assistance system of the means of locomotion (12). On the basis of the warning signal, the at least one assistance system can then initiate suitable measures (13).

Description

The present invention relates to a method, a computer program with instructions and a device for reducing expected restrictions of a sensor system of a means of transport due to environmental influences during the operation of the means of transport. The invention further relates to a means of locomotion in which a method according to the invention or a device according to the invention is used.

An increasing number of assistance systems are used in modern motor vehicles. For example, describes DE 10 2015 222 061 A1 a lidar sensor for motor vehicles, with a beam source that emits light in the visible or infrared range, a receiver for receiving the light from the beam source reflected on surfaces in the vicinity of the vehicle, and an electronic evaluation device for evaluating the transit times of the light emitted and received again. The light source is set up to emit light at at least two different wavelengths. The evaluation device has a spectral evaluation channel for evaluating intensities of the received light at different wavelengths. The lidar sensor described makes it possible to detect puddles on the roadway.

Many assistance systems use camera images that are captured, for example, by means of a camera installed behind the windshield of the motor vehicle. For example, describes DE 11 2009 000 949 T5 a method of detecting a clear path for a vehicle using a camera. In the method, a floor area in front of the vehicle is imaged with the camera in order to generate a floor image. The ground image is then analyzed to formulate a clear path that is free of objects that restrict the vehicle's travel. A component piece of the soil image is identified iteratively, a feature is extracted from the component piece, and the component piece is classified on the basis of the feature.

A wet or dirty windshield can impair image capture by the camera, which in turn can have a negative effect on the performance of the assistance systems. In the worst case, soiling of the pane can lead to complete blindness of the camera. A clean and completely transparent windshield, at least in the area of the camera, is therefore more and more important with regard to the function of the assistance systems and thus also to safety.

Soiling of the pane or water on the pane are currently detected with measuring systems that are based on the refraction effect and evaluate current measured values of a photo transistor. Based on these measured values, the windshield wipers are activated to clean the windshield. A predictive control of the windshield wipers is not possible with such measuring systems.

With this in mind, describes EP 1 674 356 A1 a method for evaluating signals from at least one rain sensor and at least one camera, the signals from each rain sensor automatically controlling windshield wipers and the signals from each camera being evaluated for driver assistance systems. In the method, all signals are functionally coupled in such a way that the signals from one additionally support and improve the functions of the other. If, for example, it is determined that the image is blurred and out of focus, then, given a positive signal from the rain sensor at the same time, it can be concluded that a surge of water is to be expected on the windshield. In this case, a wiping process can be initiated or changed in advance so that the wiping starts or is accelerated shortly before the expected impact of the surge of water on the windshield. A positive signal from the rain sensor can in turn be used to adapt the image processing and the settings of an image sensor of the camera to the current weather situation.

It is an object of the invention to provide improved solutions for reducing expected restrictions of a sensor system of a means of transport due to environmental influences during the operation of the means of transport.

This object is achieved by a method with the features of claim 1, by a computer program with instructions according to claim 8 and by a device with the features of claim 9. Preferred embodiments of the invention are the subject of the dependent claims.

• - Empfangen von Informationen zu einer Umgebung des Fortbewegungsmittels;
• - Ermitteln, dass Einschränkungen der Sensorik durch Umgebungseinflüsse zu erwarten sind, auf Basis der Informationen zur Umgebung des Fortbewegungsmittels; und
• - Ausgeben eines Warnsignals über zu erwartende Einschränkungen der Sensorik an zumindest ein Assistenzsystem des Fortbewegungsmittels.

According to a first aspect of the invention, a method for reducing expected restrictions of a sensor system of a means of transport due to environmental influences when operating the means of transport comprises the steps:

• - Receiving information about the surroundings of the means of locomotion;
• - Determine that restrictions of the sensor system due to environmental influences are to be expected, based on the information on the surroundings of the means of transport; and
• - Outputting a warning signal about expected limitations of the sensor system to at least one assistance system of the means of locomotion.

• - Empfangen von Informationen zu einer Umgebung des Fortbewegungsmittels;
• - Ermitteln, dass Einschränkungen der Sensorik durch Umgebungseinflüsse zu erwarten sind, auf Basis der Informationen zur Umgebung des Fortbewegungsmittels; und
• - Ausgeben eines Warnsignals über zu erwartende Einschränkungen der Sensorik an zumindest ein Assistenzsystem des Fortbewegungsmittels.

According to a further aspect of the invention, a computer program contains instructions which, when executed by a computer, cause the computer to carry out the following steps to reduce expected limitations of a sensor system of a means of transport due to environmental influences when operating the means of transport:

• - Receiving information about the surroundings of the means of locomotion;
• - Determine that restrictions of the sensor system due to environmental influences are to be expected, based on the information on the surroundings of the means of transport; and
• Output of a warning signal about expected limitations of the sensor system to at least one assistance system of the means of locomotion.

The term computer is to be understood broadly. In particular, it also includes control devices and other processor-based data processing devices.

The computer program can, for example, be provided for electronic retrieval or can be stored on a computer-readable storage medium.

• - eine Empfangseinheit zum Empfangen von durch die er Sensorik erfassten Informationen zu einer Umgebung des Fortbewegungsmittels;
• - eine Analyseeinheit zum Ermitteln, dass Einschränkungen der Sensorik durch Umgebungseinflüsse zu erwarten sind, auf Basis der Informationen zur Umgebung des Fortbewegungsmittels; und
• - eine Ausgabeeinheit zum Ausgeben eines Warnsignals über zu erwartende Einschränkungen der Sensorik an zumindest ein Assistenzsystem des Fortbewegungsmittels.

According to a further aspect of the invention, a device for reducing expected limitations of a sensor system of a means of locomotion due to environmental influences during the operation of the means of locomotion:

• a receiving unit for receiving information about the surroundings of the means of locomotion detected by the sensor system;
• an analysis unit for determining that restrictions of the sensor system due to environmental influences are to be expected, based on the information on the surroundings of the means of locomotion; and
• an output unit for outputting a warning signal about expected limitations of the sensor system to at least one assistance system of the means of locomotion.

With the solution according to the invention, it is continuously determined during operation whether limitations in the sensor system, e.g. due to water or dirt, are to be expected.

For this purpose, the available information can be examined, e.g. with regard to environmental conditions, such as puddles, road pollution, etc., and means of transport moving ahead. If a risk is recognized, e.g. a high probability that the windshield will be soiled in the immediate future and thus the camera behind the window will be impaired, the assistance systems receive information about the impending risk. You can then initiate suitable measures. In this way, any impairment of the function of the assistance systems is reduced.

According to one aspect of the invention, the information about the surroundings of the means of locomotion is provided by a rain sensor, a light sensor, a camera, a radar sensor, a lidar sensor or an ultrasonic sensor. In addition to the available rain information from the rain sensor and information about the lighting conditions from the light sensor, images from the camera or lidar data in particular can be examined in relation to the environmental conditions, e.g. for the presence of puddles or road contamination. In addition, the presence of forward or oncoming means of locomotion can be determined from the camera data, the lidar data or the radar data. All of these factors influence the likelihood that sensor limitations will occur.

According to one aspect of the invention, the determination that limitations of the sensor system due to environmental influences are to be expected takes place by means of an AI system. AI algorithms are particularly suitable for determining the probability of whether and to what extent the windshield is soiled or the sensors are impaired, taking into account the given information about the surroundings. For example, such an AI system can first be learned in the course of test drives and then evaluate the current measured values for a defined time window during driving.

According to one aspect of the invention, the at least one assistance system causes an increase in the amount of recorded route data or object data in response to the warning signal or carries out a precalculation of trajectories of the means of transport or other traffic objects for an extended period of time. When a potential impending danger due to blindness of the sensor system is recognized, as much route and object data as possible is recorded, ie more than as a rule, in order to be able to safely drive through the phase of impending blindness. For this purpose, object data in particular can be collected for an extended period of time. In addition, the precalculated Trajectories of the means of transport and the other traffic objects are calculated for an extended period of time. In normal situations, a trajectory of approx. 3-4 seconds is predicted. If an imminent danger is recognized, this time can be increased in order to cover a larger predicted period. For this purpose, trajectories that are otherwise irrelevant can also be stored in order to bridge the danger time with valid information. If sensor blindness or sensor limitation actually occurs, the assistance systems can orient themselves on the basis of the previously calculated trajectories of the means of transport and the traffic objects and thus drive through the phase of blindness with a minimal or at least significantly reduced risk potential.

According to one aspect of the invention, the at least one assistance system carries out a braking maneuver or an evasive maneuver in response to the warning signal. Depending on the severity of the sensor system's limitations and the traffic situation, it may be sufficient to simply reduce the speed and increase the distance to other road users. If the traffic situation permits, interventions in the longitudinal and transverse dynamics can be carried out, e.g. to avoid a puddle or splash water, etc. and thus avoid the risk of temporary blindness of the sensors. Depending on the traffic situation, braking or a simple evasive maneuver may be sufficient. These maneuvers are carried out on the basis of previously calculated object trajectories to prevent accidents or minimize risks as much as possible. In an emergency, the assistance systems can bring about a safe state, e.g. by emergency braking.

According to one aspect of the invention, a windshield wiper of the means of transport is activated in response to the warning signal, a warning is output to an operator of the means of transport or information about a hazard is transmitted to a backend. Thanks to the predictive control of the windshield wipers, the phase in which the sensors are restricted can be kept as short as possible. The warning to the operator also ensures that he does not rely exclusively on the assistance systems, but that he pays increased attention to the control of the means of transport. The transmission of information on dangerous spots to a backend makes it possible to make this available to other road users and thus to increase road safety overall.

A method according to the invention or a device according to the invention is preferably used in a (partially) autonomous or manually controlled means of locomotion. The means of transport can in particular be a motor vehicle, but also a ship, a manned or unmanned aircraft, e.g. a drone or a Volocopter, etc.

• 1 zeigt schematisch ein Verfahren zum Reduzieren von zu erwartenden Einschränkungen einer Sensorik eines Fortbewegungsmittels durch Umgebungseinflüsse beim Betrieb des Fortbewegungsmittels;
• 2 zeigt eine erste Ausführungsform einer Vorrichtung zum Reduzieren von zu erwartenden Einschränkungen einer Sensorik eines Fortbewegungsmittels durch Umgebungseinflüsse beim Betrieb des Fortbewegungsmittels;
• 3 zeigt eine zweite Ausführungsform einer Vorrichtung zum Reduzieren von zu erwartenden Einschränkungen einer Sensorik eines Fortbewegungsmittels durch Umgebungseinflüsse beim Betrieb des Fortbewegungsmittels; und
• 4 stellt schematisch ein Fortbewegungsmittel dar, in dem eine erfindungsgemäße Lösung realisiert ist.

Further features of the present invention will become apparent from the following description and the appended claims in conjunction with the figures.

• 1 shows schematically a method for reducing expected restrictions of a sensor system of a means of locomotion due to environmental influences during the operation of the means of locomotion;
• 2 shows a first embodiment of a device for reducing expected restrictions of a sensor system of a means of locomotion due to environmental influences during the operation of the means of locomotion;
• 3 shows a second embodiment of a device for reducing expected restrictions of a sensor system of a means of locomotion due to environmental influences during the operation of the means of locomotion; and
• 4th shows schematically a means of locomotion in which a solution according to the invention is implemented.

For a better understanding of the principles of the present invention, embodiments of the invention are explained in more detail below with reference to the figures. It goes without saying that the invention is not restricted to these embodiments and that the features described can also be combined or modified without departing from the scope of protection of the invention as defined in the appended claims.

1 shows schematically a method for reducing expected restrictions of a sensor system of a means of locomotion due to environmental influences during the operation of the means of locomotion. In a first step, information about the surroundings of the means of locomotion is received 10. This information can be provided, for example, by a rain sensor, a light sensor, a camera, a radar sensor, a lidar sensor or an ultrasonic sensor. On the basis of the information on the surroundings of the means of transport, it is determined 11 that limitations of the sensor system due to environmental influences are to be expected. An AI system, for example, can be used for this purpose. A warning signal about expected restrictions of the sensor system is output to at least one assistance system of the means of locomotion 12. On the basis of the warning signal, the at least one assistance system can then be suitable Initiate measures 13. For example, the at least one assistance system can cause an increase in the amount of recorded route data or object data or carry out a precalculation of trajectories of the means of transport or other traffic objects for an extended period of time. Another possibility is that the at least one assistance system carries out a braking maneuver or an evasive maneuver. Furthermore, it is possible, in response to the warning signal, to activate a windshield wiper of the means of transport, to output a warning to an operator of the means of transport or to transmit information about a hazard to a backend.

2 shows a simplified schematic representation of a first embodiment of a device 20th to reduce expected limitations of a sensor system of a means of transport due to environmental influences during the operation of the means of transport. The device 20th has an entrance 21 , via which a receiving unit 22nd information U receives to an environment of the means of locomotion. This information U can for example by a rain sensor 420 , a light sensor 421 , a camera 422 , a radar sensor 423 , a lidar sensor 424 or an ultrasonic sensor 425 to be provided. A unit of analysis 23 determined on the basis of the information U the surroundings of the means of transport, whether or that restrictions of the sensors due to environmental influences are to be expected. The analysis unit 23 can be implemented as an AI system, for example. If restrictions are to be expected, an output unit 24 a warning sign W. about expected limitations of the sensors via an output 26th the device 20th at least one assistance system 41 of the means of transport issued. Based on the warning signal W. at least one assistance system can do that 41 then initiate suitable measures, for example an increase in the amount of recorded route data or object data or carry out a pre-calculation of trajectories of the means of transport or other traffic objects for an extended period of time. Another possibility is that the at least one assistance system carries out a braking maneuver or an evasive maneuver. Furthermore, it is possible, in response to the warning signal, to activate a windshield wiper of the means of transport, to output a warning to an operator of the means of transport or to transmit information about a hazard to a backend.

The receiving unit 22nd , the analysis unit 23 and the output unit 24 can from a control unit 25th being controlled. Via a user interface 28 If necessary, settings of the receiving unit can be made 22nd , the analysis unit 23 , the output unit 24 or the control unit 25th be changed. The one in the device 20th Accruing data can be stored in a memory if required 27 the device 20th be stored, for example for a later evaluation or for use by the components of the device 20th . The receiving unit 22nd , the analysis unit 23 , the output unit 24 as well as the control unit 25th can be implemented as dedicated hardware, for example as integrated circuits. Of course, they can also be partially or completely combined or implemented as software that runs on a suitable processor, for example on a GPU. The entrance 21 and the exit 26th can be implemented as separate interfaces or as a combined bidirectional interface.

3 shows a simplified schematic representation of a second embodiment of a device 30th to reduce expected limitations of a sensor system of a means of transport due to environmental influences during the operation of the means of transport. The device 30th assigns a processor 32 and a memory 31 on. The device is, for example 30th a computer, workstation or control device. In the storage room 31 instructions are stored that the device 30th when executed by the processor 32 cause the steps to be carried out in accordance with one of the methods described. The ones in memory 31 Stored instructions are thus embodied by the processor 32 executable program which implements the method according to the invention. The device has an entrance 33 to receive information. From the processor 32 generated data is via an output 34 provided. In addition, they can be in memory 31 be filed. The entrance 33 and the exit 34 can be combined to form a bidirectional interface.

The processor 32 may comprise one or more processor units, for example microprocessors, digital signal processors or combinations thereof.

The memory 27 , 31 of the described embodiments can have volatile and / or non-volatile storage areas and comprise a wide variety of storage devices and storage media, for example hard disks, optical storage media or semiconductor memories.

4th shows schematically a means of locomotion 40 represents, in which a solution according to the invention is implemented. In this example it is the means of transportation 40 to a motor vehicle. The motor vehicle has a number of assistance systems 41 on, one of which is exemplary is shown. In addition, there is a sensor system 42 available from the assistance systems 41 is used and through the information U to the environment of the motor vehicle can be detected. A device according to the invention 20th determined on the basis of the sensors 42 information collected U whether. or that restrictions of the sensors 42 are to be expected due to environmental influences and gives a corresponding warning signal W. to the assistance systems 41 out. They can then initiate suitable measures. Further components of the motor vehicle are, for example, a navigation system 43 and a data transmission unit 44 . By means of the data transmission unit 44 can connect to a backend 50 be set up, in particular for the transmission of information on recognized danger spots, which can then be made available to other road users. A memory is used to store data 45 available. The data exchange between the various components of the motor vehicle takes place via a network 46 .

List of reference symbols

10

Receiving information about the surroundings of the means of locomotion

11

Determine that limitations of the sensors due to environmental influences are to be expected

12th

Output of a warning signal to an assistance system

13th

Initiation of suitable measures

20th

contraption

21

Entrance

22nd

Receiving unit

23

Analysis unit

24

Output unit

25th

Control unit

26th

Exit

27

Storage

28

User interface

30th

contraption

31

Storage

32

processor

33

Entrance

34

Exit

40

Means of transportation

41

Assistance system

42

Sensors

43

navigation system

44

Data transmission unit

45

Storage

46

network

50

Backend

420

rain sensor

421

Light sensor

422

camera

423

Radar sensor

424

Lidar sensor

425

Ultrasonic sensor

U

Information about an environment

W.

Warning sign

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102015222061 A1 [0002]
• DE 112009000949 T5 [0003]
• EP 1674356 A1 [0006]

Claims (10)

Method for reducing expected limitations of a sensor system (42) of a means of transport (40) due to environmental influences when operating the means of transport (40), with the following steps: - Receiving (10) information (U) on the surroundings of the means of locomotion (40); - Determination (11) that restrictions of the sensor system (42) are to be expected due to environmental influences, on the basis of the information (U) about the surroundings of the means of locomotion (40); and - Outputting (12) a warning signal (W) about expected restrictions of the sensor system (42) to at least one assistance system (41) of the means of locomotion (40). Procedure according to Claim 1 , the restrictions of the sensor system (42) being restrictions due to water or dirt. Procedure according to Claim 1 or 2 , wherein the information (U) about the surroundings of the means of locomotion (40) by a rain sensor (420), a light sensor (421), a camera (422), a radar sensor (423), a lidar sensor (424) or an ultrasonic sensor (425) to be provided. Method according to one of the preceding claims, wherein the determination (11) that restrictions of the sensor system (42) are to be expected due to environmental influences is carried out by means of an AI system. Method according to one of the Claims 1 to 4th , wherein the at least one assistance system (41) in response to the warning signal (W) causes an increase in the amount of recorded route data or object data or carries out a pre-calculation of trajectories of the means of transport (40) or other traffic objects for an extended period of time. Method according to one of the Claims 1 to 4th , wherein the at least one assistance system (41) performs a braking maneuver or an evasive maneuver in response to the warning signal (W). Method according to one of the preceding claims, wherein in response to the warning signal (W) a windshield wiper (43) of the means of locomotion (40) is activated, a warning is output to an operator of the means of locomotion (40) or information about a hazard is sent to a backend is transmitted. Computer program with instructions which, when executed by a computer, enable the computer to carry out the steps of a method according to one of the Claims 1 to 7th to reduce expected restrictions of a sensor system (42) of a means of locomotion (40) due to environmental influences during the operation of the means of locomotion (40). Device (20) for reducing expected restrictions of a sensor system (42) of a means of locomotion (40) due to environmental influences during operation of the means of locomotion (40), with: - A receiving unit (22) for receiving (10) information (U) detected by the sensor system (42) about the surroundings of the means of locomotion (40); - An analysis unit (23) for determining (11) that restrictions of the sensor system (42) due to environmental influences are to be expected, on the basis of the information (U) about the surroundings of the means of locomotion (40); and - An output unit (24) for outputting (12) a warning signal (W) about expected limitations of the sensor system (42) to at least one assistance system (41) of the means of locomotion (40). Means of locomotion (40), characterized in that there is a device (20) according to Claim 9 has or is set up, a method according to one of the Claims 1 to 7th to reduce expected restrictions of a sensor system (42) of the means of locomotion (40) due to environmental influences during the operation of the means of locomotion (40).

Images