DE102020215401A1 - optical sensor - Google Patents

Abstract

An optical sensor (100), comprising: - an optically transmissive covering device (10); - a detection device (20) for detecting an object by a sensor; and - a checking device (20) for checking an angular resolution of the optical sensor (100) using a test object detected by the detection device (20), wherein a signal (S) can be output by the checking device (20) if the angular resolution is insufficient is.

Description

The invention relates to an optical sensor. The invention also relates to a method for operating an optical sensor. The invention also relates to a computer program product.

State of the art

Highly automated vehicles (SAE levels 3-5) will increasingly be used on public roads in the coming years. All known concepts of automated vehicles require a combination of different environment detection sensors known per se, such as cameras, radar, lidar, etc. The latter environment detection sensors are in principle laser scanners that emit laser light pulses and times of arrival at an object Measure and evaluate reflected laser light. The LiDAR sensors can determine a distance to the object from the measured time-of-flight.

There are various concepts of how such automated vehicles can be implemented. All of these approaches require a wide variety of sensors (e.g. video cameras, LiDAR, radar, ultrasonic sensors, etc.), with LiDAR sensors (optical sensors that use laser light to generate 3D point clouds of the environment) playing an increasingly important role.

DE 10 2018 118 679 A1 discloses autonomous and semi-autonomous vehicles and an apparatus for enhanced target object detection in a vehicle equipped with laser detection and a LIDAR ranging system.

US 2019/0188498 A1 discloses a detection of ground markings, in particular road markings or markings on vehicle parking spaces.

Disclosure of Invention

The object of the invention is to provide an improved optical sensor.

• - eine optisch durchlässige Abdeckeinrichtung;
• - eine Erfassungseinrichtung zum sensorischen Erfassen eines Objekts; und
• - eine Überprüfungseinrichtung, mittels derer eine Winkelauflösung des optischen Sensors anhand eines von der Erfassungseinrichtung detektierten Prüfobjekts zu prüfen, wobei im Falle einer nicht ausreichenden Winkelauflösung von der Überprüfungseinrichtung ein Steuersignal ausgebbar ist.

According to a first aspect, the invention provides an optical sensor, comprising:

• - an optically transmissive cover means;
• - A detection device for sensory detection of an object; and
• - A checking device, by means of which an angular resolution of the optical sensor can be checked using a test object detected by the detection device, it being possible for the checking device to output a control signal if the angular resolution is insufficient.

In this way an optical sensor is provided which can determine and check its own angular resolution. In this way, for example, disadvantageous effects as a result of soiling, aging, scratching of a covering device of the optical sensor, which can result in a reduced angular resolution, can advantageously be detected. When such a degradation is detected, a suitable action can advantageously be initiated, for example in the form of a degradation of an autonomous driving function of an at least partially automated vehicle equipped with the optical sensor.

• - Sensorisches Erfassen eines Prüfobjekt;
• - Ermitteln einer Winkelauflösung des optischen Sensors mittels des Prüfobjekts; und
• - Ausgeben eines Steuersignals im Falle, dass die ermittelte Winkelauflösung des optischen Sensors nicht definiert ausreichend ist.

According to a second aspect, the object is achieved with a method for operating an optical sensor, having the steps:

• - Sensory detection of a test object;
• - Determining an angular resolution of the optical sensor by means of the test object; and
• - Output of a control signal in the event that the determined angular resolution of the optical sensor is not defined as sufficient.

In this way, performance indicators of the optical sensor are monitored, which advantageously supports improved sensing of an environment using the proposed optical sensor.

Preferred embodiments of the optical sensor are subject of dependent claims.

An advantageous development of the optical sensor is characterized in that the angular resolution can be checked while the optical sensor is in operation. In this way, online monitoring can advantageously be implemented when the optical sensor is used. A certain latency time when determining the angular resolution can be accepted as long as the measurements are repeated at a high rate.

A further advantageous development of the optical sensor is characterized in that an edge between two areas is used as the test object, with it being checked how a transition between the two areas is detected. A kind of "edge resolution" of the optical sensor is advantageously recorded and evaluated in this way.

A further advantageous development of the optical sensor provides that at least one of the following is used as the test object: Driving lane marking, transition from lane to shoulder, crash barrier, traffic sign. In this way, it is advantageously possible to select from a number of potentially suitable different test objects, in which case the test object which is most promising can be used to determine the angular resolution.

A further advantageous development of the optical sensor provides that a point spread function applied to the test object is evaluated. The point-spread function mentioned represents a measure of the angular resolution and, in principle, represents an output variable for the measurement carried out.

A further advantageous development of the optical sensor provides that a width of a plateau and a width in a lower area of an intensity curve obtained with the point spread function are compared to one another and evaluated. Advantageously, an evaluation of the intensity curve that is easy to implement mathematically is used to determine the angular resolution of the optical sensor.

A further advantageous development of the proposed optical sensor is characterized in that the optical sensor has a Gaussian or a rectangular point spread function. Both types of point spread function are present in known optical sensors and represent an intrinsic characteristic of the sensor. It primarily depends on an optical design of the optical sensor and can be used to carry out the proposed method for determining the angular resolution.

A further advantageous embodiment of the proposed optical sensor is characterized in that an analysis of an intensity curve over an angle is carried out using the test object. In order to determine the angular resolution that has just been realized, the aforementioned edge resolution capability of the optical sensor is thus determined and evaluated in a simple manner. The detected angle can be a horizontal angle and/or a vertical angle in relation to the optical sensor.

Further advantageous embodiments of the proposed optical sensor are characterized in that the optical sensor is a LiDAR sensor or a camera. This advantageously provides a number of implementation options for the proposed optical sensor, with the proposed method being particularly easy to implement using a LiDAR sensor.

An advantageous embodiment of the proposed method is characterized in that a repetition rate of the measurement of the angular resolution is set in a defined manner. A measurement intensity can advantageously be defined, with the measurement intensity being able to be carried out with a frequency that corresponds to a frame rate of the LiDAR sensor. In normal operation of the optical sensor, this advantageously results in a particularly low computing outlay for determining the angular resolution.

The invention is described in detail below with further features and advantages on the basis of several figures. Components that are the same or have the same function have the same reference symbols. The figures are intended in particular to clarify the principles which are essential to the invention and are not necessarily drawn to scale.

Device features disclosed result analogously from corresponding disclosed method features and vice versa. This means in particular that features, technical advantages and explanations relating to the optical sensor result in an analogous manner from corresponding explanations, features and advantages of the method for operating an optical sensor and vice versa.

• 1 eine vereinfachte Darstellung einer Helligkeitsverteilung einer Straßenmarkierung;
• 2 zwei verschiedene Varianten einer Point-Spread-Function eines LiDAR-Sensors;
• 3 unterschiedliche Intensitätsdiagramme einer detektierten Straßenmarkierung;
• 4 ein Blockdiagramm eines vorgeschlagenen optischen Sensors;
• 5 ein Ablaufdiagramm eines vorgeschlagenen Verfahrens zum Betreiben eines vorgeschlagenen optischen Sensors; und
• 6 ein Ablaufdiagramm eines vorgeschlagenen Verfahrens zum Betreiben eines vorgeschlagenen optischen Sensors.

In the figures shows:

• 1 a simplified representation of a brightness distribution of a road marking;
• 2 two different variants of a point spread function of a LiDAR sensor;
• 3 different intensity diagrams of a detected road marking;
• 4 a block diagram of a proposed optical sensor;
• 5 a flowchart of a proposed method for operating a proposed optical sensor; and
• 6 a flowchart of a proposed method for operating a proposed optical sensor.

Description of Embodiments

In addition to the range, the angular resolution is also a critical performance indicator (key performance indicator, KPI) of an optical sensor. The angular resolution of the optical sensor can deteriorate under various circumstances, for example due to misalignment of the optics in the sensor due to aging of the adhesive, dirt or raindrops the cover glass of the sensor, scratches on the cover glass, etc. If there is such a reduction in the angular resolution of the optical sensor, objects can no longer be sharply separated and are therefore perceived by the central evaluation unit as a common object.

Under certain circumstances, this can lead to dangerous situations if, for example, a bicycle drives past a parked truck and is assigned to the truck due to a reduced resolution. An autonomous vehicle equipped with the optical sensor can then assume that no dynamic objects are present and can thus overlook the cyclist. To avoid such a situation, the LiDAR sensor should be able to measure its own angular resolution online.

A core idea of the present invention is in particular to provide an optical sensor that is able to determine and evaluate its own angular resolution.

For this purpose, an analysis of test objects (e.g. road markings) in a point cloud of the optical sensor in the form of a LiDAR sensor is proposed. Road markings usually have a legally specified width, so the angular resolution of the LiDAR sensor can be determined based on the measured width of the lane markings in the LiDAR sensor. The road marking is characterized in particular by the fact that it has two areas with different, defined levels of brightness, with the two areas being separated from one another in principle by an edge. However, other types of test objects are also conceivable that have such defined brightness ranges, such as a crash barrier, a transition between a roadway and an adjacent shoulder, traffic sign, etc.

• - Echtzeit-Überwachung der Winkelauflösung ist möglich, solange Straßenmarkierungen oder sonstige Prüfobjekte im Umfeld des optischen Sensors vorhanden sind
• - Keine zusätzliche Sensorik oder Bauteile erforderlich, sondern nur eine Nutzung von Punktwolken
• - Kostengünstig
• - Wenig rechenintensiv, weil eine Breite der Straßenmarkierung direkt vermessen werden kann

The following effects, among others, can be advantageously achieved in this way:

• - Real-time monitoring of the angular resolution is possible as long as there are road markings or other test objects in the vicinity of the optical sensor
• - No additional sensors or components required, just use of point clouds
• - Inexpensive
• - Less computationally intensive because a width of the road marking can be measured directly

Road markings are subject to the “Guidelines for Road Markings” and thus have firmly defined line widths, depending on whether the journey is on the motorway or other roads and whether they are guidelines or lane boundaries. LiDAR sensors can detect road markings because they are more reflective than the road. The points of the road markings appear with a higher intensity in the point cloud and are used, for example, to position the vehicle in the lane.

The proposed LiDAR sensor preferably works with a frame rate of the order of approx. 10 Hz, the frame rate corresponding to a sum of all detector pixels of a detection device 20 .

Road markings are usually clearly recognizable on the road due to an increased intensity of the measured signal.

It is proposed to use this data to monitor the angular resolution of an optical sensor, in particular a LiDAR sensor. The horizontal extent of the road markings is a measure of the angular resolution of the LiDAR sensor. This is shown schematically in the 1-3 shown. The convolution of the rectangular function of the brightness distribution H over the angle α of the 1a , 1b with the Point-Spread-Function flattens the edges of the road marking.

The horizontal reflectivity distribution of a white pavement marking on a dark pavement can be approximated as a rectangular function as shown in Figs 1a , 1b is indicated with gradients of brightness H over a horizontal angle α. In the point cloud data, the road marking appears as a convolution of this rectangle function with the LiDAR sensor's point spread function (PSF), as shown in Figs 3a , 3b implied.

the 2a and 2 B show schematically two different point spread functions of optical sensors: a Gaussian function as a point spread function ( 2a) causes a flattening of the edges similar to an error function ( 3a) , whereas a rectangular point spread function ( 2 B) beveling of the edges of the road marking, as described in 3b is indicated.

Said folding causes a broadening B2 of the detected road marking am Foot of the detected mark, as in FIGS 3a , 3b implied. The broadening B2 can be defined, for example, as the width of the peak at 10% of the maximum intensity I. At the same time, it causes the plateau of the detected marking to narrow by a certain amount, with the width B1 of the plateau being defined, for example, as the width of the peak at 90% the maximum intensity I can be defined. The poorer the angular resolution of the LiDAR sensor, the broader the intensity I is at the foot of the intensity curve or the narrower the plateau of the intensity curve.

As a result, these two characteristics of the intensity curve can be used as a measure for the angular resolution of the optical sensor 100. The width B2 is preferably used at 10% maximum intensity, since the plateau may disappear completely in the case of particularly poor resolutions. From this, the resolution can then be calculated in a first approximation as the difference between the measured width and the expected theoretical width of the road marking. However, other percentages of the maximum intensity I are also conceivable for the widths B2.

the 3a , 3b show a distribution of the brightness of a road marking of the point spread function of the LiDAR sensor and the detected road marking as a function of the horizontal angle a. The detected road marking is a convolution of the brightness of the road marking and the point spread function of the LiDAR sensor. this is in 3a for a Gaussian-like point spread function and in 3b for a rectangular point spread function of the LiDAR sensor.

The proposed method can either be used for a single horizontal LiDAR scan plane or information from multiple or all scan planes containing road markings or other suitable objects to be inspected (e.g. transition from lane to shoulder, crash barrier, traffic sign, etc.) can be combined. Furthermore, the resolution estimation can be improved by averaging the data from several consecutive frames of the LiDAR sensor.

In order to keep the computing effort for determining the angular resolution as low as possible, the following, in 5 suggested a method that can easily be incorporated into a typical acquisition pipeline. Since ground plane estimation of detected objects is carried out in most detection pipelines anyway, this data can be used directly for determining the angular resolution. If the optical sensor 100 detects an unexpectedly poor resolution, the autonomous vehicle equipped therewith can take appropriate measures.

This can include, for example, requesting a cleaning process for the optically transmissive cover device 10 if the reduction in resolution is caused by contamination on the sensor cover glass. Furthermore, the automated vehicle can, for example, reduce its speed to ensure that all objects are resolved in time. It is also conceivable that the vehicle can request a service/workshop visit in order to restore the degraded optical sensor 100 to full performance.

4 shows a basic block diagram of a proposed optical sensor 100 in the form of a LiDAR sensor. A laser 1 for the repetitive emission of optical transmission radiation can be seen through an optically transparent cover device 10 (eg cover glass, for example made of polycarbonate). A detection device 20 for detecting the radiation reflected by an object can also be seen. A defined test object (for example lane markings, edge between lane and shoulder, traffic sign, crash barrier, etc.) is evaluated in the above-mentioned manner by means of the detection device 20 . Depending on the determined angular resolution, the detection device 20 outputs a control signal S, with which a defined action for an at least partially automated vehicle (not shown) equipped with the optical sensor 100 can be initiated, for example a degradation of a driving function, an initiation a braking manoeuvre, bringing the vehicle into a safe state, etc.

The proposed optical sensor 100 can be designed, for example, as an ASIC or FPGA of detector pixels of the LiDAR sensor, which enables hardware-related and therefore efficient evaluation of the extensive measurement data. Alternatively, it is also conceivable to carry out the evaluation of the data on a central processing unit inside or outside the automated vehicle equipped with the lidar sensor. As a result, the proposed method can be implemented as a computer program product which is executed on assigned computer hardware.

5 shows an exemplary sequence of how the proposed method can be carried out as part of a detection activity of an optical sensor 100. In one step 200 If a cloud of points is detected by means of the optical sensor 100, in a step 210 a roadway is determined (ground estimation) by means of the cloud of points detected. In a step 220 an object segmentation is carried out. In a step 230 an object classification takes place and in a step 240 the detected and classified object is tracked. Steps 200 to 240 thus represent a perception or detection activity of optical sensor 100.

In the course of the detection activity of steps 200 to 240, an angular resolution of the optical sensor 100 can advantageously be determined, e.g. cyclically at defined intervals. For this purpose, in a step 211, the intensity of the road is estimated by determining a mean value of the detected intensities.

In a step 212, a road marking arranged on the road is identified by its higher reflectivity. In a step 213, the width of the identified road marking is measured and in a step 214, the angular resolution of the optical sensor 100 is determined.

A repetition rate for carrying out the determination of the angular resolutions can advantageously be set in a defined manner, as a result of which an angular resolution can advantageously be carried out during normal operational operation of the optical sensor 100 . This advantageously supports a rapid reaction to a possible short-term degradation of the angular resolution due to weather influences.

The invention can be used to particular advantage in partially, highly and fully automated vehicles n (level 1-5).

6 shows a basic sequence of an embodiment of the proposed method for operating an optical sensor 100.

In a step 300, a test object is detected by sensors.

In a step 310, an angular resolution of the optical sensor 100 is determined by means of the test object.

In a step 320, a control signal S is output if the determined angular resolution of the optical sensor 100 is not defined as sufficient.

Although the proposed optical sensor 100 is disclosed above primarily in the form of a LiDAR sensor, it is also conceivable to implement the optical sensor 100 in a different technical way, for example as a camera.

In summary, the present invention proposes an optical sensor and a method for operating an optical sensor with which an angular resolution can be detected in a simple manner.

Those skilled in the art will recognize that numerous modifications are possible without departing from the essence of the invention.

QUOTES INCLUDED IN DESCRIPTION

This list of documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• DE 102018118679 A1 [0004]
• US 2019/0188498 A1 [0005]

Claims (13)

Optical sensor (100) comprising: - an optically transmissive cover means (10); - A detection device (20) for detecting an object by sensors; and - a checking device (20), by means of which an angular resolution of the optical sensor (100) can be checked using a test object detected by the detection device (20), it being possible for the checking device (20) to output a control signal (S) if the angular resolution is insufficient . Optical sensor (100) after claim 1 , characterized in that the check of the angular resolution can be carried out in an operative mode of the optical sensor (100). Optical sensor (100) after claim 1 or 2 , characterized in that an edge between two areas is used as the test object, it being checked how a transition between the two areas is recognized. Optical sensor (100) according to one of the preceding claims, characterized in that at least one of the following is used as the test object: lane marking, transition from lane to shoulder, crash barrier, traffic sign. Optical sensor (100) according to one of the preceding claims, characterized in that a point spread function applied to the test object is evaluated. Optical sensor after claim 5 , characterized in that a width (B1) of a plateau and a width (B2) in a lower area of an intensity profile obtained with the point spread function are related to one another and evaluated. Optical sensor (100) after claim 5 or 6 , characterized in that the optical sensor has a Gaussian or a rectangular point spread function. Optical sensor (100) according to one of the preceding claims, characterized in that an analysis of an intensity curve over an angle (a) is carried out by means of the test object. Optical sensor (100) according to one of the preceding claims, characterized in that the optical sensor (100) is a LiDAR sensor or a camera. Method for operating an optical sensor (100), comprising the steps: - Sensory detection of a test object; - Determining an angular resolution of the optical sensor (100) by means of the test object; and - Outputting a control signal (S) in the event that the determined angular resolution of the optical sensor (100) is not defined sufficient. procedure after claim 10 , A point spread function of the optical sensor (100) applied to the test object being evaluated. procedure after claim 10 or 11 , wherein a repetition rate of the measurement of the angular resolution is set in a defined manner. Computer program product with program code means set up to carry out the method according to one of Claims 10 until 12 , if it runs on an optical sensor (100) or is stored on a computer-readable data carrier.

Images