DE202014105727U1 - sensor system - Google Patents

Abstract

Optical sensor system with a distance sensor (14), in particular a laser scanner, which determines distance information for a first monitoring area (34), wherein the first monitoring area (34) defines a flat surface, which preferably runs parallel to a background (12), and an evaluation unit , which is designed to detect objects (30, 32) on the basis of the distance information, and to output a warning signal when an object (30, 32) is detected, characterized in that the sensor system additionally comprises a depth camera (20) which forms a three-dimensional image of a second monitoring area (36) generated.

Description

The present invention relates to an optical sensor system having a distance sensor, in particular a laser scanner, which determines distance information for a first monitoring area, wherein the first monitoring area defines a flat surface, which preferably runs parallel to a background. The optical sensor system further comprises an evaluation unit, which is designed to recognize objects based on the distance information, and to output a warning signal when an object is detected. Furthermore, the present invention also relates to a vehicle with the optical sensor system.

Such optical sensor systems find use in a variety of applications. For example, they may be stationary and serve to monitor a shelter of a dangerous machine. In this case, by means of the first monitoring area, which runs in particular parallel to the ground, e.g. the legs are detected by persons. If the presence of a person or an object in the shelter is detected by the sensor system in this way, the machine can be switched off so as not to endanger the person.

In addition, optical sensor systems can also be used for mobile applications, in particular they can enable operation of vehicles and self-driving vehicles. Self-propelled vehicles, also called AGVs (Automated Guided Vehicles), are used e.g. used for the automatic transport of objects, for example in automated manufacturing plants. To avoid collision with objects (i.e., obstacles or people), a vehicle-mounted laser scanner of the sensor system, which is mounted about 15 to 30 cm above the ground, and is used, for example, is commonly used. scans a monitoring area oriented in the direction of travel of the vehicle. In general, the laser scanner can also be a distance sensor whose monitoring area preferably extends parallel to the substrate and defines a flat surface. By mounting the distance sensor spaced from the ground, a false detection of the ground as an obstacle during pitching movements of the vehicle when accelerating or braking or when driving over bumps can be avoided.

Due to the arrangement of the distance sensor, obstacles that are lower than the distance of the first monitoring area or of the distance sensor from the ground are not detected both in stationary and in mobile use of the sensor system. In a collision e.g. a self-driving vehicle with such obstacles may cause damage to the vehicle and the obstacle. In addition, especially in the use of self-driving vehicles in hospitals also a risk of injury exist. Even with the stationary use of the sensor system, failure to detect objects can lead to security risks.

It is therefore an object of the invention to specify an optical sensor system which can also detect flat objects and obstacles, wherein the detection should be as simple and cost-effective as possible.

According to the invention, this object is achieved by an optical sensor system according to claim 1 and in particular by the fact that the sensor system additionally comprises a depth camera which generates a three-dimensional image of a second monitoring area.

The depth camera is preferably a TOF camera (i.e., a Time Of Flight camera) or a stereo camera. Such an additional depth camera can be significantly less expensive than, for example, the use of a laser scanner with a three-dimensional monitoring area. In addition, the three-dimensional images captured by the depth camera can be processed more quickly and easily, thereby enabling the detection of flat obstacles in an economical and thus cost-effective manner.

Furthermore, it is advantageous that no high demands are placed on the depth camera, for example, the depth camera may not be able to detect a remission of less than 2%, since such specifications for personal protection can already be met by the distance sensor.

During operation of the sensor system, the distance sensor scans a line-shaped area and delivers distance information, ie depth information, for the scanned line.

Preferably, the first monitoring range of the distance sensor and the second monitoring range of the depth camera "overlap", which results in the possibility to combine the information collected by distance sensor and depth camera information with each other. In this way, there is also a certain redundancy of the acquired information, which can be exploited for better recognition of objects.

The distance sensor and the depth camera respectively transmit the distance information or the acquired three-dimensional images to the evaluation unit. This can then perform an obstacle detection. The evaluation unit communicates with a control unit, for example, and transmits a warning signal to the control unit if an object or obstacle has been detected. The control unit can then initiate appropriate measures in response to the warning signal, eg the stop of the machine or of the vehicle or a warning of a user by optical or acoustic signals. In principle, the warning signal can also initiate such actions directly.

Compared to a sole use of the distance sensor, additional information about the surroundings of the optical sensor system is detected by the combination of distance sensor and depth camera, whereby a better object recognition is made possible. In particular, measured values above and below the flat area of the first monitoring area can also be detected by the depth camera.

The distance sensor and / or the depth camera can be intrinsically safe and have secure communication interfaces. The distance sensor is preferably designed as a laser scanner with a rotatable scan head, which performs distance information via a light transit time measurement of laser pulses emitted by the distance sensor itself.

To increase security, the evaluation unit can perform cyclic tests on the depth camera.

Advantageous developments of the invention are specified in the description, the drawings and the dependent claims.

According to a first advantageous embodiment, the evaluation unit for evaluating the distance information and the three-dimensional images is a computing device of the distance sensor. This means that the data processing of distance sensor and depth camera takes place wholly or partly in the distance sensor. It is exploited that the computing device of the distance sensor is usually very powerful and is designed as a safe or intrinsically safe component. In addition, it is advantageous that the depth camera then requires only a rudimentary control, whereby a complex and therefore expensive additional data processing in the depth camera can be omitted. The three-dimensional images generated by the depth camera can thus be processed by the computing device of the distance sensor. The computing device of the distance sensor can then also communicate with the control unit, for example, to transmit a warning signal.

Due to the joint processing of the data of the distance sensor and the depth camera, only one data interface from the evaluation unit to the control unit is necessary. In addition, a single configuration of the distance sensor software may be sufficient for the operation of the distance sensor and the depth camera.

Alternatively, an external evaluation unit may also be provided which, in particular, carries out the data processing for the distance sensor and the depth camera and communicates with the control unit.

According to a further advantageous embodiment, the depth camera is aligned with the ground, wherein the depth camera preferably detects the ground below the first monitoring area at least partially. The depth camera and the distance sensor can be arranged at a distance from one another, wherein the depth camera and the distance sensor are aligned "obliquely" to each other. As a result, the first monitoring region of the distance sensor and the second monitoring region of the depth sensor can intersect at least in regions. Due to the such orientation of the depth camera (e.g., obliquely downward), the depth camera, when used in a vehicle, can detect the ground ahead of the self-propelled vehicle in the direction of travel, and thus also detect low obstacles that present a danger of collision to the vehicle.

Preferably, the evaluation unit is designed to receive and process three-dimensional images or image data of the depth camera, wherein the evaluation unit is designed to compare the three-dimensional images with a reference contour and to detect obstacles based on the comparison. The reference contour can be stored for this purpose in the evaluation unit, wherein the reference contour can be calculated, for example, from the relative mounting positions of the distance sensor and the depth camera. Alternatively, the reference contour can be taught in with the aid of test specimens. A change in the three-dimensional images that does not match the reference contour can then indicate an object or obstacle.

According to a further advantageous embodiment, the evaluation unit is designed to use a background as reference contour. The substrate is particularly suitable as a reference contour, since it is usually a flat, flat surface.

Preferably, the substrate covered by the first monitoring region, ie the substrate underlying the first monitoring region, is used as reference contour. The substrate usually has a good remission behavior, as a result of which reliable measured values can be recorded by the depth camera.

If, during operation, the depth camera determines a smaller distance to the background in the three-dimensional image than would be the case according to the reference contour, the evaluation unit recognizes a survey which is classified as an object or obstacle in particular from a predetermined height. In the opposite case, a measured value enlarged in comparison to the reference contour can be, for example, a step or a pothole.

Preferably, the second monitoring area is illuminated by means of a, preferably modulated, light source. The modulated light source is coupled to the depth camera to allow a light transit time measurement. The light source can be arranged at a distance from the depth camera. As a light source, a light emitting diode (LED) or a laser can be used. The modulation frequency of the light source is preferably adapted to the distance to be measured, the desired resolution and / or the required frame rate. For example, a modulation frequency of 50 MHz is suitable for a maximum range of 3 m.

Also preferably, the light source is designed to illuminate areas of the second monitoring area farther from the depth camera with higher illumination intensity. As the distance to the depth camera increases, the depth camera perceives the ground at a shallower angle of incidence, decreasing the relative brightness of the depth camera in that area. Increasing the illumination intensity in this area compensates for the greater distance and thus the flatter angle of incidence, thereby avoiding dynamic problems and improving measurement accuracy.

For example, a parabolic, i. square, increase the illumination intensity over the distance to the light source or the distance to the vehicle by means of beam shaping and / or changed control of the light source done.

According to a further advantageous embodiment, the depth camera is designed to transmit only those data of the three-dimensional image to the evaluation unit, which originate from the area of the first monitoring area. For example, the viewing angle of the depth camera can be so great that the depth camera detects a spatial area extending laterally beyond the first monitoring area. In this case, it may be sufficient to transmit only data to the evaluation unit which originate from the limited space area below or above the first monitoring area or from the first monitoring area itself (ie from a space area which does not protrude laterally beyond the first monitoring area). This limited space area may be referred to as an overlap area.

This results in data reduction in the depth camera, i. the three-dimensional image can be cropped. As a result, this leads to a reduction of the evaluation unit, which has to cope with lower data rates. The evaluation unit can thus be constructed from lower-cost components. In addition, the evaluation can be done faster.

Alternatively or additionally, a data reduction can take place in such a way that the depth camera of the evaluation unit notifies only the differences of the respectively current three-dimensional image from the reference contour. This means that only data is transmitted to the evaluation unit, for example, come from an object or obstacle and thus deviate from the reference contour by a predetermined threshold.

According to a further advantageous embodiment, the depth camera is designed to record a three-dimensional image more frequently than the distance sensor scans the first monitoring area. Increasing the take-up rate will provide faster response to obstacles. In addition, obstacles detected by the distance sensor can be made more plausible more quickly by the additional data of the depth camera.

The invention also relates to a vehicle having an optical sensor system of the type described above and a control unit. The evaluation unit is designed to recognize objects based on the distance information, and to output a warning signal to the control unit when an object is detected. In particular, the vehicle may also be a self-propelled vehicle.

For the vehicle, the combination of distance sensor and depth camera is also advantageous in that e.g. above the first monitoring area in the travel of the vehicle projecting obstacles can be detected. For example, the depth camera for detecting such obstacles may also be aligned parallel to the distance sensor.

The control unit may serve to control the movement of the vehicle and the vehicle to steer. Based on the warning signal, the control unit can decide whether the vehicle is stopped, for example. In addition to the warning signal, the position of the object and its size can also be transmitted to the control unit, whereupon the control unit can change the movement of the vehicle and control the vehicle in such a way that the obstacle is bypassed.

According to an advantageous embodiment of the vehicle, the evaluation unit is designed to adapt the reference contour to acceleration-dependent movements of the vehicle. The reference contour can therefore be adjusted automatically, for example, to a pitching motion during acceleration or braking of the vehicle. For this purpose, the control unit of the evaluation unit can transmit current acceleration values, on the basis of which the evaluation unit adapts the reference contour in each case. Alternatively, the evaluation unit can also refer to measured values of an acceleration sensor. Furthermore, it is also possible to take account of an inclined position when driving through curves or when driving on slopes which are e.g. is determined by a position sensor. Due to the adaptation of the reference contour, an undesired stop of the vehicle can be avoided by falsely assuming obstacles during a pitching motion.

Preferably, the distance sensor can detect distance information in an angular range of 180 °, this angle range being arranged in the direction of travel of the vehicle. The monitoring area formed in this way thus defines a flat area, which may be limited, for example, to a size of 3 × 3 m.

Alternatively, the three-dimensional images or data acquired by the depth camera can also be transmitted to the control unit without processing in order to enable parking of the vehicle or detection of pallets and the like. In this way, the sensor system expands the possible uses of the self-driving vehicle.

Incidentally, the statements relating to the sensor system according to the invention, with regard to advantages and particular embodiments, apply to the vehicle according to the invention.

Hereinafter, the invention will be described purely by way of example with reference to the drawings. Show it:

1 a vehicle according to the invention in a schematic side view;

2 a schematic side view of an inventive optical sensor system, which is attached to a stationary machine;

3 a schematic plan view of a first and a second monitoring area; and

4 a brightness distribution in the second monitoring area.

1 shows a self-driving vehicle 10 , which is on a surface 12 emotional. In the direction of travel forward is on the vehicle 10 a laser scanner 14 with a rotating scan head 16 appropriate.

From the scan head 16 becomes a laser beam 18 emitted, which is parallel to the ground 12 spreads. Meets the laser beam 18 to an obstacle, such as the object described below 30 That's how the laser beam gets 18 to the laser scanner 14 reflected, with the laser scanner 14 calculated from the light transit time a distance to the obstacle.

Spaced to the laser scanner 14 is above the laser scanner, a depth camera, namely a TOF camera 20 arranged. The field of vision of the TOF camera 20 is on the ground 12 in front of the vehicle 10 directed. The field of vision 22 The camera thus cuts the plane in which the laser beam 18 runs.

The TOF camera 20 has a resolution of 80 × 60 pixels in this embodiment and generates a frame rate in the range of 50 Hz. The laser scanner 14 has a scan frequency of 25 Hz. The frame rate of the TOF camera 20 is thus twice as large as the scanning frequency of the laser scanner 14 ,

The from the TOF camera 20 captured three-dimensional images are using a secure data connection 24 to the laser scanner 14 transfer. The laser scanner 14 processes both the three-dimensional images of the TOF camera by means of an integrated computing device (not shown) which serves as an evaluation unit 20 as well as those of the laser scanner 14 determined distance information. Detects the computing device of the laser scanner 14 an obstacle, then a warning signal by means of a second secure data connection 26 to a control unit 28 of the vehicle 10 transmitted. The control unit 28 then slows down the vehicle 10 off or bypass the obstacle.

In 1 is an object 30 shown on the ground 12 in front of the vehicle 10 lies and in the field of vision 22 the TOF camera 20 located. The object 30 is too shallow to get away from the laser beam 18 of the laser scanner 14 to be recorded.

From the vehicle 10 seen from behind the object 30 is located in the example shown a step 32 in which the underground 12 drops abruptly. Also the stage 32 is not from the laser beam 18 of the laser scanner 14 captured, but is in the field of vision 22 the TOF camera 20 ,

The TOF camera 20 is by programming a reference contour of the ground 12 known, which corresponds to a level surface, ie the ground should not be an object 30 and no level 32 exhibit. When creating the three-dimensional images, the TOF camera recognizes 20 in the area of the object 30 shortened distance values and in the range of the step 32 extended distance values compared to the stored reference contour. The differences to the reference contour are from the TOF camera 20 to the computing device of the laser scanner 14 transmitted, whereupon the computing device recognizes that in front of the vehicle 10 an object on the ground 12 is arranged behind which a step 32 lies. The object 30 can potentially be a collision and the stage 32 a crash of the vehicle 10 cause. For this reason, a warning signal is sent to the control unit 28 transmitted. The control unit 28 then, for example, causes a dodge of the vehicle 10 ,

In 2 is a stationary machine, namely a press 100 , shown schematically. In front of the press 100 extends a shelter 33 that of the laser scanner 14 and the TOF camera 20 is detected. The operation of the laser scanner 14 and the TOF camera 20 do not differ from the under reference to 1 described method of operation. The remarks to 1 So meet up too 2 - with the differences explained below - too.

Unlike the vehicle 10 becomes the operation of the press 100 stopped as soon as an object 30 that is in the shelter 33 is detected. This will prevent the object 30 or persons from the press 100 be endangered.

In addition, the stage 32 - unlike the vehicle 10 - at the press 100 not recognized as an object or obstacle. Instead, there is the level 32 always at the same distance from the stationary press 100 and is thus part of the reference contour.

3 shows the area in front of the vehicle 10 or the shelter 33 in front of the press 100 in a view from above. It is a rectangular first surveillance area 34 represented by that of the laser beam 18 stretched over the swept area. The laser beam 18 goes over the first surveillance area 34 however, only distance information from the first monitoring area becomes available 34 used in the determination of obstacles.

The TOF camera 20 has an opening angle, ie a field of view, of about 30 °, which is why the of the TOF camera 20 covered second surveillance area 36 is substantially triangular in plan view. Data reduction will be provided by the TOF camera 20 only takes into account image data, which consists of an overlapping area 38 of the first surveillance area 34 and the second monitoring area 36 come. In the overlap area 38 lies the object 30 which is based on the data of the TOF camera 20 as an obstacle ( 1 ) or as part of the reference contour ( 2 ) is recognized. The stage 32 is in 3 Not shown.

The TOF camera 20 includes a modulated LED light source (not shown) that includes the second monitoring area 36 illuminated. An illumination intensity I (in W / cm ² ) is in 4 opposite the distance x to the front end of the vehicle 10 plotted. In this case, the illumination intensity I increases approximately quadratically in order to achieve essentially the same measurement accuracy over the entire second monitoring area 36 to ensure.

LIST OF REFERENCE NUMBERS

10

 vehicle

12

 underground

14

 laser scanner

16

 scan head

18

 laser beam

20

 TOF camera

22

 field of vision

24

 first secure data connection

26

 second secure data connection

28

 control unit

30

 object

32

 step

33

 shelter

34

 first surveillance area

36

 second surveillance area

38

 overlapping area

100

 Press

Claims (12)

Optical sensor system with a distance sensor ( 14 ), in particular a laser scanner, which provides distance information for a first monitoring area ( 34 ), the first monitoring area ( 34 ) defines a flat surface which is preferably parallel to a substrate ( 12 ), and an evaluation unit, which is formed on the basis of the distance information objects ( 30 . 32 ) too recognize, and upon detection of an object ( 30 . 32 ) emit a warning signal, characterized in that the sensor system additionally includes a depth camera ( 20 ) comprising a three-dimensional image of a second monitoring area ( 36 ) generated. Optical sensor system according to claim 1, characterized in that the evaluation unit comprises a computing device of the distance sensor ( 14 ). Optical sensor system according to claim 1 or 2, characterized in that the depth camera ( 20 ) on the ground ( 12 ), whereby the depth camera ( 20 ) prefers the underground ( 12 ) below the first monitoring area ( 34 ) at least partially detected. Optical sensor system according to one of the preceding claims, characterized in that the evaluation unit is formed, three-dimensional image data of the depth camera ( 20 ), wherein the evaluation unit is preferably designed to compare the three-dimensional images with a reference contour and, based on the comparison of objects ( 30 . 32 ) to recognize. Optical sensor system according to claim 4, characterized in that the evaluation unit is formed, the underground ( 12 ) as a reference contour. Optical sensor system according to one of the preceding claims, characterized in that the second monitoring area ( 36 ) is illuminated by means of a modulated light source. Optical sensor system according to claim 6, characterized in that the light source is formed by the depth camera ( 20 ) further away areas of the second monitoring area ( 36 ) with higher illumination intensity (I). Optical sensor system according to one of the preceding claims, characterized in that the depth camera ( 20 ) is adapted to transmit only data of the three-dimensional image to the evaluation unit, which from the area of the first monitoring area ( 34 ) come. Optical sensor system according to one of the preceding claims, characterized in that the depth camera ( 20 ) is adapted to take a three-dimensional image more frequently than the distance sensor ( 14 ) the first surveillance area ( 34 ). Vehicle ( 10 ) with an optical sensor system according to one of the preceding claims, and a control unit ( 28 ), wherein the evaluation unit is formed, based on the distance information objects ( 30 . 32 ) and recognizing an object ( 30 . 32 ) a warning signal to the control unit ( 28 ). Vehicle ( 10 ) according to claim 10, characterized in that the evaluation unit is designed, the reference contour to acceleration-dependent movements of the self-propelled vehicle ( 10 ). Vehicle ( 10 ) according to claim 10 or 11, characterized in that the vehicle ( 10 ) is self-propelled.

Images