EP3724676A1

EP3724676A1 - Sensor system for three-dimensionally detecting a scene with different photon accumulations

Info

Publication number: EP3724676A1
Application number: EP18825924.6A
Authority: EP
Inventors: Urs Hunziker; Johannes Eckstein; Christian Seiler
Original assignee: Bircher Reglomat AG
Current assignee: Bircher Reglomat AG
Priority date: 2017-12-12
Filing date: 2018-12-11
Publication date: 2020-10-21
Also published as: DE102017129626A1; WO2019115561A1

Abstract

The invention relates to a sensor system (100) as well as a method for three-dimensionally optically detecting a scene (190). The sensor system (100) comprises: (a) a sensor (110) for measuring distances based on a light propagation time; (b) a control unit (140) for controlling the operation of the sensor (110) in such a way that, with a first detection of the scene (190), a first number of photons are accumulated per pixel (322a) of the sensor (110), and with a second detection of the scene (190), a second number of photons are accumulated per pixel (322b) of the sensor (110), wherein the second number is greater than the first number; and (c) a data processing device (150) connected downstream of the sensor (110), which is configured to evaluate the scene (190) based on a first result of the first detection of the scene (190) and/or a second result of the second detection of the scene (190). The invention also relates to different uses of a sensor system (100) of this type.

Description

Sensor system for three-dimensional capture of a scene with different

photons accumulations

Technical area

The present invention relates to a sensor system and a method for three-dimensional acquisition of a scene based on transit time measurements. Furthermore, the present invention relates to several uses of such

Sensor system.

Background of the invention

For opening and / or closing of openings are often by means of

Actuators operated closing body used, which facilitate the handling of the respective closure body for operators or operated fully automatically without any action, for example, when an object to be passed the opening passes into the region of the opening. Such an opening may, for example, be a passageway in a building. On

Closing body may for example be a door or a gate.

To automatically open and close a high level of operational safety

To achieve closing closure bodies, it is known to detect the area in front of or within an opening which can be covered by a closing body by means of an optical sensor system. This can on the one hand be ensured that when shooting the opening accidentally an object, such as a person, is clamped by the closing body.

In addition, in some applications, such a sensor system cause automatic opening of the closing body or the opening. In this context, it is also known that such a sensor system or a data processing device connected downstream of such a sensor system uses known methods of image processing

Performs object detection and releases the opening only if an authorized to pass the openings object in the range of (still

closed opening). Such object recognition can

for example, a face recognition.

From EP 2 453 252 B1, a 3D sensor system is known for the field of application of the monitoring of automatically opening doors and / or gates, which is based on the principle of transit time measurement of light beams emitted by illumination sources and after an at least partial reflection or 180 ° Backscatter be detected by a light receiver. Such sensor systems are commonly referred to as "time-of-flight" (TOF) sensor systems. However, TOF sensor systems have the disadvantage that with

increasing distance d of the object to be detected, the intensity of the (backscattered) measuring light to be detected by a light receiver of the TOF sensor is weakened in two respects. In the case of a punctiform illumination light source without a special focus, it scales

Attenuation of the illumination light emitted by the illumination sources with l / d ^/ 2, where d is the distance to the illumination light source. The same applies to the measurement light, if one a point of the object, at which the

Illumination light is scattered isotropically, perceived as a point light source. As a result, this leads to a l / d 'scaling of the intensity of the received measurement light. In the case of beamforming realized in any manner, for example focusing, of the illumination light, of the measuring light and / or in the case of non-isotropic scattering of the illumination light with a preferred beam

Transmission of the measuring light in the direction of the light receiver, is the

Intensity attenuation correspondingly lower, but still contributes to one significant loss of light output at. This in turn leads to a

correspondingly poor energy efficiency of a TOF sensor system.

It is an object of the present invention to provide an energetically efficient and nevertheless reliable three-dimensional detection of a scene which also includes relatively distant objects.

Summary of the invention

This task is solved by the objects of the independent ones

Claims. Advantageous embodiments of the present invention are described in the dependent claims.

According to a first aspect of the invention, a sensor system for

described three-dimensional optical detection of a scene. The described sensor system includes (a) a sensor for measuring distances based on a light transit time; (b) a control unit for controlling the operation of the sensor such that (bl) accumulates a first number of photons upon a first acquisition of the scene per pixel of the sensor and (b2) a second acquisition of the scene per pixel of the sensor amount of

Photons is accumulated, the second number being greater than the first number; and (c) a downstream of the sensor

A data processing device configured to evaluate the scene based on a first result of the first capture of the scene and / or a second result of the second capture of the scene.

The sensor system described is based on the knowledge that two different (image) data sets can be made available for energy-efficient three-dimensional detection and evaluation of a scene of a data processing device, in which case at least in one sub-region a light receiver of the sensor per pixel a different number of photons were accumulated or collected. Since a relative pixel error caused by an ever-present statistical (photon) noise is smaller with a larger number of photons accumulated per pixel, the statistical accuracy of scene detection becomes larger when a larger number of photons per pixel are accumulated.

In the case of the sensor system according to the invention, the data processing device can decide, depending on a respectively detected characteristic of the scene, whether the first result or the second result is used for a (sufficiently accurate) scene evaluation. A combination of the two results in terms of the most accurate scene evaluation is possible.

The respectively required accuracy of the scene evaluation may depend on the current and / or on an earlier determined optical characteristic of the scene. Thus, for example, an evaluation of the first result of the first detection of the scene, which does not have sufficient accuracy for at least a partial area of the scene, can

Data processing device to cause for a common

Scene evaluation (alone or in addition) to use the second result of the second capture of the scene. Further, what can be energetic

For reasons particularly efficient, the control unit may be made to cause the second capture of the scene with a second number of photons accumulated per pixel. Also, the control unit can

Adjust ratio between a first number of first detections and a second number of second detections within a certain period of time, so that of those surveys that require more energy, only as many (and not more) are made, as it is for a required accuracy of scene capture and

Scene evaluation is required. The "different number of photons accumulated per pixel" may refer to a fixed predetermined period of time, which is determined, for example, by the repetition rate of a repetitive reading of the pixels. This may in particular mean that the time voltage is determined by the time difference between two consecutive pixel readings. In this case, the absolute number of accumulated photons available for one pixel of an image or result is different for both acquisitions. The image with the higher number of photons accumulated per pixel is then the more accurate image, which in particular has a lower (photon) noise.

Alternatively, the periods of time during which the respective photons are accumulated may be different for the two acquisitions. For example, this "difference" may be chosen such that the absolute number of photons accumulated per pixel is the same in both acquisitions. Then, the image associated with the shorter amount of time may be used for scene detection with a higher temporal resolution, which may be particularly advantageous in fast moving objects.

For example, the control unit may provide that of a total number of first acquisitions and second acquisitions, one of the two types of acquisitions that requires more energy will only account for a certain percentage. In this case, the energetically more complex observations can be used for a more accurate scene capture.

The term "scene" may in particular be understood to mean that spatial area which is optically detected by the sensor system. Objects in the scene are recognized by a suitable image analysis. For this purpose, the data processing device can make use of known methods for image evaluation and / or image analysis. The Data processing device may accordingly be an image processing processor configured to apply known methods for image evaluation and / or image processing.

The term "characteristic of a scene" can be understood as the entirety of all spatial structures which are detected by the sensor. In this case, some structures can be recognized as being relevant and other structures as being less or even irrelevant by the data processing device by means of image processing and / or image recognition.

The term "result" may in particular be an analysis of an image captured by the sensor or of a part of an image. The result may also be at least one piece of information about an object in the scene. Such information may be the type of object that (optical)

Nature of the object, the current location of the object, a speed of the object, etc.

This document uses the terms "illumination light" and "measurement light". The term "illumination light" are those

To understand electromagnetic waves, which are emitted by a lighting unit of the sensor and hit the object of the scene. The "measurement light" are the electromagnetic waves scattered back and forth from the object, which are received by the light receiver of the sensor and used for the three-dimensional evaluation of the scene, together with the corresponding TOF distance information.

The terms "optical" and / or "light" may refer to electromagnetic waves having a particular wavelength or frequency or a particular spectrum of wavelengths or frequencies. In particular, the electromagnetic waves used can be assigned to the spectral range visible to the human eye. Alternatively or In combination, electromagnetic waves associated with the ultraviolet (UV) or infrared (IR) spectral regions may also be used. The IR spectral range can extend into the long-wave IR range with wavelengths between 3.5 pm to 15 pm, which are determined by means of the

Light receiver of the sensor can be detected.

According to one exemplary embodiment of the invention, the sensor system further has a holder which is mechanically coupled to the sensor, wherein the holder is designed such that the sensor system can be attached to a stationary holding structure with respect to the scene to be detected.

In terms of clarity, the holder ensures that the described sensor system can be a stationary system which has a certain spatially fixed detection range and thus always monitors the same scene. By comparing different temporally spaced detections of the dependent of the orientation of the sensor system scene spatially stationary objects that are present in the scene can be detected in an image analysis and hidden in a further image analysis with respect to movement profiles. As a result, computing power can be saved and the energy efficiency of the described sensor system can be improved.

The stationary support structure may be directly or indirectly mechanically coupled to a device for controlling a coverage characteristic of an opening to be passed through the object by at least one closure body.

Preferably, this device in addition to a suitable guide or

Storage of the closing body an actuator for moving the

Closing on, in particular for moving the closing body between a closed position and an open position (and vice versa). In the field of building security, the opening may be an entrance, for example for a person or a vehicle, the closure body may be a door, for example a front door or a garage door. The stationary one

Holding structure may be, for example, the stationary frame structure of an entrance, such as the frame of a door.

According to a further embodiment of the invention is the

Data processing device is further configured such that a

Covering characteristic of an opening to be passed by an object by at least one closing body is controllable. As a result, the opening, which is, for example, an entrance (or an exit) of a building, can be automatically monitored in an energetically favorable manner, and by suitable actuation of an actuator, the closing body can be moved automatically between an open position and a closed position. For this purpose, the data processing device of the described

Sensor system coupled with the control of a known control system for a closure body.

According to a further embodiment of the invention, the control unit has an interface and the control unit is further configured such that the accumulation of the first number of photons and / or the accumulation of the second number of photons is controllable by an external control signal.

Illustratively, the described interface allows external control of the operation of the sensor device. In this case, triggered by an external device with respect to the sensor device, the accuracy of at least one of the two scene captures can be adapted to the respective requirements. In simple but very advantageous embodiments for most applications is by such an external trigger signal simply between the two types of detection with different

Switched photon accumulation per pixel. For example, the external control signal may be indicative of the state of a system attached to the sensor system, for example, a

Monitoring system, an automatic door opening system, etc. Further, e.g. be detected by another sensor that an object in

Movement is, which can be reliably detected even with "little accumulated photons". In this case, it is sufficient if an energy-consuming, more complex second acquisition of the scene is not carried out so frequently in comparison with the less energy-consuming first acquisition of the scene. Possibly. For example, after a one-time scene capture of the second type, the scene can only be performed with first acquisitions (of the first type).

In another very specific embodiment, the interface is connected to a manually operated button for activating an automatic door. The control signal transmitted via the interface may then cause the scene to be detected in a room area located near the relevant door opening with a raised area

Accuracy is performed. Such a different accuracy for different subregions of the scene can, for example, be achieved by a targeted combination of pixels by means of so-called binning, which

will be explained below.

According to a further embodiment of the invention, the first

Detecting the scene with a first measurement period and the second detection of the scene performed with a second measurement period, wherein the second measurement period is longer than the first measurement period. This has the advantage that the different photon accumulation can be realized in a simple manner by a suitable adaptation of the (different) integration times of the light receiver of the sensor.

Illustratively, the first capture of the scene takes place with a first one Exposure time and the second detection of the scene takes place with a second exposure time, which is longer than the first exposure time. The

Exposure times, which naturally give the temporal resolution of the

Sensor systems can determine configuration and / or

Depend on measurement parameters of the described sensor system. Such

Measurement parameters are, for example, (a) the (expected) distance to an object to be detected, (b) an energy available for the operation of the sensor system, in particular for a (pulsed or temporally modulated) illumination of the scene, which illumination is required for a TOF measurement is, (c) an optical reflection and / or scattering behavior of the surface of a

capturing object in the scene, etc.

According to a further embodiment of the invention is the

Data processing device coupled to the control unit and the

Data processing device is further configured, depending on the first result and / or the second result via the control unit

Accumulate the number of photons for at least one subsequent first detection and / or to control a subsequent second detection.

This means that the (different) photon accumulation can be adapted dynamically to the characteristic of the three-dimensional scene to be detected. The characteristic of the scene depends on the actual (and not the expected) optical properties of at least one object of the scene. Therefore, the scene evaluation dependent control of photon accumulation provides a control mechanism for the

Photon accumulation.

It should be noted that a suitable photon accumulation can also be found by a learning process in which one and the same scene is recorded and evaluated multiple times or in which scenes are similarly recorded and evaluated. It is further pointed out that a repetition rate or frequency with which subsequently the scene with first acquisitions and / or with second acquisitions is detected and evaluated can also depend on the first result and / or on the second result. For example, one can

energetically efficient scene evaluation done by first with a relatively low repetition rate the scene or a therein

located object is detected. If it is determined that this object changes with some dynamics or another object enters the scene, then scene capture and scene evaluation can be performed at a comparatively high repetition rate. The high repetition rate may be based on first scene acquisitions with a first photon accumulation and / or on second scene captures with a second photon accumulation

Refer to photon accumulation.

According to a further embodiment of the invention, the control unit and the sensor are configured in such a way that the second number of accumulated photons, which is higher in comparison to the first number, is realized by combining adjacent single pixels into one pixel. Such

Summing up pixels, which is also known as binning, has the advantage of being uncomplicated and also fast or dynamic

can be carried out. In particular, it is not necessary to equip the sensor with two different light receivers which, instead of combining the individual pixels, have larger pixels with a correspondingly reduced spatial resolution.

It is noted that over the area of the concerned

Binning can also be performed locally in only at least a portion of the active surfaces of the light receiver. Although this leads to an inhomogeneous spatial resolution, which is not necessarily desired. The disadvantage of such inhomogeneous spatial resolution However, it is overcompensated in many applications by the advantages of different photon accumulation described above. A local "binning" can, at least in some known sensors without special electronic or apparatus elements simply by a

corresponding control of the light receiver take place, which control determines the "binning" and thus the operating mode of the sensor.

It is further noted that the binning described also responds to at least one previously detected and evaluated

Scene characteristic (automatically) can be activated.

According to a further exemplary embodiment of the invention, the sensor has an illuminating device for emitting illuminating light onto an object to be detected in the scene and a light receiver for receiving measuring light, which is illuminating light scattered at least partially on an object of the scene and impinging on the light receiver. The sensor is configured to measure the light propagation time based on (a) a measurement of the time between emitting a pulse of the illumination light and receiving the measurement light or measurement light pulse associated with the pulse and / or (b) measuring a phase shift between a temporal modulation of the illumination light and an associated modulation of the

Measuring light. This has the advantage that the described sensor system can be realized in a suitable manner depending on the particular application. In some embodiments, the sensor is configured such that it is possible to switch between the two different measurement principles "pulse mode" and "phase measurement" flexibly or, if necessary, between the two different measurement principles.

According to a further embodiment of the invention is the

Lighting device configured to provide the illumination light with a deviating beam cross section of a circular shape. This has the advantage that in scenes which are "not round", insufficient illumination of Corner areas of the scene can be avoided. Although insufficient illumination of the corner regions could possibly be prevented by an overall increased intensity of the illumination light, in this case the middle regions of the scene would be overexposed, which would be very disadvantageous, at least from an energetic point of view.

In many embodiments or applications of the sensor system, it is advantageous if the illumination light has a rectangular beam cross section. Preferably, the beam cross-section is adapted to achieve as homogeneous as possible illumination of the shape he scene to be detected. Suitable shaping of the beam cross section can not only be achieved by a corresponding shaping of the luminous area of the illumination device, but also the beam cross section can be suitably adapted by optical components such as mirrors and refractive optical elements (for example lens system). Diffractive optical elements (DOEs) can also be used, which optionally even allow a dynamic and / or scene-dependent shaping of the beam cross-section.

According to a further embodiment of the invention is the

Lighting device configured to emit for the first detection of the scene, the illumination light with a first illumination intensity and for the second detection of the scene, the illumination light with a second

Emitting illumination intensity, wherein the second intensity is greater than the first intensity. This has the advantage that the inventive

different photon accumulation can be realized in a particularly simple manner simply by a corresponding control of the lighting device.

It should be noted that the first illumination intensity and / or the second illumination intensity may be homogeneous or alternatively inhomogeneous over the entire scene. In the case of an inhomogeneous lighting Intensity preferably those portions of the scene are illuminated with a higher illumination intensity, which are particularly relevant for object detection or in which are relevant objects of the scene.

It is further noted that (regardless of a different illumination for both scene acquisitions) in further embodiments, the characteristic of the illumination, in particular the illumination intensity, depending on the ambient light and / or the ambient conditions can be adjusted. So, for example, at an intense

Sunlighting required for the transit time measurement light (and thus the illumination light) to be sufficiently bright. clear

In other words, the sunlight has to be "drowned out". An adaptive control or regulation of the operation of the illumination device with, if possible, a reduction of the intensity of the illumination light thus enables an energetically efficient operation of the sensor system. The same applies to disturbances due to external influences, especially weather conditions such as rain, snow

(Snowfall and / or lying snow in the scene with a high reflection), hail, fog, smoke, hovering particles, etc.

Sunlight and / or external disturbance can significantly degrade TOF measurement conditions. However, by an energy-optimized operation of the sensor system, the described sensor system can still work (with a reduced range of functions). For example, by a reduced frequency or

Repeat rate of scene captures detection of moving objects only slower or with a certain delay. Further, in situations having high energy consumption per scene capture (and scene evaluation), a basic functionality of the sensor can be ensured that is illuminated with higher intensity, but for the purpose of energy saving pauses between the different scene captures are inserted (automatically) , According to a further embodiment of the invention is the

Lighting device (i) at least indirectly coupled to the data processing device and (ii) configured to control a lighting characteristic of the emitted illumination light in response to the first result and / or the second result.

This clearly indicates that at least one of the two results is a controlled variable for adjusting the characteristic of the lighting. In this way, a dynamic adaptation of the illumination to a previously detected and expected scene characteristic of the scene can advantageously be carried out, and a scene-dependent, at least approximately optimal illumination can be realized.

The control of the lighting device can be of current

Ambient conditions that are reflected in the first result and / or in the second result of the scene evaluation. Such

Ambient conditions may be weather conditions such as the presence of rain, snow, hail, fog, smoke, suspended particles, etc. in the scene.

According to a further embodiment of the invention is the

Lighting device configured, the lighting characteristic of the emitted illumination light in response to an external

Control signal to control.

The control signals can via a corresponding data input of the

Sensor system, in particular a data input of the control unit, are received. In order to realize a suitable adaptation of the illumination characteristic, the control of the illumination device can be carried out by the control unit of the sensor system described above become. A control or adaptation of the lighting characteristic can therefore not (only) of those generated in the context of the TOF measurement

Depending on the reception data or the results of the scene evaluation. According to the embodiment described here, a suitable adaptation of the illumination characteristic takes place externally.

In principle, external control signals can be indicative of all features and / or states which have an influence on the backscattering behavior of the

Have illumination light. Such a feature is, for example, a (color) change of an object to be detected and / or an object newly entering or leaving the scene, which object

Backscattering at least partially influenced.

According to a further embodiment of the invention is the

Lighting characteristic determined by at least one of the following features: (a) illumination intensity for at least a portion of the scene, (b) differences in illumination intensity between different

Subregions of the scene, (c) wavelength of the illumination light, (d) spectral distribution of the illumination light, (e) polarization direction of the illumination

Illumination light, and (f) intensity distribution for different

Polarization directions of the illumination light.

By an application-specific use of one of these features or by a suitable combination of at least two of these features, the objects to be recognized can be illuminated particularly well and recognized as a result with particularly high accuracy and reliability.

With regard to the size of the subregions of the scene, there are no particular restrictions. In extreme cases, a partial area can be assigned to exactly one pixel of the sensor. The wavelength, frequency or color of the illumination light and the spectral distribution of the illumination light can by a suitable control of spectrally different light-emitting elements, in particular LEDs with

different color, varies or adapted to the expected characteristics of the scene. In particular, the selected wavelength or spectral distribution of the illumination light may depend on the color and thus on the optical backscattering behavior of the object.

The polarization direction of the illumination light can be adjusted in a known manner, for example by the use of polarization filters.

All mentioned characteristic features can optionally be adapted dynamically to changing scenes, so that the best possible illumination of the scene can always be achieved. This can be an optimal

Lighting characteristic can also be determined according to the principle of "try-and-error" or by other statistical optimization procedures. This can be done dynamically during a real operation of the sensor system or as part of a calibration by means of a detection of suitable reference objects.

According to a further aspect of the invention, a method for the

described three-dimensional optical detection of a scene. The method includes (a) providing a sensor for measuring distances based on a light transit time; (b) controlling the operation of the sensor such that (bl) accumulates a first number of photons upon a first detection of the scene per pixel of the sensor and (b2) on a second detection of the scene per pixel of the sensor a second number Photons is accumulated, the second number being greater than the first number; and (c) a

Evaluation of the scene by means of a downstream of the sensor

Data processing device based on a first result of the first detection of the scene and / or a second result of the second detection of Scene.

The method described is based on the knowledge that the scene can be detected and evaluated in an energy-efficient manner by three-dimensional recording and evaluation of a scene based on two different (image) data sets. In order to achieve a certain degree of accuracy, it is not always necessary to carry out that scene detection and scene evaluation, which is the more complex from the energy point of view.

According to an embodiment of the invention, the method further comprises (d) detecting an object in the scene; (e) comparing the detected object with at least one comparison object; and,

if the object agrees with a comparison object within predefined permissible deviations, (f) identifying the object as an object authorized for a specific action.

The approved action may, for example, be an authorized passage through an opening in a building, which opening is closed by a closure body prior to identification as an approved object and is opened only after successful identification by a corresponding movement of the closure body. The objects to be identified may preferably be persons and / or vehicles. Successful identification may be to control or activate a closure mechanism for a closure body prior to opening a building. For some

Embodiments is merely determined by a (very simple) object recognition that it is a living object to a human and not an animal, such as a bird or a bat, is. An opening of the passage can then take place only if a human was recognized.

By using the 3D TOF (depth) information of the sensor can at Persons, for example, a reliable face recognition can be realized, which can be used to activate a shutter mechanism. Here, certain areas of the scene, for example, by a

particularly high intensity illumination and captured with increased accuracy, which in turn leads to an energy reduction over the whole scene in high definition or full illumination.

Instead of persons embossed signs, in particular license plates of vehicles can be reliably detected and recognized in other embodiments. Namely, by the 3D information, the embossing of the mark can be recognized well, resulting in improved reading reliability. In particular, in this way a possible forgery of a

Identification of the license plate.

According to a further aspect of the invention, a use of a sensor system described above for controlling a

Covering characteristic of an opening to be passed by an object by at least one closing body.

The described use is based on the finding that a detection and evaluation of an optical scene made in an energy-efficient manner can be used advantageously in passages which can be closed by a closing body. This is especially true for passages which a closure or a

Covering characteristic, which is controlled by the described sensor system or at least co-controlled. Because such sensor systems

Usually powered by the closure systems for the closure body, it is particularly important to get along with a relatively small amount of energy and still reliable

Scene evaluations to come. By the inventive use of the above-described

Sensor system can be monitored in an energetically efficient manner and larger distances, which naturally leads to an earlier detection of an opening requirement of the closure body, which can be of great advantage, especially in fast-moving objects. Furthermore, the scene can be detected with a wider detection angle, which, for example, leads to an early detection of itself transversely to the opening

Cross-traffic and thus lead to a more reliable detection of objects in the security area of the closure system. This can be suppressed in cross traffic unnecessary opening request.

According to one embodiment of the invention, the opening is an entrance or an exit, in particular an emergency exit in a building. By the

Recognizing an existing, but possibly non-moving object in a single passage, an input or output monitors, in particular a blocked emergency exit detected, and the corresponding information to an affiliated system, for example, to a monitoring system, transmitted.

According to a further embodiment of the invention, the object is a person or a vehicle. In this case, the building may in particular be a house or a garage.

According to a further aspect of the invention, there is described a use of a sensor system described above for detecting and / or controlling traffic flows of objects moving through a scene of the sensor system, the scene being represented by a spatial

Detection range of the sensor system is determined. This described use is based on the finding that it is in a traffic detection and / or traffic flow control on a

Energy-efficient sensor technology arrives, since this sensor is typically constantly in operation. The objects relevant to the traffic flow in question can be, for example, persons, vehicles, products such as e.g. Be packages, suitcases, etc. Since a plurality or even a multiplicity of 3D sensors are usually used for such applications, energy savings have a particularly positive effect here.

It should be noted that embodiments of the invention have been described with reference to different subject matters. In particular, some embodiments of the invention are described with apparatus claims or use claims and other embodiments of the invention with method claims. However, it will be readily apparent to those skilled in the art upon reading this application that, unless explicitly stated otherwise, in addition to a combination of features belonging to a type of subject matter, any combination of features that may result in different types of features is also possible

Subject matters belong.

Before describing exemplary embodiments of the invention with reference to the drawings, some technical considerations related to the invention will be described here.

In general, TOF-based sensor systems can be subdivided into two fundamentally different classes both with regard to the illumination light and with regard to the measurement light, which can be combined as desired. Bl: The first alternative (Bl) for the lighting is characterized by the fact that the scene by means of a single illumination light beam high

Focusing and low divergence (ie high collimation) is scanned sequentially. For each position of the illumination light beam in the scene, a measurement of the duration of the illumination light and the measurement light

performed. The scanning can be realized using movable optical components, in particular mirrors. Alternatively or in

Combination can be used for sequential scanning of the scene with the

Illuminating light beam can be used a solid body, which manages without mechanically moving parts and has integrated photonic structures or circuits. With a suitable control of these structures, the illumination light beam is then directed to the desired location of the scene. Such a solid is known for example from US 2015/293224 Al.

B2: The second alternative (B2) for lighting is characterized by the fact that the entire scene is illuminated (all at once and flatly). If necessary, the intensity of the illumination light in selected subregions of the scene can be (selectively) increased in order to enable improved 3D object detection at these locations. Such spatially uneven distribution of the illumination light can be done without moving optical components

For example, by means of a so-called. Diffractive optical element (DOE) done.

Ml: A first alternative (Ml) for the measurement is based on pulsed

Illumination light beams. In this case, a "travel time" of a light pulse is detected on the receiver side for each pixel within a time window, and the distance is derived therefrom.

M2: The second alternative (M2) for the measurement is based on a temporal, preferably sinusoidal, modulation of the illumination light with a predetermined frequency, with appropriate values for this frequency depending on the expected transit time or the maximum detection distance. On the side of the light receiver, the phase difference is measured for each pixel and derived therefrom the distance information.

Both measuring principles Ml and M2 are based on an integration of the number of photons or the photoelectrons generated in the light receiver, which arrive on each pixel to be measured. In this context, it is obvious that an ever-present light or photon noise depends on the number of photons accumulated in a pixel. Therefore, the higher the number of accumulated photons, the more accurate the distance information obtained from the TOF measurement becomes.

Further advantages and features of the present invention will become apparent from the following exemplary description of presently preferred embodiments. The individual figures of the drawing of this document are merely schematic and not to scale.

Short description of the drawing

Figure 1 shows the use of a sensor system for controlling a

Covering characteristic of an opening by means of closing bodies designed as sliding doors.

Figure 2 shows the use of a sensor system for detecting a

Traffic flow of transported on a conveyor belt objects.

Figures 3a and 3b illustrate a collection of single pixels of a light receiver. FIGS. 4a to 4c show different beam cross sections of an illumination light for adapting the illumination to the shape of the illumination

capturing scene.

Detailed description

It should be noted that in the following detailed description, features of different embodiments, with the corresponding features or components of another

Embodiment are the same or at least functionally identical, are provided with the same reference numerals or reference numerals, which are identical in the last two digits with the reference numerals of corresponding same or at least functionally identical features or components. In order to avoid unnecessary repetitions, features or components already explained on the basis of a previously described embodiment will not be explained in detail later.

Before the figures are described in detail in this section, some aspects of a number of embodiments are explained below.

Given the importance of energy - efficient operation in many applications of 3D sensor systems, it can be applied to the

The light receiver can optimize the optical energy of the measuring light by illuminating the scene differently, depending on the nature of the scene.

So z. As in a door opening system for activating a door opening above all relevant when approaching objects at a higher speed from the outside to the monitored zone. Suction Spontaneous starts of objects inside the monitored zone, which (after a short time) also with a high

Being able to move at speed are rather atypical. So can be at a (easy to implement) embodiment of the invention, especially the edge regions of the scene with a particularly high intensity of

Monitor illumination light. An object recognition inside the

monitored zone can easily be done with a "wake-up delay", which allows monitoring with a less powerful inside

To realize illumination light. This can be realized by a function-dependent illumination in which an energy optimization can be achieved by controlling the radiation-beam-dependent illumination energy over a time average. Different strategies can be used.

(1) In the case of an illumination device which consists of a plurality of individual elements each supplying a certain intensity of illumination, for example an array of light-emitting diodes, the illumination intensity of at least one individual element can be varied with respect to the other elements. This variation can already be constructive in the construction of the corresponding

Lighting device can be realized, for example by the use of light emitting diodes of different intensity. Alternatively or in combination, this can also be achieved by the type of control of the LEDs, each with an individually adjustable current. Such

Current adjustment can also be done when calibrating the sensor system

for example, by way of a (first) installation of the sensor system done.

In addition, individual power control can also be performed dynamically during operation of the sensor system. So can

For example, when from a solid angle region little intensity of measurement light strikes the light receiver, light emitting diodes associated with this solid angle region are more energized. Such a procedure is particularly well suited for the illumination alternative B2 described above in combination with the two measurement principles M1 or M2 also described above. (2) In lighting devices, which in accordance with the above-described lighting principle Bl means of a

Lighting beam scanning the entire scene sequentially or scan, is typically at any time the current solid angle of the

Illuminating beam known. By varying the intensity of the

Depending on the respective spatial angle, illumination light (ray) can thus be controlled or regulated in a controlled manner depending on the optical characteristic of the detected scene (and optionally with dynamic control or regulation depending on the intensity of the measurement light scattered back from the respective solid angle). Thus, for example, a static scene can be measured, with the appropriate intensity for each solid angle being taught to illuminating light.

An optional dynamic adjustment of the illuminating light intensity can be carried out both in real time, in that those solid angle ranges from which only a small intensity of measuring light is received, directly, i. without delay, be more illuminated. Such a variation may also be adaptive from one scene capture to the next

Scene capture are performed. This means that depending on the evaluation result of a last scene capture for the next

Scene capturing those solid angle areas that were "overexposed", now be "exposed" with a lower intensity of illumination light. Similarly, "underexposed" solid angle ranges may be brightened by a stronger "exposure" at a next scene capture. Such a procedure is particularly well suited for the lighting alternative Bl described above in combination with the two measuring principles Ml or M2.

(3) Regardless of the two aforementioned strategies, the frequency or repetition rate of entire scene acquisitions can also be adjusted if necessary reduce (and turn off the lighting between the scene captures), which also contributes to a (further) energy savings. Thus, for example, in a sensor system for a gate, where for a door opening request (i) during the day both vehicles and persons and (ii) at night only persons must be detected, during the night time, the frequency or the repetition rate of scene captures to reduce. With only halving the frequency, this results in a nearly 50% saving in energy consumed by the sensor system. A reduction in the frequency of

"nocturnal" scene captures without a (precise) detection of vehicles is usually not critical because pedestrians usually have a lower speed compared to vehicles, so that their reliable detection even with a reduced number of scene captures and thus with a correspondingly lower Time resolution is possible.

Similar procedures with a reduction of the repetition rate are always possible if a certain time delay is permissible for a detection of a movement and / or if only less fast moving objects have to be recognized. Then, namely, a "lighting break" can be inserted regularly between two scene captures.

(4) In the case of a door opening system with the door open, as long as there are objects in the transit zone or in a security area, only that passage zone must be monitored, because closing the door is in any event out of the question, as long as there are objects within the security area are located (and also recognized). For example, the lighting for the TOF sensor system described in this document can be reduced to a subarea of the scene to be monitored, thereby saving energy. Alternatively, the detection of an object which is likely to pass through a passage sufficient to trigger a corresponding opening signal, so that if necessary existing in the scene further object need not be observed and saved a corresponding amount of energy can be.

(5) Furthermore, when using the TOF sensor system described in this document for monitoring a door which can be closed by a door, for example, a higher temporal and / or spatial resolution in the vicinity of the door (so-called near zone) may be required. Then it can be ensured with a particularly high degree of reliability that no extremities (hand, foot, fingers, etc.) of a person are trapped by an accidental closing of the door.

Furthermore, for a detection of a moving object, for example, which is located relatively far away from the door (in a so-called remote zone), a comparatively low temporal and / or spatial resolution is sufficient than would be required for the near zone. This insight can be said to

Use energy optimization that is much less for the remote zone

Monitoring points with sufficient illumination intensity or

Lighting energy must be supplied (low spatial resolution), while in the Nahzone a feinmaschigere monitoring (greater spatial resolution) of the scene (and thus a higher illumination intensity per space sector) is necessary.

(6) In addition, irrespective of the strategies described above, in many applications the illumination intensity in a portion of the scene can be selectively focused by appropriate focusing of the scene

Illumination light can be increased. Although the lateral resolution of the scene detection decreases, because such a focusing part of the

Subarea is supplied with less illumination light than another part of this subarea. However, if such a measure still provides a sufficient spatial resolution for the particular application

is guaranteed, then this is a further significant energy savings possible. 1 shows the use of a sensor system 100 for controlling a coverage characteristic of an opening 184 depending on the characteristic of a scene 190 monitored by the sensor system 100. According to the embodiment shown here, the opening 184 is a passenger entry into a building or garage entrance for motor vehicles a garage. The corresponding input structure is provided with the reference numeral 180.

The input structure 180 includes a stationary support structure 182 which includes a frame and a guide for two sliding doors

Closing 186 represents. The sliding doors 186 can each be represented by means of a motor 187 along the two thick double arrows

Move instructions are moved. The actuation of the motors 187 takes place, as explained below, by means of the sensor system 100 described in this document.

The sensor system 100 has a TOF sensor 110, a control unit 140, a data processing device 150 and a database 160. The TOF sensor 110 contains all the optical components of the sensor system 100

accommodated. On a housing of the TOF sensor 110, a holder 111 is provided. By means of this holder 111, at least the TOF sensor 110 is attached to the stationary support structure 182 in a mechanically stable and spatially fixed manner. The sensor system 100 (as opposed to the representation of FIG. 1) is preferably constructed as a module which, in addition to the TOF sensor 110, also has the control unit 140, the data processing device 150 and the database 160 within a compact design.

According to the exemplary embodiment illustrated here, the control unit 140 has an interface 142 via which an external control signal 142a can be received. The external control signal may be from an attached system (not shown), such as a

Monitoring system, which controls the operation of the sensor device 100 depending on external parameters. Such an external parameter may, for example, be a previously known object property.

Signaling data transmitted via the interface 142 may also include information about the detected and evaluated scene 190. Such information may be, for example, the information that a license plate advertised for search was recognized, that a parking lot was illegally occupied, that a suspicious object is in the monitored scene 190, etc. In this case, a corresponding flow of information is provided by the sensor system 100 More specifically, from the control unit 140 to the attached system.

Alternatively or in combination, in the exemplary embodiment described here, the data processing device 150 is also provided with an interface 152, which can also receive an external control signal, which is provided with the reference numeral 152a. This control signal 152a can be used to make the operation of the data processing device 150 at least partially dependent on external information.

In particular, an a priori knowledge about an object 195 can be transmitted via the control signal 152a for an improved evaluation of the detected scene 190. As described in detail above, the scene detection is done by a "two-stage detection" with different ones

Photons accumulations.

The TOF sensor 110 comprises an illumination device 130, for example an array of light-emitting diodes, which illuminates the scene 190 and thus also the object 195 in the scene 190 with a pulsed and temporally modulated illumination light 131. Further, the TOF sensor 110 includes a Light receiver 120, which receives illumination light 131 backscattered from the object 195, which is referred to as measuring light 196 in this document. The spatial detection of the scene 195 takes place on the basis of the principles explained in detail above of a transit time measurement, which is also referred to as Time Of Flight (TOF) measurement. The corresponding TOF data is transferred to the data processing device 150. This can be done directly or indirectly via the control unit 140.

It should be noted that the illumination device 130 in addition to the illumination units shown in Figure 1 can also have other lighting units that illuminate the scene 190 from a different angle. Likewise, the two lighting units can also be arranged outside the housing of the TOF sensor 110 and thus of the

Light receiver 120 further spaced. On the principles of

performed TOF measurement does not change anything.

According to the exemplary embodiment illustrated here, based on this 3D data, the detected optical scene 190 is identified by means of suitable methods

Image evaluation evaluated. For this purpose, first data are obtained from the above-described first scene capture with the first photon accumulation and / or second data from the second one described above

Scene capture used with the second photon accumulation.

To capture a motion profile of the object 195, an image of the scene 190 with the first is within certain time intervals

Scene capture and / or the second scene capture detected. Thus, the data processing device 150 is capable of determining not only its velocity as an absolute value but as a motion vector (with direction information) based on respective positional shifts of the object 195. A knowledge of the exact position and / or the movement profile of the object 195 can then be advantageously used to both motors 187 in a suitable manner by the data processing device 150 to control. According to the embodiment shown here, with a closed opening 184, the sliding doors 186 are only opened when the object 195 actually moves in the direction of the opening 184. If the object 195 is a vehicle of so-called cross-traffic, which essentially moves past the opening 184, then the corresponding motion vector of the

Data processing device 150 is detected and there is no opening of the sliding doors 186th

After passing the object 195 through the opening 184, it can be quickly closed again, for example, to pass further

Prevent objects. In this connection, it is obvious that not only the time at which the opening must be released, but also the period of time for which the opening 184 must be kept open depends on the speed of the object 195 with which it is moving in the direction of the opening 184 and through the opening 184 therethrough. By detecting the object 195, an optimization of the operation of the input structure 180 can be achieved by a suitable control of the motors 187

In fact, people should open a passageway earlier than a lifeless object and close it later to relieve people of the fear of colliding with the sliding doors.

According to the embodiment illustrated here, the sensor system 100 is also capable of performing object recognition. This is what the

Data processing device 150 to a stored in the database 160 record of reference objects corresponding to selected objects that are authorized to pass through the opening 184. This means that with a suitable approach of the object 195 to the input 184, the sliding doors 186 are opened only when the detected object 195 at least approximately coincides with one of the stored reference objects. This clearly means that in the use of the sensor system 100 described here, the coverage characteristic of the opening 184 depends not only on the motion profile of the object 195, but also that object-based access control still takes place.

FIG. 2 shows a further use or a further use of the invention

Sensor system 100. For reasons of clarity, only the TOF sensor 110 of the sensor system 100 is shown in FIG.

According to the embodiment illustrated here, the TOF sensor 110 detects a traffic flow of (different) objects 295a, 295b, 295c, 295d, and 295e that are on a conveyor belt 298 and move through a scene 290 along the direction of movement represented by an arrow. Reliable knowledge of the number and / or type of objects 295a-295e may be used in the field of logistics for traffic flow control. Merely an example of such control of a

Traffic flow is the control of luggage transport in an airport. In this case, labels on the relevant objects 295a - 295e can also determine the type of the respective object. It should be noted, however, that use in an airport is just one example of a variety of other traffic control applications.

FIGS. 3a and 3b illustrate a combination of individual pixels of a light receiver 320a designed as a semiconductor or CCD chip;

320b. The light receiver 320a has a plurality of light-sensitive or

Photon collecting pixels 322a. As shown here

Embodiment, the pixels 322a are associated with the full spatial resolution of the light receiver 320a, which resolution is predetermined by the semiconductor architecture of the chip 320a. In the light receiver 320b, four of the light-sensitive pixels (for a full resolution) are respectively added to a higher-level pixel 322b (for an increased resolution)

Photon accumulation per pixel at the expense of a reduced spatial

Resolution). Illustratively, a pixel 322b accumulates four times as much light as compared to a single pixel 322a.

Such a "binning" reduces the required (minimum) intensity of the detected measuring light, which is needed to evaluate the corresponding image area of the scene. Since the intensity of the measurement light depends directly on the intensity of the illumination light, binning reduces the intensity of the illumination light and thus reduces the energy consumption of the sensor system.

The described "binning" can also be realized dynamically by a corresponding control of one and the same light receiver 320a or 320b. The light receiver is either in a first operating mode (with full resolution) or in a second operating mode (with

operated photon-collecting pixels) operated. Switching between different modes of operation may be controlled by external control signals. Alternatively or in combination, such switching may also depend on the result of a scene evaluation, so that the "binning" mode of operation is regulated for a next scene capture.

It should be noted that more than two different

Operating modes, each with a different strong summary of pixels can be used. Furthermore, it is possible to combine a different number of individual pixels into different pixels of the light receiver in each case. Then individual can

Portions of the scene with a higher spatial resolution (and less photon accumulation) and other portions of the scene with a lower spatial resolution (and higher photon accumulation) be recorded. The described local and differently strong grouping of pixels can be dynamic or adaptive in exactly those

Subareas can be performed in which there is currently a specific object.

Figures 4a to 4c show different beam cross sections of a

Lighting light to adjust the lighting to the shape of the

capturing scene. A first illumination light 431a illustrated in FIG. 4a has a substantially circular beam cross section and is preferably suitable for "round scenes". However, for most applications which do not detect (and evaluate) a "round scene", a beam cross section deviating from a circular shape is suitable. FIG. 4b shows an illumination light 431b with an elliptical beam cross section. FIG. 4c shows

Illumination light 431c with a rectangular beam cross section. As

already mentioned above, the beam cross section through a

corresponding shaping of the illumination device and / or by optical components such as mirrors and refractive optical elements (e.g.

Lens system) are suitably adapted to the scene to be detected. Diffractive optical elements (DOEs) can also be used, which optionally even allow a dynamic and / or scene-dependent shaping of the beam cross-section.

It is noted that the term "comprise" does not include other elements

excludes and that the "one" does not exclude a plurality. Also, elements described in connection with different embodiments may be combined. It should also be noted that reference signs in the claims should not be construed as limiting the scope of the claims. Reference sign:

100 sensor system

110 TOF sensor

111 bracket

120 light receiver

130 lighting device

131 illumination light

140 control unit

142 interface

142a external control signal

150 data processing device

160 database

152 interface

152a external control signal

180 entrance structure

182 stationary holding structure

184 opening / entrance

186 closing body / sliding door

187 engine

190 scene

195 object

196 measuring light

290 scene

295a-e objects

298 conveyor belt

320a / b light receiver / sensor chip

322a pixels

322b parent pixel / pooled pixel a Round cross-section illumination lightb Elliptical cross-section illumination lightc Illuminating light of rectangular cross-section

Claims

Patent claims

A sensor system (100) for three-dimensional optical detection of a scene (190) comprising the sensor system (100)

a sensor (110) for measuring distances based on a

Transit Time;

a control unit (140) for controlling the operation of the sensor (110) such that

at a first detection of the scene (190) per pixel (322a) of the sensor (110) a first number of photons is accumulated and

upon a second acquisition of the scene (190), accumulating a second number of photons per pixel (322b) of the sensor (110), the second number being greater than the first number; and

a data processing device (150) connected downstream of the sensor (110) and configured to evaluate the scene (190) based on a first result of the first capture of the scene (190) and / or a second result of the second capture of the scene (190).

2. Sensor system (100) according to the preceding claim, further comprising a holder (111) which is mechanically coupled to the sensor (110), wherein the holder (111) is formed such that the sensor system (100) at a reference can be attached to the scene to be detected stationary holding structure (182).

A sensor system (100) according to any one of the preceding claims, wherein the data processing means (150) is further configured such that a coverage characteristic of an aperture (184) to be passed by an object (195) is controllable by at least one closure body (186).

4. The sensor system according to claim 1, wherein the control unit has an interface and the control unit is further configured such that the accumulation of the first number of photons and / or the accumulation of the second one Number of photons from an external control signal (142 a) is controllable.

The sensor system (100) according to one of the preceding claims, wherein the first detection of the scene (190) is performed with a first measurement duration and the second detection of the scene with a second measurement duration, wherein the second measurement duration is longer than the first measurement duration.

6. The sensor system according to claim 1, wherein the data processing device is coupled to the control unit and the data processing device is further configured via the control unit depending on the first result and / or the second result (140) accumulating the number of photons for at least one

to control subsequent first detection and / or subsequent second detection.

A sensor system (100) according to any one of the preceding claims, wherein the control unit (140) and the sensor (110) are configured such that the higher second number of accumulated photons compared to the first number is obtained by combining adjacent single pixels (322a ) to a pixel (322b).

A sensor system (100) according to any one of the preceding claims, wherein the sensor comprises

an illumination device (130) for emitting illumination light (131) onto an object (195) to be detected in the scene (190) and

a light receiver (120) for receiving measurement light (196) which is at least partially scattered on an object (195) of the scene (190) Illuminating light (131) incident on the light receiver (120), wherein the sensor (110) is configured to measure the light propagation time based on

(a) a measurement of the time between a transmission of a pulse of the illumination light (131) and the reception of the pulse belonging to the

Measuring light (196) and / or

(b) a measurement of a phase shift between a temporal

Modulation of the illumination light (131) and an associated modulation of the measurement light (196).

9. sensor system (100) according to the preceding claim, wherein the

Lighting device (130) is configured

to provide the illumination light (131) with a beam cross-section (431b, 431c) deviating from a circular shape (431a).

10. Sensor system (100) according to one of the two preceding claims, wherein the illumination device (130) is configured,

for the first detection of the scene (190) emitting the illumination light (131) with a first illumination intensity and

for the second detection of the scene (190), emitting the illumination light (131) at a second illumination intensity, the second intensity being greater than the first intensity.

11. Sensor system (100) according to one of the three preceding claims, wherein the illumination device (130)

(i) is at least indirectly coupled to the data processing device (150) and

(ii) is configured to provide a lighting characteristic of the emitted one

To control illumination light (131) in response to the first result and / or the second result.

12. Sensor system (100) according to one of the four preceding claims, wherein the illumination device (130) is configured, the

Lighting characteristic of the emitted illumination light (131) in

To control dependence on an external control signal (142a).

13. Sensor system (100) according to one of the two preceding claims, wherein

the lighting characteristic is determined by at least one of the following features:

(a) illumination intensity for at least a portion of the scene (190),

(b) differences in illumination intensity between different

Parts of the scene (190),

(c) wavelength of the illumination light (131),

(d) spectral distribution of the illumination light (131),

(e) polarization direction of the illumination light (131), and

(f) intensity distribution for different polarization directions of the

Illuminating light (131).

14. A method for three-dimensional optical detection of a scene (190), comprising the method

Providing a sensor (110) for measuring distances based on a light transit time;

Controlling the operation of the sensor (110) such that

Evaluating the scene (190) by means of a sensor (110)

Downstream data processing device (150) based on a first result of the first capture of the scene (190) and / or a second result of the second capture of the scene (190).

15. A method according to the preceding claim, further comprising

Detecting an object (195) in the scene;

Comparing the detected object (195) with at least one

Comparison object; and,

if the object (195) agrees with a comparison object within predetermined permissible deviations, identifying the object (195) as an object (195) permitted for a particular action.

16. Use of a sensor system (100) according to one of the preceding claims 1 to 13 for controlling a coverage characteristic of an object (195) to be passed through opening (184) by at least one closure body (186).

Use according to the preceding claim, wherein

the opening (184) is an input or an output, in particular a

Emergency exit in a building.

Use according to one of the two preceding claims, wherein the object (195) is a person or a vehicle.

19. Use of a sensor system (100) according to one of the preceding claims 1 to 13 for detecting and / or controlling traffic flows of objects (295a-295e) which move through a scene (290) of the sensor system (100), wherein the Scene (190) by a spatial

Detection range of the sensor system (100) is determined.