DE202015102019U1 - Camera for taking pictures of a detection area - Google Patents

Abstract

Camera (10), in particular a 3D camera, which has an image sensor (16, 16a-b) for taking pictures of a detection area (12), an inertial sensor (30) for detecting a change in movement of the camera (10) and an evaluation unit (26, 28, 34) for processing the images and the change in movement, characterized in that the evaluation unit (26) is adapted to generate a reliability measure of the images based on a change in movement during the recording.

Description

The invention relates to a camera for taking pictures of a detection area according to the preamble of claim 1.

In contrast to a conventional camera, a 3D camera also captures depth information and thus generates three-dimensional image data with distance or distance values for the individual pixels of the 3D image, which is also referred to as a distance image or depth map. The extra distance dimension can be used in a variety of applications to gain more information about objects in the scene captured by the camera and to solve various industrial sensor tasks.

In automation technology, objects can be detected and classified on the basis of such image information in order to make further automatic processing steps dependent on which objects are preferably recognized, including their position and orientation. Thus, for example, the control of robots or various actuators can be supported on a conveyor belt.

In mobile applications, a vehicle gains information about its surroundings via three-dimensional image data, and in particular a planned route. This can be used for autonomous locomotion or driving assistance.

When a presence of persons is possible or even desired and provided, as in a so-called cooperative workstation, security aspects are often added. A typical safety application is to secure a dangerous machine, such as a press or a robot, where protection is provided when a body part is engaged in a hazardous area around the machine. Depending on the situation, this can be the shutdown of the machine or the move to a safe position. With the additional depth information, three-dimensional protection areas can be defined which are more precisely adaptable to the danger situation than two-dimensional protection fields, and it can also be better assessed whether a person approaches the danger source in a critical manner. Security applications also include securing the movement of a vehicle.

To determine the depth information, various methods are known, such as time-of-flight measurements, interferometry or triangulation. In turn triangulation methods can be used to distinguish between light-section and projection methods and stereoscopy. In the light-section method, the object is moved under the sensor and a 3D point cloud is generated from the strip-based depth information obtained. In the projection method, for example, a striped pattern is projected onto the surface to be scanned, and depth information is derived from changes in the pattern by the objects in the scene. Alternatively, a so-called self-similar, ie locally unique pattern is projected. The projection method with a structured pattern is used, for example, in US Pat US Pat. No. 7,433,024 used. In this case, a multiplicity of reference images of the structured illumination pattern are taught in at different intervals. A later captured image is compared to these reference images to estimate object distances. Stereoscopic methods are based on two-eyed spatial vision and search for associated image elements in two images taken from different perspectives, whose disparity in knowledge of the optical parameters of the stereo camera estimates the distance by triangulation.

Especially in mobile applications, such as on a forklift or truck, it can lead to large vibrations of the vehicle and thus also mounted on a camera. As a result, motion artifacts such as smearing of the images occur. In some cases it is possible to correct or at least recognize this by analyzing and evaluating the image data. But that means a lot of computational effort and is only partially successful.

For simple photo cameras is known to detect the proper motion with a sensor to compensate for a blur, such as by corresponding counter-movement in the optics. This is not intended for a much more complex 3D camera and works only for small intrusive movements.

The DE 10 2009 007 842 A1 discloses a video-based driver assistance system with a stereo camera, which determines from data of a rotation rate or acceleration sensor, an angular and positional change between two shots and corrects the resulting pixel offset. But that does not help against movement artifacts within a shot.

From the U.S. 5,585,875 For example, a vibration correction system is known in a camera. It detects with a sensor proper movements, which are caused by hand shaking, for example, and includes an optical correction system for it. This correction system is blocked if a malfunction is detected in it or in the sensor. That's for an automatically working camera, especially in safety-related application, not a viable option because it does not eliminate severe shocks and their effects.

It is therefore an object of the invention to improve the reliability of a moving camera.

This object is achieved by a camera for capturing images of a detection range according to claim 1. Preferably, it is a 3D camera, which generates three-dimensional images from the signals of one or more image sensors. Depending on the 3D method, the corresponding evaluation is at least partially integrated in the image sensor. The camera includes an inertial sensor, ie a sensor that detects a change in the movement of the camera. It is primarily about sudden changes in movement as when driving over a bump. Although typical acceleration and braking processes or cornering of a vehicle can also be detected via an inertial sensor, they are often not critical in comparison to the exposure or detection times. The invention is based on the basic idea of evaluating the change in movement during a recording. As a result, a measure of reliability can be obtained that provides information about how strongly the image is likely to be affected by the change in movement. The reliability measure may be a number or a number field that quantifies the impairment due to the motion change, but also simple information such as "blurred" or "not blurred". In other words, it is checked whether the image is blurred and to what extent it can still be used in this case.

The invention has the advantage that a reliable image acquisition is made possible even under vibration. If at the moment of vibration no reliable operation is possible, so that is recognized. On the one hand this prevents false positive detections of objects that are not actually present. Otherwise they would possibly be rated as obstacles, inadmissible protective field interventions and the like, thus unnecessarily reducing the availability. On the other hand, it can no longer happen that due to a shock an actually existing obstacle is overlooked. This would be highly critical, especially in a safety-related application, since it can lead to accidents. The vibration is detected according to the invention and can be taken into account in the signal analysis or at least reliably detected when the camera does not provide sufficiently reliable images.

The inertial sensor is preferably an acceleration sensor and / or a rotation rate sensor. Thus, the change of motion in the translation and / or rotation direction is determined. A sudden rotation will regularly be the lesser problem, so preferably the acceleration is measured.

The evaluation unit is preferably designed to store or output the movement change during the recording and / or the reliability measure with the associated images. Thus, the change in movement in a post-processing, in particular an image correction can be used, or there is a subsequent re-evaluation of the impairment of the change in movement. By storing or outputting the reliability measure, the captured images are quasi tagged or tagged, and it is quickly recognized whether and to what extent the quality of the images is considered degraded due to a change in motion.

The evaluation unit is designed to evaluate the change in movement in amplitude and / or direction and optionally to save it with the image. The amplitude is obviously a quantitative measure of the vibration and differentiates between small and large vibrations. The direction may also play a role, however, since, depending on the application, movement changes in certain directions are more critical than in other directions.

The evaluation unit is preferably designed to ignore the change in movement depending on the reliability measure, to use other methods for processing the image data or to discard the images. The image is thus treated differently depending on the loss of quality due to changes in movement. One possibility is to treat the mentioned picture, which is only slightly impaired, as normal, to use other, usually more powerful, or at least disturbance-tolerant, image processing in the event of a greater change in movement and to no longer evaluate an image obtained under excessive change of motion.

The evaluation unit is preferably designed to generate a warning or switch-off signal if the reliability measure for a minimum number of successive recordings is below a minimum value. This is particularly relevant in safety applications, such as the monitoring of a protective field or obstacle detection on a vehicle. For this purpose, it is reliably established that, due to movement changes, sufficient detection by the camera can not be expected in order to transfer a monitored source of danger to a safe state, for example to shut down a machine or to decelerate a vehicle. Preferably, directly successive images are checked, but this does not have to be followed very strictly. For example, a shutdown can take place if, for example, only one or a few recordings of N recordings are usable. In addition, it is conceivable to quantitatively assess the respective reliability measure. For example, several images can be tolerated without warning or shutdown, the reliability level is just below the minimum value, while even one or a few completely blurred images trigger a warning or shutdown.

The camera is preferably designed to record three-dimensional images as a stereo camera and has for this purpose at least two camera modules each with an image sensor in mutually offset perspective and a stereoscopic unit, in which by means of a stereo algorithm 'associated sub-areas are detected in captured by the two camera modules image data and whose distance is calculated on the basis of the disparity. The stereoscopic method has, like all 3D methods, even without change of movement, the additional difficulty of gaining complete range data at all. This increased complexity makes it even more useful, at least to compensate for disturbances due to movement changes by other measures or at least to recognize. The stereo camera may include a lighting unit for generating a patterned illumination pattern to become independent of environmental conditions.

A conceivable alternative to a stereo camera is a camera according to the projection method or with active triangulation, which has a triangulation unit for recording three-dimensional images, which correlates a projected structured illumination pattern with image data of the image sensor for calculating the three-dimensional image. Here, the lighting unit is not optional, but part of the 3D process.

The camera is preferably configured to record three-dimensional images as a time of flight camera and has for this purpose a lighting unit and a light-time unit to determine the light transit time of a light signal emitted by the lighting unit, remitted to objects in the detection area and detected in the image sensor. In the case of time of day cameras, the evaluation is often at least partially integrated into the image sensor. For example, in some phase methods, a three-dimensional image is generated from four images. This takes a comparatively long time with typically a few 10 ms, so that vibration can have a major impact.

The invention will be explained in more detail below with regard to further features and advantages by way of example with reference to embodiments and with reference to the accompanying drawings. The illustrations of the drawing show in:

1 a schematic representation of a stereo camera; and

2 a schematic representation of a light time camera.

1 shows a schematic representation of a 3D camera 10 for taking three-dimensional images of a detection area 12 in one embodiment as a stereo camera. The evaluation of the vibration or change of movement of the 3D camera described below using exemplary embodiments 10 is in principle also interesting for simple, ie two-dimensional cameras, in particular for their use in safety technology. However, the invention preferably aims at 3D cameras, since movement changes are particularly critical here and it is often difficult enough to gain seamless images even without a movement problem. Therefore, the invention will also be explained with reference to 3D cameras, wherein by concrete description of a stereo camera and later a light time camera no limitation to a particular type of 3D camera should be implied. Other 3D experiences are also conceivable, in particular an initially mentioned projection method with only one camera and correlation with its illumination pattern (active triangulation).

The 3D camera 10 For example, it can be mounted in a mobile application on a vehicle and used by a driver assistance system or even for autonomous driving (AGV). However, vibrations, vibrations or other unfavorable movement changes also occur in actually static mounting, for example on stationary machines which are exposed to vibrations or themselves vibrate at least in a few steps.

In the 3D camera 10 are two camera modules 14a -B mounted at a known fixed distance from each other and each take pictures of the detection area 12 on. In every camera is an image sensor 16a B, usually a matrix-shaped receiving chip which receives a rectangular pixel image, for example a CCD or a CMOS sensor. The image sensors 16a -B is in each case associated with a lens with an imaging optics, which as a lens 18a -B is shown and may be implemented in practice as any known imaging optics.

In the middle between the two camera modules 14a -B is a lighting unit 20 with a light source 22 shown, this spatial Arrangement is to be understood only as an example. The lighting unit 20 can be arranged differently and also externally, differing from the illustration. For passive stereoscopy could be on the lighting unit 20 even be waived. The lighting unit 20 generated by means of a pattern generating element 24 in the detection area 12 a structured lighting pattern. The illumination pattern should preferably be at least locally unique or self-similar in the sense that structures of the illumination pattern do not lead to apparent correlations, or clearly identify a lighting area.

With the two image sensors 16a -B and the lighting unit 20 is a combined control and evaluation unit 26 in the following, only as an evaluation unit 26 referred to as. By means of the evaluation unit 26 The structured illumination pattern is generated and receives image data of the image sensors 16a b. From these image data, a stereoscopic unit calculates 28 the evaluation unit 26 with a stereoscopic algorithm known per se three-dimensional image data (distance image, depth map) of the detection area 12 , The evaluation unit 26 and the stereoscopic unit 28 Deviating from the representation, they can be implemented in any way distributed on one or more digital components and can also have analog circuit components.

In the 3D camera 10 is still an inertial sensor 30 integrated, detects the movement changes and their transmission with the evaluation unit 26 connected is. The inertial sensor 30 determines the rate of rotation and / or acceleration of the 3D camera 10 and is designed for this purpose in any known manner, for example as a piezoceramic or because of its particular compactness as a MEMS (Micro Electro Mechanical System). The evaluation unit 26 evaluated on the basis of the movement changes during a recording in a reliability measure, the extent to which an impairment of the generated three-dimensional images is caused. These can be both individual images and a series of consecutive images.

For example, three-dimensional images are discarded, marked as invalid or at least potentially defective when the inertial sensor 30 while recording has registered a shock whose amplitude is above a specified limit. With the help of the inertial sensor 30 On the one hand, it can be prevented that a false positive object detection occurs during a vibration, ie motion artifacts are used to detect an object that does not exist. This would lead to unnecessary reactions and thus a restriction on the availability of a system that is part of the 3D camera 10 is.

Especially in the case of safety applications in which the 3D camera 10 For example, monitors a protective field or the environment of a vehicle, is due to the evaluation using the inertial sensor 30 the ability to detect problematic situations and, if necessary, generate a warning or shutdown signal. For example, a machine or vehicle is precautionarily stopped if more than N consecutive images are invalid or potentially defective due to excessive vibration. This will prevent a potentially critical object from being overlooked and cause an accident.

The recorded during a vibration three-dimensional images do not necessarily have to be completely discarded. Alternatively, these images may be generated using specially adapted algorithms using the additional knowledge from the inertial sensor 30 be analyzed and evaluated that a vibration during image acquisition has occurred, possibly also taking into account from the measurement data of the inertial sensor 30 known amplitude and direction.

Depending on the application, not only the amplitude, but also the vectorial direction of the deflection of a movement change or vibration can be relevant. For example, it may be that a deflection in the direction of the sensor axis (measuring direction) is more critical than perpendicular to the measuring direction or vice versa. Therefore, the reliability measure, in particular the identification or marking of invalid or potentially defective three-dimensional images, in addition to the amplitude of the vibration, can also be dependent on the direction of the deflection, wherein at least certain inertial sensors 30 are also able to grasp this direction.

In a further embodiment, a plurality of threshold values are defined for the optionally direction-dependent evaluated amplitude of the change in movement or for the reliability measure in order to react selectively depending on the degree of impairment by the vibration. For example, a slight vibration is tolerated without special measures, a certain amount of vibration leads to the production or processing of a three-dimensional image with special algorithms, and only in the case of severe vibration is a three-dimensional image completely discarded.

About an exit 32 can the 3d camera 10 output three-dimensional images or other measurement results, such as the inertial sensor 30 certain movement change, raw image data of a camera module 14a -B, but also evaluation results such as object data or the identification of certain objects. Especially in safety applications, the detection of an impermissible intervention in protective fields in the detection area 12 have been defined, or the already explained succession of N images in case of excessive vibration lead to the output of a safety-related shutdown signal. This is the exit 32 then preferably as a safety output (OSSD, output signal switching device) running and the 3D camera 10 Total constructed in accordance with relevant safety standards fail-safe.

2 shows in a further block diagram an alternative embodiment of the 3D camera 10 as a light runtime camera. Here and in the following, the same reference numerals designate the same or corresponding features. At the relatively coarse level of presentation, the time of day camera differs from a stereo camera mainly in the absence of a second camera module. Such a construction is also that of a 3D camera, which estimates distances in a projection method from distance-dependent changes of the illumination pattern. Another difference is that the evaluation is different. This is in place of the stereoscopic unit 28 a light time unit 34 in the evaluation unit 26 provided, which measures the light transit time between sending and receiving a light signal. The light runtime unit 34 can also directly at least partially into the image sensor 16 be integrated, for example in a PMD chip (photon mixing detection). Accordingly, an adapted unit for evaluating the illumination pattern is provided in a 3D camera for a projection method.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• US 7433024 [0006]
• DE 102009007842 A1 [0009]
• US 5585875 [0010]

Claims (8)

Camera ( 10 ), in particular 3D camera, which has an image sensor ( 16 . 16a -B) for taking pictures of a coverage area ( 12 ), an inertial sensor ( 30 ) for detecting a movement change of the camera ( 10 ) and an evaluation unit ( 26 . 28 . 34 ) for processing the images and the movement change, characterized in that the evaluation unit ( 26 ) is adapted to generate a reliability measure of the images based on a change of motion during the recording. Camera ( 10 ) according to claim 1, wherein the inertial sensor ( 30 ) is an acceleration sensor and / or a rotation rate sensor. Camera ( 10 ) according to claim 1 or 2, wherein the evaluation unit ( 26 ) is adapted to store or output the change of motion during the recording and / or the reliability measure with the associated images. Camera ( 10 ) according to one of the preceding claims, wherein the evaluation unit ( 26 ) is designed to evaluate the change in movement in amplitude and / or direction. Camera ( 10 ) according to one of the preceding claims, wherein the evaluation unit ( 26 ) is adapted to ignore the change of movement depending on the reliability measure, to use other methods for processing the image data or to discard the images. Camera ( 10 ) according to one of the preceding claims, wherein the evaluation unit ( 26 ) is adapted to generate a warning or shutdown signal when the reliability measure for a minimum number of consecutive shots is below a minimum value. Camera ( 10 ) according to one of the preceding claims, which is designed to receive three-dimensional images as a stereo camera and to at least two camera modules ( 14a -B) each with an image sensor ( 16a B) in a staggered perspective and a stereoscopic unit ( 28 ), in which by means of a stereo algorithm 'associated sub-areas in of the two camera modules ( 14a -B) recorded image data and whose distance is calculated on the basis of the disparity. Camera ( 10 ) according to one of claims 1 to 6, which is designed to receive three-dimensional images as a light runtime camera and to a lighting unit ( 20 ) and a light time unit ( 34 ) in order to determine the light transit time of a light signal emitted by the illumination unit ( 20 ) to objects in the detection area ( 12 ) and in the image sensor ( 16 ) is detected.

Images