DE102008058798A1 - Stereocamera device for use in monitoring device for e.g. wind-power plant, has radiation beam divided into partial radiation beams deflected on cameras over optical deflection element attached to cameras - Google Patents

Abstract

The device (1) has infrared cameras (3a, 3b) arranged at a defined distance to each other and provided with a calibration device (19) for sequential automatic calibration of the cameras, where the calibration device exhibits a radiation source (21) and a reference beam input (20). A collimated radiation beam (22) output from the radiation source is divided into two partial radiation beams (22a, 22b). The partial beams are deflected on the cameras over an optical deflection element e.g. penta-prism (23a), attached to the cameras. An optical separating element is designed as a 90-degree-prism. Independent claims are also included for the following: (1) a monitoring device comprising a stereocamera device (2) a method for sequential automatic calibration of a stereocamera device (3) a computer program comprising a set of instructions to perform a method for the sequential automatic calibration of the stereocamera device (4) a computer program product comprising the set of instructions to perform a method for the sequential automatic calibration of the stereocamera device.

Description

The The invention relates to a stereo camera device having at least two adjusted, arranged at a defined distance from each other and aligned Thermal imaging cameras, which with a calibration device to their continuous automatic calibration is provided. Of Furthermore, the invention relates to a method for continuous automatic calibration of such a stereo camera device. The invention also relates to a computer program and a computer program product with program code means to perform such a method. Moreover, the invention also relates to a monitoring device for wind turbines, buildings with transparent Areas, Runways and / or Air Corridors of Airports with a stereoscopic capture approaching or existing birds or bird swarms.

at Landing and landing of aircraft often leads to collisions with birds or bird swarms. Especially when Crossing of air routes of birds of regional and national Bird migration, which often takes on landscape structures like waters, valleys or shorelines orient, this danger is greatly increased. In such Collisions can damage u. a. on the engines come from aircraft.

In the older not pre-published DE 10 2008 018 880.8 For this purpose, a monitoring device for wind turbines, buildings with transparent areas, runways and / or flight corridors of airports with a stereoscopic detection of approaching or existing birds or bird swarms for carrying out a monitoring Verrender is proposed, where parameters such as altitude, flight direction, airspeed, Type and size of birds or flocks of birds can be determined. In the field of wind turbines, buildings with transparent areas, runways and / or flight corridors, at least one stereo camera device is provided, which has at least two mutually defined and adjusted spacing synchronously during recording cameras, in particular thermal imaging cameras, whose recording times at least are approximately identical and their respective fields of view have an overlapping area.

Become Cameras, in particular thermal imaging cameras or thermal imaging apparatus to measurement tasks, such as the aforementioned When using stereoscopy, these should have constant characteristics, in particular possess mechanical stability. As in the stereo evaluation The images can not be achieved by simple measures can be kept in the construction of the stereo camera device must be determined by suitable methods how the system properties change, in order to be able to correct these downstream. The above-mentioned stereo camera device for determination the presence or the speed of flight of the birds is based on a precise determination of that in a relatively short time Time interval traveled by the animals. Is to exact location at two points necessary, which are very sensitive against a relative error between the two lines of sight the cameras is. The aforementioned stereo camera devices should be calibrated as much as possible over that long periods of high-precision stereoscopic measurements can be. For a construction of such measuring systems is a detailed adjustment strategy for each assembly step necessary, which ensures that in the last step at the site with the then limited resources available comprehensive calibration of the system can be achieved.

deviations performed by the construction of the stereo camera device Calibration can be over the desired operating time Often not completely avoided. So it may be z. B. by a bending of the basic mechanical or basic structure of the stereo system or due to thermal changes the structural elements and also the individual thermal imaging cameras to changes in the line of sight of the thermal imaging cameras come. For continuous recognition and correction of these deviations Measures should be conceived that coincide with run the actual measurements in the context of the stereo image evaluation can, without disturbing them and of course even no additional errors in the stereo camera device may bring in.

Strength Temperature fluctuations and / or mechanical stresses z. B. Vibrations can cause changes in camera parameters the stereo camera device lead, in particular of changes in the position and position as well as the orientation of the respective camera to lead. Therefore, it may be necessary the stereo camera device recalibrate at short intervals or should the used stereo camera device to be able to provide a continuous To carry out self-calibration.

The DE 10 2007 050 558 A1 relates to a method for the continuous self-calibration of an image receiving device, in particular a stereo camera system in which by means of a stereo method, a pair of images is detected, based on which a rectified image pair is determined by a pixel-by-pixel correspondence analysis along epipolar lines.

A system and a method for generating spatial image representations, in particular of spatial thermal images, is disclosed in US Pat EP 1 484 628 A1 specified.

For further prior art is on the JP 01251990 A directed.

outgoing thereof is the object of the present invention, a Stereo camera device and a method for continuous automatic Calibration of a stereo camera device or a computer program and to create a computer program product which is a simple one and accurate tracking calibration of the stereo camera device allow changes to those determined at start-up calibration System parameters are determined and corrected if necessary, in particular without additional disturbances or errors to produce by the Nachführkalibrierung.

These The object is achieved by claim 1 solved. The object is also solved by claim 10. Regarding the method for continuous automatic calibration a stereo device, the object is achieved by claim 13. Regarding the computer program or the computer program product the object is achieved by claim 17 or claim 18.

According to the invention a stereo camera device with at least two adjusted to each other in defined spaced and aligned thermal imaging cameras proposed, which with a calibration device to the continuous automatic calibration is provided, the calibration device a radiation source and a reference beam path, by means of which a reference image from the radiation source to the respective Imaged thermal imaging camera, wherein one of the radiation source outgoing collimated beam in at least two partial beams is divided, wherein the at least two partial beams in each case via at least one of the respective thermal imaging camera associated optical deflecting be directed to this and wherein at least one of the optical deflection elements as pentaprism is trained.

Of Another is a method according to the invention for continuous automatic calibration of the invention Stereocameraeinrichtung proposed, wherein the reference image the at least two thermal imaging cameras in predetermined, in particular special localized at regular intervals becomes, according to which the relative position and / or the relative orientation of the reference image to the reference images of the further thermal imaging cameras is determined, and what happens when a change is detected a corresponding correction of the parameters for the stereo evaluation determined and made.

By these measures will allow stereo camera facilities, especially to calibrate stereo thermal imaging cameras so that over very long periods of time highly accurate stereoscopic Measurements can be made. There are system accuracies achieved by passive measures not feasible are. The proposed method is based on a start calibration and on an automatic tracking calibration, which the changes against those present at the start calibration System parameters for stereo evaluation or image processing determines and, if necessary, makes appropriate corrections.

In order to keep the line of sight of the two thermal imaging cameras, which are mounted on a base structure with possibly higher base length, aligned for a long time, a reference to the axis of a reference beam path or reference collimator is produced. For this purpose, the beam emanating from the reference beam path is divided and the resulting partial beams are deflected with suitable deflection elements such as mirrors or prisms into the respective entrance pupil of a thermal imaging camera. Through the use of at least one pentaprism as optical deflection of a thermal imaging camera ensures that the reference beams generated by the deflection are always aligned perpendicular to the axis or the reference beam regardless of the exact angular position of the pentaprism, as it is a peculiarity of the pentaprism that the Exit angle of the light beam is always 90 degrees to the entrance angle. Thus, the pentaprism is invariant to interference (eg tilting of the pentaprism). Any changes in angle of the reference beam path or the radiation source or the collimator erzeu same-like shelves in both thermal imaging cameras and can thus be detected and distinguished from the relevant errors or deviations. The reference beams for both thermal imaging cameras in each case remain perpendicular or parallel to the current axis of the reference beam path, so that the Sichtli The two thermal imaging cameras can be controlled electronically or via the stereo evaluation in the image processing.

According to the invention be further provided that at least one optical sub-element for dividing the beam into the partial beams is provided.

Thereby can different, in particular symmetrical reference beam paths be used.

Advantageous it is when the optical divider element is formed as a 90 degree prism is. Tilting the 90 degree prism again has the same direction Shift the reference images or target images result, which can also be compensated, so no further errors or deviations are introduced.

Very It is advantageous if the reference beam path is symmetrical is, since then the geometric conditions for the at least two thermal imaging cameras are the same. For this can be a mirrored 90-degree prism for pupil division in the Reference beam path introduced. By the additional 90 degree prism are advantageously introduced no new errors that are not are recognizable.

to Collimation of the beam emanating from the radiation source the reference beam path can be a collimator or reference collimator exhibit.

Of the Reference beam should of course not even uncontrollable errors or deviations in the stereo camera device contribute. Possible sources of error are primarily unwanted changes in the positioning of the optical Elements of the reference beam path, d. H. of the Pentaprism, the 90 degree prism and collimator. Purely translational displacements are unproblematic. Because the added optical Components have only flat surfaces and in the collimated optical reference beam path are arranged, occurs only a pupil shift on and no change in the image position.

Advantageous if the radiation source is a thermoelectric element, in particular has a Peltier element. Such a thermoelectric Element or cooling element can be under the action of Electricity for both heating and cooling be used. Accordingly, the target or reference image can be displayed (Reference image shows a temperature difference to the ambient temperature on) or hidden (reference image has ambient temperature). The reference image can be, for example, as a crosshair or crosshair structure be formed and from a support element (for example punched out of sheet metal) or etched into it. Such a carrier element can then with the thermoelectric Element connected or at least part of the thermoelectric Be element.

According to the invention be further provided that the exit surface of at least a pentaprism at least approximately in the range of Entrance pupil of the respective associated thermal imaging camera is arranged.

The Division of the beam path in observation and reference beam path should be in the area of the entrance pupil of the thermal imager so that their image quality is not impaired becomes.

Advantageous it is when the reference beam path has a tube. Thereby unwanted emissions are minimized. additionally the tube can have matt black inner surfaces and have additional aperture rings.

According to the invention be further provided that in the propagation direction of the observation light of respective thermal imaging camera in front of the optical deflecting element the thermal imager to calibrate the detector of Thermal imager one in the observation beam path of Thermal imaging camera swiveling defocusing lens is provided.

Infrared detector arrays typically have a strong inhomogeneity of their single detector elements in terms of dark current and gain. In order to still be able to achieve an acceptable image quality, a calibration of the individual detector elements should be carried out (so-called Non-Uniformity Correction, NUC). However, for a sufficiently good homogenization of the image background, this is usually not sufficient, since there are other sources of radiation in the infrared range in addition to the desired scene radiation which contribute undesirably to the observation image information. Due to the expected strong and also inhomogeneous radiation from the reference beam path into the thermal imaging camera, the calibration of the individual detector elements should already be able to detect as many of these radiation components as possible. Since the internal temperature of the stereo camera device and the scene temperature (eg, cold sky) may be very different from each other, the calibration should be made as close to the scene temperature as possible. It is therefore very advantageous during the calibration to swivel in a defocusing lens between the at least one pentaprism and a viewing window of the stereo camera device in order to detect the unwanted radiation components during the calibration. In order to prevent the reference structure or the reference image from being removed in advance by the calibration, the radiation source should be switched off during the calibration or regulated correspondingly to the internal temperature of the stereo camera device.

In Claim 10 is an alternative embodiment of the invention Stereo camera setup with two adjusted, to each other in defined Spaced and aligned thermal imaging cameras indicated, which with a calibration device to the continuous is provided with automatic calibration, being used as a calibration device the two thermal imaging cameras each in the area of their entrance pupils have an optical deflecting element, which of the first Thermal camera or emitted by the detector own Heat radiation as a reference image in the observation beam path the second thermal imaging camera deflects.

Around aligned the line of sight of the two thermal imaging cameras can also be passive, d. H. advantageously without additional Heat or radiation source a direct reference between the two axes of the thermal imaging cameras are made. For this purpose, in each case a small part of the pupil of a thermal imaging camera with a deflecting element, for. A prism or mirror, be redirected into the pupil of the other thermal imaging camera. Because the temperature of the detector surfaces relative to the scene is very low, each thermal imager captures a cold Image section of the detector surface of its counterpart embedded in the scene. Used for the optical deflecting element used a pentaprism, then it is ensured that through double deflection generated reference beams, independent from the exact angular position of each pentaprism, always are parallel to the output beam, and thus by the respective other thermal imaging camera formed reference beam path none generated another error.

In Claim 12 is a monitoring device for Wind turbines, buildings with transparent areas, Runways and / or flight corridors of airports with a stereoscopic capture approaching or existing birds or swarms of birds. This can be advantageously because of the simple and accurate Nachführkalibrierung their at least one inventive Stereo camera setup over very long periods of time working independently.

by virtue of a recognized relative displacement of the reference image to the Reference images of the other thermal imaging cameras can be a Displacement of a detector of at least one of the thermal imaging cameras or a tilting or bending of at least one of the thermal imaging cameras parallel and / or perpendicular to the reference plane or to a base structure or baseline of the stereo camera device are detected.

by virtue of a relative rotation of the reference image to the reference images the further thermal imaging cameras can at least a twist one of the thermal imaging cameras or a detector at least one of the thermal imaging cameras detected around its optical axis become.

Advantageous it is when a start calibration of the stereo camera device is carried out.

To Construction of the thermal imaging cameras of the stereo camera device There may be residual errors which are not caused by adjustment can be eliminated. Such errors can then only by calibration with the help of suitable image processing methods be reduced. For this purpose, suitable targets or targets in used the landscape at a known location. The distance should be ideally correspond to the later working distance, However, this is often not guaranteed by spatial conditions is. The calibration process is then performed by comparisons with the known target positions in the scene. The result of the calibration is a constant mapping of the field angles to the pixels of the thermal imagers. Since from this time changes in the arrangement may result in parallel immediately with the invention Method for continuous automatic calibration of the stereo camera device to be started. This can then relative to the later Measurements show changes in the arrangement.

The inventive method for continuous automatic calibration of the invention Stereo stereo device is preferably realized as a computer program on an image processing device of the stereo camera device according to the invention. For this purpose, the computer program is stored in a memory element of the image processing device. By processing on a microprocessor of the image processing device, the method is carried out. The computer program can be stored on a computer-readable data medium (floppy disk, CD, DVD, hard disk, USB memory stick or the like) or an Internet server as a computer program product and can be transferred from there into the memory element of the image processing device. Such a computer program or computer program product with program code means is specified in claim 17 or claim 18.

advantageous Refinements and developments of the invention will become apparent the dependent claims. Below are the drawings in principle embodiments of the Invention specified.

It demonstrate:

1 a schematic representation of a stereo camera device according to the invention in a monitoring device;

2 a simplified representation of an arrangement of the stereo camera device according to the invention 1 in the area of a flight corridor of an aircraft;

3 a simplified representation of a start calibration of a stereo camera device;

4 a schematic representation of a first embodiment of the stereo camera device according to the invention;

5 a simplified schematic representation of the beam path of the stereo camera device according to the invention according to the first embodiment with an angle change of a collimator;

6 a schematic representation of a second embodiment of the stereo camera device according to the invention;

7 a simplified schematic representation of the beam path of the stereo camera device according to the invention according to the second embodiment with a tilting of the collimator;

8th a simplified schematic representation of a beam path of the stereo camera device according to the invention according to the second embodiment with a tilting of a divider element;

9 a simplified schematic representation of a beam path of the stereo camera device according to the invention according to the second embodiment with an indication of image layers;

10 a simplified schematic representation of a field of view of a reference beam path of the stereo camera device according to the invention;

11 a simplified representation of an entrance pupil of a thermal imaging camera of the stereo device according to the invention with a pentaprism;

12 a simplified representation of the pitch of the entrance pupil of the thermal imager;

13 a simplified representation of the distribution of the field of view of a thermal imaging camera in the reference beam path;

14 a schematic representation of the functional principle of a pentaprism;

15 a simplified representation of error propagation of manufacturing errors in a pentaprism;

16 a simplified representation of the beam offset due to positioning errors of a pentaprism;

17 a schematic representation of a third embodiment of the stereo camera device according to the invention;

18 a schematic representation of a portion of the stereo camera device according to the invention according to the second embodiment;

19 a schematic representation of a portion of the stereo camera device according to the invention according to the second embodiment with pivoted Defokussierlinse; and

20 a schematic representation of the stereo camera device according to the invention according to the second embodiment with pivoted Defokussierlinse.

The The invention will be described below as part of a monitoring device for wind turbines, buildings with transparent Areas, Runways and / or Air Corridors of Airports described. Of course, the invention is not limited to this application.

In 1 is a stereo camera device according to the invention 1 a monitoring device 2 for runways and / or flight corridors 11 (please refer 2 ) of airports with stereoscopic detection of approaching birds 6 or bird swarms 6 ' , where parameters such as flight altitude, flight direction, airspeed and type / size of birds 6 or the bird swarms 6 ' can be determined. One or more such stereo camera devices 1 are in the area of the runways (not shown) and / or the flight corridors 11 arranged and have at least two mutually arranged in a defined and adapted distance during recording synchronously running thermal imaging cameras 3a . 3b on. The recording times of the thermal imaging cameras 3a . 3b are at least approximately identical and their respective fields of view 4a . 4b have an overlapping area 5 on. In the overlapping area 5 becomes an object as a bird 6 detected. The two thermal imaging cameras 3a . 3b are adjusted to each other and calibrated. For the thermal imaging cameras 3a . 3b Both thermal imaging areas such as LWIR, MWIR, VLWIR, FIR, and SWIR, NIR are considered.

The stereo camera device 1 has an image processing device 7 on which to process the with the two thermal imaging cameras 3a . 3b recorded image data is provided.

The stereo camera device 1 also has a radio station 8th as an interface, in particular network interface, for communication with further stereo camera devices 1 or with higher-level systems, in particular air traffic control systems 9 , on (in 1 through the double arrow 8th' indicated). The stereo camera device 1 works autonomously. However, by means of the networking or the radio transmission via the radio station 8th other stations or stereo camera devices 1 get connected. The information as well as the recordings are therefore available outside the individual station. Mainly these data are transmitted to the air traffic control.

On the image processing device 7 the stereo camera device 1 Among other things, there is a monitoring procedure for runways and / or flight corridors 11 from airports with which approaching birds 6 or bird swarms 6 ' stereoscopically by means of the monitoring device 2 or the stereo camera device 1 parameters such as flight altitude, flight direction, airspeed and type / size of the birds 6 or the bird swarms 6 ' or whose swarm density is determined. The parameters are determined by means of a stereo evaluation. In this case, by the at least two angles of view on the at least two thermal imaging cameras 3a . 3b the stereo camera device 1 recorded area 5 absolute space points of the birds to be detected 6 or bird swarms 6 ' certainly. The speed of flight of the birds 6 or the bird swarms 6 ' is determined by a consideration over a corresponding period of time. Also birds 6 or bird swarms 6 ' at a greater distance can be detected, with a correspondingly longer focal length for the two thermal imaging cameras 3a . 3b is used. In addition, flying objects such as model airplanes, stunt kites or the like may also be used by the stereo camera device 1 be detected (not shown).

Based the parameter is evaluated and, if necessary a corresponding warning message is issued.

How out 2 can be seen, monitors a stereo camera device 1 with a stereo field of view or overlapping Be rich 5 a known flight route 10 of birds 6 or bird swarms 6 ' , The stereo camera device 1 is arranged so that a flight corridor 11 or an intersection area 12 of flight corridor 11 with the known flight route 10 of the birds 6 or bird swarms 6 ' is monitored. In the flight corridor 11 is an airplane 13 exemplified. An arrival time of the captured birds 6 or the captured bird swarms 6 ' at the crossing area 12 with the flight corridor 11 of the plane 13 is also determined.

Upon detection of birds 6 or bird swarms 6 ' If appropriate, a warning message is issued to initiate countermeasures or abatement or avoidance measures to the superordinate system, in particular the air traffic control system 9 , or to take-off or landing aircraft 13 output.

The stereo camera device 1 should have constant properties, in particular mechanical stability. Since the achievable in the stereo evaluation of the image processing accuracies not by simple measures in the mechanical arrangement of the stereo camera device 1 must be kept constant, must be determined by means of suitable methods, as the system properties of the stereo camera device 1 change over time, in order to be able to correct them if necessary.

After building the thermal imaging cameras 3a . 3b the stereo camera device 1 There may be residual errors that can not be eliminated by mechanical adjustment. Such errors can only be achieved by means of on-site calibration by means of suitable image processing methods on the image processing device 7 to reduce. In 3 is the basic structure as part of a stereo camera device according to the invention 1 shown. The thermal imaging cameras 3a . 3b are on a basic structure 14 in a housing 15 arranged and aligned with each other at a defined distance. The housing 15 has lookout window 16 for the observation beam paths or lines of sight 17a . 17b for the visual fields 4a . 4b the thermal imaging cameras 3a . 3b on. When the thermal imaging cameras 3a . 3b on the basic structure 14 in preferably air-conditioned housing 15 are mounted, all previous residual errors add to the assembly errors and the through the view window 16 caused angular errors. For start calibration, suitable targets or goals are set 18 used in the landscape at a known position. As goals 18 For example, posts, panels or the like are suitable. The distance should ideally correspond to the later working distance. The start calibration process is then done by locating the targets 18 through the stereo camera device 1 with a measurement of the associated angular coordinates and a comparison with the actual known positions of the targets 18 in the countryside. The result is a constant calibration of the thermal imagers 3a . 3b ie the individual pixels are assigned field angles. Furthermore, a plausibility test is carried out with the known thermal imaging camera data. Since from this point on changes in the arrangement may result, for example, due to temperature changes caused bending of the basic structure 14 , the first measurement of the follow-up calibration should be carried out afterwards. Thus, it is possible the stereo camera device 1 calibrate automatically so that highly accurate stereoscopic measurements are possible over very long periods of time. As a result, system accuracies can be achieved that are not achievable through passive measures. An automatic follow-up calibration or self-calibration is performed, which detects changes compared to the system parameters available during the start calibration.

How out 4 can be seen, according to the invention a stereo camera device 1 with two adjusted to each other in a defined spaced and aligned thermal imaging cameras 3a . 3b proposed, which with a calibration device 19 is provided to the continuous automatic calibration, wherein the calibration device 19 a reference beam path 20 with a radiation source designed as a thermoelectric element or Peltier element with a reference structure 21 by means of which a reference image from the radiation source 21 to the respective thermal imaging camera 3a . 3b being imaged, one from the radiation source 21 outgoing collimated beam 22 in two partial beams 22a . 22b is divided, the two partial beams 22a . 22b each via one of the respective thermal imaging camera 3a . 3b associated optical deflecting element 23a . 23b be directed to this and wherein the optical deflecting elements as pentaprisms 23a . 23b are formed. The reference beam path 20 has a collimator for collimating the beams 24 on.

How further from the 4 can be seen, the thermal imaging cameras 3a . 3b detectors 25a . 25b with not shown individual detector elements. The pentaprisms 23a . 23b have entry surfaces 26a . 26b and exit surfaces 27a . 27b on. The exit surfaces 27a . 27b the pentaprisms 23a . 23b are at least approximately in the area of the entrance pupil 28a . 28b the respective associated thermal imaging cameras 3a . 3b arranged.

In 5 are the observation beam paths 17a . 17b and the reference beam path 20 the stereo camera device according to the invention 1 strongly indicated. Like through the angles 29 shown results in any change in the angle of the collimator 24 in the same way shelves in both Wärldka meras 3a . 3b or on their detectors 25a . 25b , The reference beams or partial beams 22a . 22b for both thermal imaging cameras 3a . 3b In any case, they remain perpendicular to the current axis of the collimator 24 so the line of sight 17a . 17b the two thermal imaging cameras 3a . 3b if necessary, controlled and tracked electronically.

In 6 is another embodiment of a stereo camera device according to the invention 1' with a symmetrical reference beam path 20 ' shown. This is an optical divider element 30 to divide the beam 22 in the partial beams 22a . 22b intended. The optical subelement is as a 90 degree prism 30 educated. It is before geous, the reference beam path 20 symmetrical as possible, so that the geometric conditions for both thermal imaging cameras 3a . 3b as equal as possible. This is in a simple way with a corresponding mirrored 90 degree prism 30 for pupil division in the reference beam path 20 possible. The following error analysis shows that by the additional 90 degree prism 30 no new mistakes can occur, which are not recognizable.

A reference beam path 20 . 20 ' Of course, you should not create uncontrollable errors yourself. Possible sources of error in the stereo camera device according to the invention 1 . 1' are primarily unwanted changes in the positioning of the added optical elements (pentaprisms 23a . 23b , 90 degree prism 30 and collimator 24 ). Purely translatory shifts are rather unproblematic. Because the added optical components 23a . 23b . 24 . 29 only have plane surfaces and in the collimated beam path 22 are arranged, only a pupil shift and no change in the image position can occur.

For the accuracy of the position velocity calculations, all angular changes in the reference plane present in the base structure 14 and the line of sight 17a . 17b the thermal imaging cameras 3a . 3b is stretched, especially critical. A twist of pentaprisms 23a . 23b causes due to the special functional principle of these components no angle changes and thus generates no additional errors. A tilt of the collimator 24 in turn causes a same direction shift the reference images in both thermal imaging cameras 3a . 3b (please refer 7 or there angle 29 ), which can be detected and compensated and thus also generates no error.

From a tilt angle α _{1 of} the collimator 24 results in the resulting erroneous deviation δ ₁ of each reference image on the respective detector 25a . 25b with the help of the focal length F _{W of} the thermal imager 3a . 3b to: Storage error δ 1 = F W · Tan α 1 [Mm].

A tilt of the 90 degree prism 30 has like the collimator 24 a same-minded shift of the reference images in both thermal imaging cameras 3a . 3b As a result, which can also be compensated, so that no further errors arise (see 8th or there angle 29 ).

From the tilt angle α _{2 of} the 90-degree prism 30 results in the resulting erroneous deviation δ _{2 of} the individual reference images on the respective detector 25a . 25b with the help of the focal length F _{W of} the thermal imaging cameras 3a . 3b to: Storage error δ 2 = F W · Tan 2α 2 [Mm].

A rotation of the collimator 24 around its optical axis generates a same direction rotation of the reference images in both thermal imaging cameras 3a . 3b because the image layers in both thermal imaging cameras 3a . 3b , as in 9 indicated, are the same. This deviation can also be compensated so that no new errors occur. The respective image layers are in 9 by crossed arrows 31 indicated.

The effects of faulty tilting of the optical components 23a . 23b . 24 . 29 perpendicular to the reference plane are less serious, since they do not go directly into the accuracy of the stereo evaluation. A tilt of the collimator 24 and / or the 90 degree prism 30 in turn works in the same direction on both thermal imaging cameras 3a . 3b and can therefore be recognized and compensated. A tilt of a single pentaprism 23a . 23b leads to a shift of the reference image on the associated thermal imaging camera 3a . 3b , which can not be distinguished from the errors of the rest of the measuring system and therefore can lead to an additional error. With a correspondingly stable construction, however, it can be assumed that such errors are small or rare and thus tolerable.

From the tilt angle β _{3 of} the pentaprism 23a . 23b results in the resulting erroneous deviation δ _{3 of} the associated reference image on the detector 25a . 25b or the thermal imaging camera 3a . 3b with the help of the focal length F _{W of} the thermal imaging cameras 3a . 3b to: Storage error δ 3 = F W · Tan β 3 [Mm].

The entrance pupil of the collimator 24 lies in his lens (unspecified) while the entrance pupil 28a . 28b the lenses of the thermal imaging cameras 3a . 3b lie in a relatively short distance in front of the respective first lens vertex. Otherwise no lens optics in the reference beam path 20 . 20 ' is provided, results in the maximum transferable reference field FOV _{REF of} the reference beam path 20 . 20 ' from the size of the pentaprism 23a . 23b and the base length b (b = 1.5 m in the present embodiment). For larger field angles, the beams are cut up to 100%. From the in 10 shown geometry that only a field point of a thermal imaging camera 3a . 3b can still be imaged with finite brightness, if the associated beam on the entrance surface 26a . 26b of the associated pentaprism 23a . 23b meets. From the angle condition for the just no longer impinging edge beam (comparison on complete trimming of the pixel) results in a contact angle α of the field of view for the reference beam path 20 ' to: Field of view edge angle α 3 = arc tan p / b, where b is the base length and p is the height of the entrance surface 26a . 26b of Pentaprism 23a . 23b is. The full field of view FOV _REF is twice as large, ie Reference field FOV REF = 2α 3 ,

For b = 3 m and p = 10 mm, α ₃ = 0.2 degrees. The full reference field FOV _REF is thus about 0.4 degrees, which corresponds to about 24 pixels. The brightness is maximum in the middle and falls to the edge to 0%. On the detector 25a . 25b the thermal imager 3a . 3b With 640 × 512 pixels, the reference image takes a maximum of 24 × 24 pixels.

The division of the beam path in observation beam path 17a . 17b and reference beam path 20 . 20 ' should be in the entrance pupil 28a . 28b the thermal imager 3a . 3b done so that the picture quality is not affected. A realistic division ratio is off 11 seen. In the present embodiment, the entrance pupil 28a . 28b the thermal imager 3a . 3b a diameter of 27.5 mm.

The pentaprism 23a . 23b has an area of 10 mm × 10 mm, with shading 32 the pupil 28a . 28b the thermal imager 3a . 3b of 11 mm × 11 mm is present. The pupil of the reference beam path 20 . 20 ' is therefore 10 mm × 10 mm.

Due to the spatial extent of the pentaprism 23a . 23b creates additional loss of surface due to shading, as in 12 shown. The resulting field of view 33 is in 12 shown again. Through the inserted pentaprism 23a . 23b and the shading 32 becomes the area of the entrance pupil 28a . 28b of the observation beam path 17a . 17b the thermal imager 3a . 3b reduced, whereby the F-number deteriorates a bit. Thermal camera 3a . 3b previously Thermal camera 3a . 3b with pentaprism 23a . 23b pentaprism 23a . 23b Entrance pupil area [mm ² ] ⌀27.5 = 594 594 - 11 × 11 = 473 10 × 10 = 100 Effective diameter of the entrance pupil [mm] 27.5 24.5 11.3 Effective F-number F / 2.0 F / 24.2 F / 4.9

The entrance area 26a . 26b of Pentaprism 23a . 23b represents the effective pupil area for the reference beam path 20 . 20 ' as in 13 shown, the thermal imager looks 3a . 3b with the largest part 34 her entrance pupil 28a . 28b at the pentaprism 23a . 23b past. The smaller part 35 the entrance pupil 28a . 28b looks through the pentaprism 23a . 23b in the direction of the collimator 24 , Because of the big one The focal length or base length b and the small collimator pupil are filled by the collimator 24 but only the very small part 35 of the field of view with a diameter of 23 mm, while the much larger part 34 the entrance pupil 28a . 28b at the collimator 24 drop by. The part 35 must be blocked with appropriate apertures to prevent unwanted image information (eg from hot device components or the like) from being detected.

In 14 is a pentaprism 23a . 23b represented with the mechanical dimensions and the principle of operation. The height of the entrance area 26a . 26b is denoted by x. By twofold 45 degree distraction, the pentaprisms produce 23a . 23b a dropping beam that is at an angle of 90 degrees to the incident beam within manufacturing tolerances. This 90 degree deflection is independent of the angle of incidence in the plane defined by these beams. In practice, this means that a pentaprism 23a . 23b regardless of its angular position always deflects the incident beam by 90 degrees. A faulty position of the pentaprism 23a . 23b So creates no faulty storage of the desired 90 degree direction.

The accuracy of the beam deflection is determined by the manufacturing tolerances of the pentaprism 23a . 23b certainly. From the angle errors α ₄ , β ₄ and γ ₄ and the refractive index n of the prism substrate results, as in 15 the total angle error of the outgoing beam is δ ₄ = n · (3α ₄ + β ₄ + γ ₄ ).

Apart from the angle errors in the tangential plane, a pentaprism can be used 23a . 23b also have manufacturing-related pyramidal defects. The pyramidal errors can theoretically be attributed to the tilting of the reflecting surface. In pentaprisms 23a . 23b this results in relation to the tangential plane (drawing plane in 15 ), a corresponding angular displacement. Another error is that the surfaces need not be flat, but may have a very flat curvature, which leads to a refractive power. For technical reasons, this is often positive; in the general case, it will also be cylindrical. The pentaprism 23a . 23b is then a union of a lens with concave mirrors and forms an optical system with a corresponding refractive power. How out 16 As can be seen, positioning errors in the tangential plane do not produce any angular errors, but result in only a beam offset δy. As long as the pentaprism 23a . 23b is operated in the collimated beam path, this only means a pupil offset δy, which is generally harmless. The positioning errors are in 16 indicated by dashed lines. In the left part of the figure in 16 it is an angular error, in the middle part of the figure to a radial displacement and the right part of the figure to an axial displacement.

In 17 is a third embodiment of a stereo camera device according to the invention 1'' with two adjusted, spaced apart and aligned thermal imaging cameras 3a . 3b shown, which with a calibration device 19 '' is provided for their continuous automatic calibration, being used as a calibration device 19 '' the two thermal imaging cameras 3a . 3b each in the area of their entrance pupils 28a . 28b one as pentaprism 23a . 23b have trained optical deflection, which the of the first thermal imaging camera 3a or from their detector 25a from given own heat radiation as a reference image in the observation beam path 17b the second thermal imaging camera 3b deflects.

To the line of sight 17a . 17b the two thermal imaging cameras 3a . 3b can also be passive (without additional heat source or radiation source 21 ) a direct relationship between the two axes of the thermal imaging cameras 3a . 3b getting produced. This is mutually each a small part of the pupil of a thermal imaging camera 3a . 3b with prisms 23a . 23b or mirroring in the pupil of the other thermal imaging camera 3a . 3b diverted. Since the temperature of the detector surfaces is very low relative to the scene, each thermal imager sees 3a . 3b a cold image detail of the detector surface of his counterpart embedded in the scene. Used for the diversion pentaprisms 23a . 23b , then again it is ensured that the reference beams generated by double deflection are always parallel to the output beam, regardless of the exact angular position of the respective pentaprism 23a . 23b and thus the reference system does not generate new errors.

Thermal image detector arrays typically exhibit strong inhomogeneity of their single detectors in terms of dark current and gain. In order nevertheless to achieve an acceptable image quality, therefore, at least one calibration of the detector elements must be carried out. This is known as Non-Uniformity Correction (NUC). However, this is generally not sufficient for a good homogenization of the image background since, in addition to the desired scene radiation, there are other radiation sources in the infrared range which contribute undesirably to the image information. How out 18 for a cutout region of the stereo camera device according to the invention 1' As can be seen, there are the following irradiation mechanisms. In the The surfaces of the measurement setup radiate via the residual reflection of lenses, prisms 23a and lookout windows 16 on the detector surface. This effect also occurs in every thermal imager 3a on. The mounting parts of the pentaprism 23a irradiate the detector 25a homogeneous, as they are in the entrance pupil 28a lie. The inevitable field of view trim in the reference beam path 20 through a tube 36 , means, conversely, that the tube walls radiate strongly and also inhomogeneously into the visual field. The reference image in the collimator 24 and the surrounding structures inevitably become the detector 25a the thermal imager 3a sharply depicted. Overall, there is a high radiation background. Effective measures to reduce this effect are z. B. matt black inner surfaces of the tube 36 with additional aperture rings 36a and a rough matte black finish of the surface of the reference image and the interior of the collimator 24 and introduction of a field stop 37 in an intermediate image plane of the lens of the thermal imager 3a , Nevertheless, a comprehensive calibration including as many unwanted radiation sources is additionally necessary. Due to the expected strong and also inhomogeneous radiation from the reference beam path 20 ' into the thermal imager 3a the calibration method should be able to record as many of these radiation components as possible. The unwanted radiation components are in 18 indicated by dashed arrows. Since the inside temperature of the device and the scene temperature (eg, cold sky) are often extremely different, the calibration should be performed at a temperature near the scene temperature if possible. In the propagation direction of the observation light of the thermal imager 3a is before the pentaprism 23a the thermal imager 3a for calibration of the detector 25a the thermal imager 3a one in the observation beam 17a the thermal imager 3a swiveling defocusing lens 37 intended. Will the defocus lens 38 between the pentaprism 23a and the lookout window 16 Placed, recorded almost all unwanted radiation components.

The scene or landscape is denoted by the reference numeral 39 Mistake.

In 19 is the normal operating case with the defocusing lens swiveled out 38 shown in which the observation beam path 17a and reference beam path 20 ' on the detector 25a be focused. The detector 25a sees beyond unwanted heat radiation from the tube 36 and the inside of the device.

In 20 is the defocus lens 38 pivoted for calibration, causing the observation beam path 17a deliberately defocused and the image of the scene 39 on the detector 25a smeared. Overall, the detector learns 25a This results in a quasi-homogeneous illumination with thermal radiation whose intensity corresponds to the scene temperature. In addition, the unwanted heat radiation hits the tube again 36 and the inside of the device on the detector 25a , If a calibration is carried out in this configuration, then in addition to the scene-typical radiation background, all undesired effects are recorded and can thus be eliminated by suitable image correction algorithms from the subsequently recorded thermal images. In order to prevent the reference mark or the reference image from being burnt in during the calibration process, ie including in the calibration, the heat source or radiation source must be used 21 switched off behind the reference structure or better still be regulated to the internal device temperature. This is done by the thermoelectric cooler or the Peltier element of the radiation source 21 ,

On the image processing device 7 can now in the context of stereo evaluation, a method for continuous automatic calibration of the stereo camera device 1 . 1' . 1'' running, with the reference image on the two thermal imaging cameras 3a . 3b is localized at predetermined, preferably regular time intervals, after which the relative position and / or the relative orientation of the reference image to the reference images of the further thermal imaging cameras 3a . 3b is determined and after which, upon detection of a change, a corresponding correction of the parameters for the stereo evaluation is determined and made.

Due to a detected relative shift of the reference image, a shift of the detector 25a . 25b at least one of the thermal imaging cameras 3a . 3b or a tilting or bending of at least one of the thermal imaging cameras 3a . 3b parallel and / or perpendicular to the reference plane or to the baseline or base structure 14 the stereo camera device 1 . 1' . 1'' recognized. In addition, due to a relative rotation of the reference image, a rotation of at least one of the thermal imaging cameras 3a . 3b or the detector 25a . 25b at least one of the thermal imaging cameras 3a . 3b detected around its optical axis.

The inventive method for continuous automatic calibration of the stereo camera device according to the invention 1 . 1' . 1'' is preferably as a computer program on the image processing device 7 the stereo camera device according to the invention 1 . 1' . 1'' realized, including other solutions of course come into question. For this purpose, the computer program is in a memory element, not shown, of the image processing device 7 saved. By processing on a microprocessor of the image processing device 7 the procedure is carried out. The computer program can be stored on a computer-readable data carrier (floppy disk, CD, DVD, hard disk, USB memory stick or the like) or an Internet server as a computer program product and from there into the memory element of the image processing device 7 be transmitted.

1, 1 ', 1' '

Stereo camera device

2

monitoring device

3a, 3b

Thermal imaging cameras

4a, 4b

visual fields

5

overlapping Area

6 6 '

birds, flock Of birds

7

Image processing means

8th

interface

9

air traffic control system

10

air route of the birds

11

flight corridor

12

crossing area

13

plane

14

base structure

15

casing

16

Outlook window

17a, 17b

Line of sight, Observation beam path

18

aims

19 19 ', 19' '

calibration

20 20 ', 20' '

Reference beam path

21

radiation source

22

ray beam

22a, 22b

Partial beams

23a, 23b

Penta prisms

24

collimator

25a, 25b

detectors

26a, 26b

entry surface pentaprism

27a, 27b

exit area pentaprism

28a, 28b

entrance pupil Thermal camera

29

angle

30

optical Divider element / 90 degree prism

31

crossed arrows

32

shading

33

resulting Wärmebildkamerasehfeld

34

more Part of the entrance pupil

35

smaller Part of the entrance pupil

36

tube

36a

aperture rings

37

field stop

38

Defokussierlinse

39

scene

b

base length

n

refractive index

α ₃

Sehfeldrandwinkel

α ₄

angle error

β ₄

angle error

γ ₄

angle error

δ ₄

whole angle error

δ _y

positioning error

QUOTES INCLUDE IN THE DESCRIPTION

This list The documents listed by the applicant have been automated generated and is solely for better information recorded by the reader. The list is not part of the German Patent or utility model application. The DPMA takes over no liability for any errors or omissions.

Cited patent literature

• - DE 102008018880 [0003]
• - DE 102007050558 A1 [0007]
• EP 1484628 A1 [0008]
• - JP 01251990 A [0009]

Claims (18)

Stereo camera device ( 1 . 1' . 1'' ) with at least two adjusted, mutually spaced and aligned thermal imaging cameras ( 3a . 3b ), which with a calibration device ( 19 . 19 ' ) is provided for its continuous automatic calibration, wherein the calibration device ( 19 . 19 ' ) a radiation source ( 21 ) and a reference beam path ( 20 . 20 ' . 20 '' ), by means of which a reference image from the radiation source ( 21 ) to the respective thermal imaging camera ( 3a . 3b ), one of the radiation source ( 21 ) outgoing collimated beam ( 22 ) in at least two partial beams ( 22a . 22b ), wherein the at least two partial beams ( 22a . 22b ) in each case via at least one of the respective thermal imaging camera ( 3a . 3b ) associated optical deflection element ( 23a . 23b ), and wherein at least one of the optical deflection elements as pentaprism ( 23a . 23b ) is trained. Stereo camera device according to claim 1, characterized in that at least one optical splitter element ( 29 ) for dividing the beam ( 22 ) into the partial beams ( 22a . 22b ) is provided. Stereocameraeinrichtung according to claim 2, characterized in that the optical splitter element as a 90-degree prism ( 29 ) is trained. Stereo camera device according to claim 1, 2 or 3, characterized in that the reference beam path ( 20 ' . 20 '' ) is symmetrical. Stereo camera device according to one of claims 1 to 4, characterized in that the reference beam path ( 20 . 20 ' ) a collimator ( 24 ) having. Stereo camera device according to one of claims 1 to 5, characterized in that the radiation source ( 21 ) a thermoelectric element, in particular a Peltier element ( 21 ) having. Stereo camera device according to one of claims 1 to 6, characterized in that the exit surface ( 26a . 26b ) of the pentaprism ( 23a . 23b ) at least approximately in the area of the entrance pupil ( 27a . 27b ) of the respective associated thermal imaging camera ( 3a . 3b ) is arranged. Stereo camera device according to one of claims 1 to 7, characterized in that the reference beam path ( 20 . 20 ' . 20 '' ) a tube ( 35 ) having. Stereo camera device according to one of claims 1 to 8, characterized in that in the propagation direction of the observation light of the respective thermal imaging camera ( 3a . 3b ) in front of the optical deflecting element ( 23a . 23b ) of the thermal imager ( 3a . 3b ) for calibrating the detector ( 25a . 25b ) of the thermal imager ( 3a . 3b ) one in the observation beam path ( 17 ) of the thermal imager ( 3a . 3b ) in and out swinging defocussing lens ( 37 ) is provided. Stereo camera device ( 1'' ) with two aligned, spaced apart and aligned thermal imaging cameras ( 3a . 3b ), which with a calibration device ( 19 '' ) is provided for their continuous automatic calibration, being used as a calibration device ( 19 '' ) the two thermal imaging cameras ( 3a . 3b ) in each case in the area of their entrance pupils ( 27a . 27b ) an optical deflection element ( 23a . 23b ), which corresponds to that of the first thermal imaging camera ( 3a ) or of their detector ( 25a ) emitted own thermal radiation, as a reference image in the observation beam path ( 17 ) of the second thermal imaging camera ( 3b ) redirects. Stereo camera device according to one of claims 1 to 10, which at least part of a monitoring device ( 2 ) for wind turbines, buildings with transparent areas, runways and / or flight corridors ( 11 ) of airports with stereoscopic detection of approaching or existing birds ( 6 ) or bird swarms ( 6 ' ) for carrying out a surveillance procedure, with parameters such as flight altitude, flight direction, airspeed, type and size of birds ( 6 ) or the bird swarms ( 6 ' ) are determined on the basis of which an assessment is carried out and, where appropriate, issued a corresponding warning, and which is arranged in the field of wind turbines, buildings with transparent areas, runways and / or flight corridors. Monitoring device ( 2 ) for wind turbines, buildings with transparent areas, runways and / or flight corridors ( 11 ) of airports with stereoscopic coverage of themselves hens or existing birds ( 6 ) or bird swarms ( 6 ' ) to carry out a monitoring procedure, with parameters such as altitude, heading, speed, type and size of birds ( 6 ) or the bird swarms ( 6 ' ), on the basis of which an assessment is carried out and, if appropriate, a corresponding warning message is issued, and in the area of wind turbines, buildings with transparent areas, runways and / or flight corridors ( 11 ) at least one stereo camera device ( 1 . 1' . 1'' ) according to one of claims 1 to 10 with at least two thermal imaging cameras ( 3a . 3b ) is provided, which run synchronously during the recording, the recording times are at least approximately identical and their respective fields of view ( 4a . 4b ) an overlapping area ( 5 ) exhibit. Method for continuous automatic calibration of a stereo camera device ( 1 . 1' . 1'' ) according to one of claims 1 to 11, characterized in that the reference image on the at least two thermal imaging cameras ( 3a . 3b ) is localized at predetermined, in particular regular intervals, after which the relative position and / or the relative orientation of the reference image to the reference images of the further thermal imaging cameras ( 3a . 3b ), and according to which, upon detection of a change, a corresponding correction of the parameters for the stereo evaluation is determined and carried out. A method according to claim 13, characterized in that due to a detected relative displacement of the reference image, a displacement of a detector ( 25a . 25b ) at least one of the thermal imaging cameras ( 3a . 3b ) or a tilting or bending of at least one of the thermal imaging cameras ( 3a . 3b ) parallel and / or perpendicular to a reference plane or to a base structure ( 14 ) of the stereo camera device ( 1 . 1' . 1'' ) is recognized. A method according to claim 13 or 14, characterized in that due to a relative rotation of the reference image, a rotation of at least one of the thermal imaging cameras ( 3a . 3b ) or a detector ( 25a . 25b ) at least one of the thermal imaging cameras ( 3a . 3b ) is detected around the optical axis. Method according to claim 13, 14 or 15, characterized in that a starting calibration of the stereo camera device ( 1 . 1' . 1'' ) is carried out. Computer program with program code means for carrying out a method according to one of claims 13 to 16, when the program is stored on a microprocessor of a computer, in particular on an image processing device ( 7 ) of a stereo camera device ( 1 . 1' . 1'' ) according to one of claims 1 to 11. A computer program product comprising program code means stored on a computer-readable medium for carrying out a method according to one of the claims 13 to 16, when the program is stored on an image processor ( 7 ) of a stereo camera device ( 1 . 1' . 1'' ) according to one of claims 1 to 11.