EP1434184B1 - Control of a multicamera system - Google Patents

Abstract

The method involves using at least one first camera (101) and at least one second orientatable camera (102), whereby an image point is selected in the first camera's image and hence an associated real point in the scene observed by the camera is selected. A viewing beam is defined by a selected image point and the second camera is directed to the real point by orienting the second camera with the viewing beam. AN Independent claim is also included for the following: (a) an arrangement for detecting an object in a scene.

Description

The invention relates to a method for controlling a multi-camera system with the features of claim 1 and an apparatus for implementing the method with the features of claim 16.

Multicamera systems are well known. Especially in surveillance systems and alarm systems, there is often a combination of stationary cameras and pan-tilt-zoom cameras. Stationary cameras are cheaper and allow continuous monitoring of a predefined area. A disadvantage of stationary cameras, however, is that an intruder can not be tracked. If an intruder leaves the predefined surveillance field, tracking becomes impossible and the intruder is often lost. Moreover, with such systems, the coarse resolution of the camera often makes it impossible to identify an intruder or detect small objects. Therefore, pan-tilt zoom cameras are used to track and magnify an intruder detected with a stationary camera. The same problem arises in other applications, e.g. in camera systems for traffic observation or sports events in large halls or stadiums. For a user, manually controlling the pan-tilt-zoom camera to quickly and accurately capture an object is generally difficult.

Out EP-A1-0 529 317 and EP-A1-0 714 081 It is known to pass moving objects, with their pursuit with cameras, from one camera to the next. The invention describes an application within closed spaces, for example within a casino. This selects the closest available pan-tilt zoom camera that can track the object as it exits the surveillance area of the camera currently sensing the object. The applications deal with interiors in which the area to be monitored is one level - the floor level. For the objects a known size - person - and ground contact is assumed.

Out US-A-6,359,647 Another invention is disclosed which makes it possible to enable the transfer of objects between cameras and their monitored areas. The cameras used here can be both pan-tilt-zoom cameras and stationary cameras. The aim of the invention is to continuously track an object across multiple cameras, zoomed object tracking and the associated problems are not addressed. It is assumed that the objects move on one level - floor level - have a known size or the distance is determined by the focus setting of the camera lens.

Out US-A-6,215,519 It is known to use a second camera with higher resolution for tracking moving objects that have been detected in a stationary camera with a large field of view. The stationary camera is assumed to be an omnidirectional camera with very low resolution, so that the higher resolution of the second camera is only a normal resolution and accordingly does not require high accuracy. To localize the object with the second camera, it is proposed to place both cameras directly next to one another or to accept the objects on a known ground plane.

Out DE 196 39 728 A1 a video surveillance device is known. This video surveillance device has a permanently installed first video camera. The captured by the first video camera image section is divided into different evaluation windows. If a change in the image is detected in one of the evaluation windows, a second camera sets a predefined setting values, which represent the evaluation windows, via a control device.

The object of the invention is to improve the detection of an object visible in the image of a first camera by means of a second orientable camera.

This object is achieved on the basis of the method features of claim 1 and the device features of claim 16.

The camera system according to the invention consists of a first camera, also referred to as a master camera, and at least one second camera, also referred to as a slave camera. The second camera is advantageously a pan-tilt-zoom camera.

In addition, an object detector is present. As an object detector, any type of suitable device, method or human user is called, in a camera image, an object of interest, eg

People, cars, faces, moving objects, the position of a vanished object, a cloud of smoke, a shadow, etc., can detect. This can e.g. a human user who can access the object via a suitable input medium, e.g. a mouse, detected on a monitor image or other suitable representation of the camera image. Another example is video motion detectors that detect moving objects. Another possibility is to detect objects based on characteristic features, e.g. Color, shape, face recognition. A user is understood below to mean a person who operates the system. An automatic detector should be understood as meaning detectors which function without human intervention. Semi-automatic combinations are also possible in which an automatic detector makes detection suggestions that must be verified by a user. Detection should also be understood here as localization, i. E. the determination of the position of the object in the camera image. A detection generally refers to a position determination in this sense in a camera image. In the case of a user, this is done in advantageous embodiments often by a mouse click. However, a joystick, a touch-sensitive monitor, or other suitable input medium may also be used.

The position can be a point that is assigned to the object or even one or more surfaces. In the case of digital images, the position is often determined by a pixel. This pixel has a finite extent and may be e.g. be represented in a higher resolution by multiple pixels. Also, by the nature of the detector, such as a motion detector or by so-called touchscreens, one or more surfaces may be selected as the position of the object. In the following, we therefore understand one or more surfaces under a point in the image or pixel. A point in the scene imaged by the camera can therefore also have an extension.

In the following, a monitor is to be understood as meaning any suitable device for temporally variable representation of images, text and graphics. This can be, for example, CRT monitors such as PC monitors, surveillance monitors, TV or something similar. It can also be flat screens, LCD screens or other types of displays.

We call a camera image that is really visible to a user but also a non-visible representation of the same image as digital data in a computer. As far as the described method is a user, a monitor picture is meant and the fade-in or marking of lines, dots etc. takes place in the monitor picture in a form perceptible to the user. If a process part is executed by an automatic detector, it is a general, not necessarily visibly displayed image. The drawing or fade in here corresponds to a consideration of the corresponding positions in the image by the affected automatically executed process part.

We will describe the system and procedures as they stand for a user. However, this is generally intended to include the use of automatic or semi-automatic detectors. The method remains the same if a detection step means the detection by any detector. In the case of automatic detection in all detection steps, monitors and input media can be dispensed with in the device, since the information represented and entered thereby is automatically generated and taken into account in the control units.

A master camera is a camera in which an object is detected by an object detector, wherein this detection is to be used to detect the same object in further, so-called slave cameras. The master camera is often stationary, but it can also be equipped with a pan-tilt unit. In fact, each camera is suitable as a master camera in which at any given time a clear association between a point in the camera image and a so-called visual beam is given. A visual ray belonging to a pixel is the location of all points in the space that are imaged by the camera onto that pixel. For real cameras, these are straight lines for extension-free pixels, and cones corresponding to extended pixels are the sum of all straight lines. If in the following procedural given at a point an unstretched straight line is given as a ray of sight, from this cone a suitable straight line is used as a representative, usually belonging to the object center or the object base straight. This may be the case, for example, when a video motion detector detects an area in the camera image or a user does not enter a pixel-perfect position via a touchscreen. The assignment rule between pixels and visual rays, wherein the visual rays are given in a camera-fixed coordinate system, referred to as internal calibration of the camera. The position of the camera relative to a world coordinate system or other camera is referred to as external calibration. The property of the master camera refers only to the current detection task. In a further detection, the same camera can also perform the function of a slave camera if its field of view is adjustable.

The task of the slave camera is to capture the object detected in the master camera. The slave camera is any camera with adjustable field of view suitable. The adjustment of the field of view for the detection of the object by the slave camera is understood to mean the alignment of the so-called camera sector with the object. The camera vector is a predefined directional vector permanently connected to the camera; it is therefore set together with the field of view of the camera. In particularly advantageous embodiments, slave cameras are pan-tilt-zoom cameras (domes or pan-tilt heads). However, any other mechanism for adjusting the field of view is also suitable.
A zoom feature is not necessary for all applications, but often beneficial. It is assumed that the slave camera is externally calibrated, ie its position and orientation relative to the master camera is known. In addition, the adjustment mechanism of the field of view is assumed to be calibrated, ie the camera sector can be set to specific directions and the zoom can be selectively influenced. The function of the slave camera is only for this detection process. The same camera can also be a master camera in another acquisition process.

A system may consist of two or more cameras, with each camera operating as a master camera and one or more cameras having an adjustable field of view operating as slave cameras for a specific object detection. For the next object detection, these can be other camera pairings or the roles of the last pairing can be reversed. An object can also be detected by several master cameras at the same or different times. In the camera image of the master camera, it is also possible to detect a plurality of objects which are then detected by a plurality of slave cameras or an object is detected, which is then detected by a plurality of slave cameras. The search of the object is restricted in all cases to the visual beam of the object given by the respective master camera. For multiple objects, there is a line of sight through each object.
In the following the invention is illustrated by means of figures. This exemplary description does not represent a limitation or limitation of the invention to the listed examples. It merely serves to illustrate the invention and particularly advantageous embodiments.

especially the FIGS. 3 to 5 allow a quick basic understanding of the invention.

Figur 1

ein Blockschaltbild eines Systems;

Figur 2

eine schematische Darstellung einer Kamera;

Figur 3

einen schematischen Plan eines zu überwachenden Areals mit einem System aus zwei Kameras, sowie eines ausgewählten Objekts und dem dazugehörigen Sehstrahl der Masterkamera;

Figur 4

ein Kamerabild der Masterkamera aus Figur 3;

Figur 5

ein Kamerabild der Slavekamera mit eingeblendetem Sehstrahl der Masterkamera;

Figur 6

einen schematischen Plan eines zu überwachenden Areals mit einem System aus zwei Kameras, wobei die Slavekamera in mehreren Ausrichtungen entlang des Sehstrahls der Masterkamera dargestellt ist;

Figur 7

Kamerabilder der Slavekamera in Figur 6, die mit verschiedenen Ausrichtungen entlang des Sehstrahls der Masterkamera erfasst werden;

Figur 8

ein Kamerabilder der Slavekamera in Figur 6, die mit Ausrichtungen auf und neben dem Sehstrahl der Masterkamera erfasst werden, sowie eingeblendeten Pfeilen zur Steuerung der Slavekamera;

Figur 9

einen schematischen Plan eines zu überwachenden Areals mit einem System aus zwei Kameras, wobei sich Personen in verschiedenen Entfernungen entlang eines Sehstrahls der Masterkamera befinden;

Figur 10

Kamerabilder der Slavekamera in Figur 9 bei einer Ausrichtung auf die Personen in Figur 9 ohne Zoomanpassung;

Figur 11

Kamerabilder der Slavekamera in Figur 9 bei einer Ausrichtung auf die Personen in Figur 9 mit Zoomanpassung;

Figur 12

ein Kamerabild der Masterkamera aus Figur 3 mit eingeblendetem Kameravektor der Slavekamera bzw. dazugehörigem Sehstrahl;

It shows:

FIG. 1

a block diagram of a system;

FIG. 2

a schematic representation of a camera;

FIG. 3

a schematic plan of an area to be monitored with a system of two cameras, and a selected object and the associated visual beam of the master camera;

FIG. 4

a camera image of the master camera FIG. 3 ;

FIG. 5

a camera image of the slave camera with visual beam of the master camera inserted;

FIG. 6

a schematic plan of an area to be monitored with a system of two cameras, wherein the slave camera is shown in multiple orientations along the visual beam of the master camera;

FIG. 7

Camera images of the slave camera in FIG. 6 which are detected with different orientations along the visual beam of the master camera;

FIG. 8

a camera images of the slave camera in FIG. 6 , which are detected with alignments on and next to the line of sight of the master camera, as well as on-screen arrows for controlling the slave camera;

FIG. 9

a schematic plan of an area to be monitored with a system of two cameras, with people at different distances along a visual beam of the master camera are;

FIG. 10

Camera images of the slave camera in FIG. 9 in an orientation to the persons in FIG. 9 without zoom adjustment;

FIG. 11

Camera images of the slave camera in FIG. 9 in an orientation to the persons in FIG. 9 with zoom adjustment;

FIG. 12

a camera image of the master camera FIG. 3 with displayed camera sector of the slave camera or associated visual beam;

In the following description corresponding reference numerals are used in the individual figures for corresponding objects. This makes it possible to describe the invention in a coherent framework.

FIG. 1 shows the block diagram of the system. By way of example, a first camera 101 as a master camera and a second camera 102 with an adjustable field of view as a slave camera are shown here. The master camera 101 is a stationary camera. The master camera 101 is connected to a control unit 140 which connects the master camera 101 with a central control unit 142 via analog outputs, in particular, for example, for the video signal and digital outputs, in particular, for example, for control signals via networks. If digital cameras or image transmissions are used, the network for image transmission may be completely or partially digital. If the mater camera 101 has an adjustable zoom lens, it will be adjusted from the central control unit 142 via the control unit 140. The current value is stored either in the central control unit 142 or queried via the control unit 140. In the same way, the control unit 141 controls and connects the pan-tilt-zoom camera 102. Here, in addition to the zoom, Pan and Tilt can also be set and the current values can be interrogated. The central control unit 142 thus has access to the control, the current settings and the camera images of all participating cameras 101 and 102. The camera images can be displayed on the monitors 144 and 145 connected to the central control unit 142. If automatic detectors are used for the master camera 101 and / or the slave camera 102, the corresponding monitors 144 and 145 can also be connected directly to the control units 140 and 141 or directly to the cameras 101 and 102 or they can be omitted altogether. In the case of automatic detectors without user intervention or control, it is also possible to dispense with flashes in the camera or monitor image. It is also possible to use only one monitor, wherein the images from the master camera 101 and the slave camera 102 are displayed next to one another, in the so-called split method, in one another, in the so-called picture-in-picture method or in chronological succession. For the first detection in the master camera 101 while the image of the master camera 101 is switched, for the subsequent detection in the slave camera 102 then this camera is switched on.
In the following description, as far as the detector is a user, one monitor per camera image is assumed. The central control unit 142 is capable of displaying information such as a mouse pointer, marked dots and visual rays in the displayed images. If required, graphic controls, such as sliders, can also be displayed in the camera image or, if this is not screen-filling, on the monitor outside the displayed camera image. A user may make the necessary controls via an input device 143, which may be advantageously a mouse and / or keyboard and / or slider and / or control panel and / or joystick, etc. The central control unit 142 also stores the data for the internal and external calibration of the cameras 101 and 102. If available, terrain models are also stored and automatic detectors, such as motion detectors, etc. installed. All information and modules for one of the cameras 101 or 102 may also be located in the local control units 140 or 141. at these modules are advantageously modules for calibration and / or automatic detection. If such modules are installed in local control units, only the correspondingly processed information, such as the position of the detected object, converge in the central control unit 142. In the central control unit 142, all inputs and information are collected and processed, ie, for an object detected in the master camera 101, the associated visual beam 303 (in FIG FIG. 3 ) are calculated in world coordinates and, as far as the visual beam is detected by the slave camera 102, superimposed on the camera image of the slave camera 102 (visual beam 504 in FIG FIG. 5 ). Further overlays, according to the embodiments described below, are made and the slave camera 102, for example, according to the embodiments described below, controlled. If a user operates the system, appropriate inputs are taken into account by the central control unit 142 or controls corresponding to the user are offered.
FIG. 2 shows a schematic representation of a camera. As shown in this drawing, cameras are often modeled as so-called projective or even hole cameras. All visual rays pass through one point, the so-called optical center 260 of the camera. Through the optical center 260 and a pixel 206 on the image plane 262 (one pixel on the CCD or CMOS chip in digital cameras), a line of sight 203 is defined, which is the location of all points in space imaged onto the pixel 206 by the camera become. Many real cameras can accurately match this model by algorithmically correcting distortions in the image. For other cameras, there are similar models. The orientation of a camera with adjustable field of view (slave camera 102 in FIG. 1 ) does not necessarily occur in the middle of the picture. If the camera is not modeled as a projective camera, it may also be useful to define the orientation of the camera by a beam that does not pass through the optical center of the camera or there is no unique optical center. Therefore, a general camera sector 261 is defined that defines the orientation of the camera. If the camera is modeled as a projective camera, one can advantageously anchor the camera sector 261 in the optical center 260, as shown in the drawing. The camera sector 261 then corresponds to a specific line of sight and corresponds to the orientation of the camera sector the orientation of the associated pixel (pixel) on the object. For slave cameras with zoom capability, this pixel can be advantageously chosen as the zoom center, ie the pixel which remains stationary in the image as the zoom changes. Any other pixel, such as the center of the image, is also possible.

FIG. 3 shows a schematic plan (top view) of an area to be monitored with a system of two cameras 301 and 302. The camera 301 is the master camera, the camera 302, the slave camera. The two cameras 301 and 302 are eg surveillance cameras and serve for the visual surveillance of the area. In the area are the objects 311 (house), 312 - 315 (trees) and 316 (cars). The master camera 301 detects in its indicated by the boundary lines 321 field of view of this camera associated monitoring area. Surveillance areas often do not cover the whole field of vision. This is indicated by the lines 370 and 371, which represent here by way of example beginning and end of the monitoring area. In this monitoring area are the objects 314, 315, 316, as well as the object 311. If the master camera 301 equipped with an adjustable field of view, the imaged scene represents the orientation of the camera at the time of object detection. The slave camera 302 has an adjustable field of view, eg as pan-tilt-zoom camera. Line 307 denotes the line of sight associated with the camera camera of the slave camera (or the line defined by the camera vector in the general case). By aligning the slave camera with an object, we understand the orientation of this line on the object.
FIG. 4 shows a camera image of the master camera 301 FIG. 3 as it is, for example, for a user on a monitor. The image shows the objects 311, 314, 315 and 316 FIG. 3 , There is an input unit, eg a mouse, by means of which a user can select a pixel 406 in the displayed scene. This point 406 can be marked by the system on the screen, in FIG. 4 this is done by a black dot. The point 406 indicates an object in the scene, in this case a corner of the house, from the slave camera 302 FIG. 3 should be recorded. This object corresponds to the area plan of FIG. 3 the world point 305 with the associated visual ray 303. The distinction between the World point 305 and the pixel 406, and between the (world) visual ray 303 and its imaging / projection in a camera image (504 in the camera image of the slave camera in FIG. 5 ) is essential. The entire visual beam 303 is formed in the camera image of the master camera on the one pixel 406. The detector marks a pixel in the camera image of the master camera and thus a line of sight. The position of the corresponding world point on the visual ray, ie its distance from the master camera, is still unknown. The location of the world point and the effective control of the slave camera 302 is the subject of the embodiments of the invention described below. The point 406 in FIG. 4 or 305 in FIG. 3 is with adapted reference numerals also in the FIGS. 5 to 7 located.
If the slave camera 302 can be arranged close to the master camera 301, with proper alignment the visual beams of the master camera 301 and the camera camera of the slave camera run quasi parallel, the object distance is then irrelevant to the orientation of the slave camera. The detection by the slave camera 302 is relatively easy to solve in this case. A special case also arises if the terrain of the monitored area is known, especially if it is a very simple terrain form such as a floor level in interior monitoring. In this case, assuming that the object is on the ground, it is relatively easy to calculate the object distance and direct the slave camera 302 at the object. In more complex terrain, the terrain shape could also be determined by the camera system itself. This is possible, for example, by calculating distance maps by means of known stereo or multicamera methods. A distance map is a map that stores the distance to the point of interest represented for each pixel in the camera image of the master camera. Another option, if the object size is known, is to use this to estimate the object distance and thus align the slave camera 302. Of all these cases, however, only a small to moderate accuracy is to be expected because cameras can not be in the same place, as object sizes are only vaguely known and inaccurate detected or variable (different objects, different views) and there terrain forms and distance maps normally only vaguely known. For the use of terrain forms / distance maps also the ground contact of the objects must be assumed and that the in Camera image of the master camera detected point 406 is located at a known height above ground.

The required accuracy is generally determined by the zoom, i. depend on the camera opening angle. Assuming that the detection by the slave camera 302 has an accuracy of e.g. 1/10 of the camera aperture angle, this means that for a large zoom of about 2 degrees aperture angle, an accuracy of setting the slave camera 302 by 0.2 degrees. This accuracy is not achievable with the methods given above in most cases.

In all other cases where none of the above enumerated or equivalent methods are applicable or if high accuracy is required, the position of the object by its mark is not fixed in the camera image of the master camera and the object must be searched. For a user, however, it is very complicated to set a pan-tilt-zoom camera on an object and track it. Until a user has set the slave camera 302 to a moving object detected in the master camera 301, it has very often already left the monitoring field and the user has lost sight of it. In addition, in this time the user's attention is subtracted from the rest of the monitoring, so that the scenes displayed on other monitors are not monitored. Also, it is very complicated for a user with a pan-tilt zoom camera set to a large zoom to track a target. In this case, only a small amount of environmental information is available for orientation. If the object is lost for a short time, it is therefore usually not findable again. The simultaneous and independent control of Zoom, Pan and Tilt by a user makes the handling of the system slow and complicated. When fully or partially automating the detection and tracking of objects with a multi-camera system, these issues translate into increased search effort, making the application slow or even unreliable.

The object search in an effective way for the user or an automatic detector is therefore the subject of the following and further described embodiments.

By the in FIG. 4 marked point 406 becomes a line of sight 303 in FIG. 3 Are defined. For a user, entering this point means, for example, a mouse click. By a priori knowledge, the object distance can be restricted to a certain area. Often this is due to the size of the master camera 301 in FIG. 3 detected monitoring range, which is shown as an example between the boundary lines 370 and 371 in FIG. 3 lies. The slave camera 302 is then automatically set by the system (pan, tilt and zoom) as in FIG. 3 shown. This is defined by the boundary lines 322 in FIG FIG. 3 indicated field of view completely captures the line of sight 303 of the pixel 406 within the surveillance area. If complete acquisition is not possible, the largest possible part can be recorded and a warning displayed. If necessary, the focus can be automatically adjusted so that objects appear as sharp as possible along the line of sight.

FIG. 5 FIG. 12 shows the camera image of the slave camera 302 thus set as it is presented to a user on a monitor or an automatic detector. The sight ray 303 off FIG. 3 is displayed in an advantageous manner (line 504 in FIG. 5 ), so that the user can orient themselves but depending on the application, the image is not obscured. Advantageously, the line of sight 504 is displayed in color, dashed and / or semitransparent, etc. The user or an automatic detector must now search the image only along the line of sight 504 to find the pixel 506, which is an image of the world point 305 sought FIG. 3 in the camera image of the slave camera. For a user, this means eg a second mouse click. For an automatic detector, the line of sight 504 can be an imaginary line. The detection in the camera image of the slave camera determines the position of the object on the line of sight 303, and the slave camera 302 can be automatically aimed at the object and set to high zoom. Restricting the search to this line is of course advantageous for an automatic detector, but for a user as well, the insertion of the Sehstrahls 504 and the restriction of the search along the line of sight 504 be helpful. This is the case, for example, if the object is very low in contrast and the image is of poor quality or if there are a large number of similar objects, for example a person in a crowd of viewers.
However, a great advantage in any case is that the user or detector with two detections (for a user eg two mouse clicks) and two associated movements of the slave camera 302 has detected the object. If the object in the camera image of the master camera was detected as an area, the search area would be correspondingly given by the sum of the visual rays.

Another embodiment, in which the advantages of using the visual ray are particularly clear, will be described below.

FIG. 6 shows the same watch scene as FIG. 3 with a master camera 601 and a slave camera 602. In addition, the user is presented on the screen or in the input device 143 in FIG FIG. 1 a slider is offered. The goal is again to align the slave camera 602 with the world point 605. As described above, in a first step, the associated pixel 406 is detected in the camera image of the master camera ( FIG. 4 ). Unlike in the Figures 3 and 5 The slave camera 602 is already set to a large zoom, which is indicated by the boundary lines 322, 323 and 324 of the field of view for three different orientations of the slave camera in the drawing. This has the advantage, for example, that a high resolution is already given during the search and during the detection of the object by the slave camera 602. This can, for example, make the detection easier or only possible. There is no time to set the zoom after the detection of the object, in which the object could have moved on already.
As already stated, the search for the object is very difficult with large zoom and without help, since there are no orientation possibilities and the simultaneous control of Pan and Tilt, especially at high zoom and in a large search space, is difficult. The system therefore offers the user the option of moving the slave camera along the line of sight 603 by moving the slider. The search is so much easier and faster because only this one degree of freedom to use and is to search. In addition, the search on the visual ray S is automatically restricted to the range determined by the apriori knowledge. The slider can be provided with distance information on the line of sight 603, for example. The slider can also be replaced by other input media with one degree of freedom or input media whose function is limited to one degree of freedom. For example, the normal pan-tilt control of a camera via a joystick or buttons for object detection can be switched so that one of the two deflections or a pair of keys the slave camera moves on the line of sight, while the other deflection or the other pair of buttons is turned off Zoom the slave camera or the deflection of the slave camera perpendicular to the line controls or other tasks.

FIG. 7 FIG. 11 shows representations of camera images taken from the slave camera 602 in FIG FIG. 6 with the orientations 622, 623 and 624 along the line of sight 603. The camera image 722 displays the object 612 FIG. 6 , the camera image 723 shows the objects 613, 614 and 616. The object (the corner of the house) selected by the point 406 is in the camera image 724 in FIG FIG. 7 from the perspective of the slave camera 602 and zoomed in (point 706). In the case of automatic detectors, the slave camera 602 can be controlled by the same procedure, the slider is then of course omitted. Was the object in the camera image of the master camera ( FIG. 4 ) detected as an area, one of the visual rays, such as the middle, must be selected in this embodiment.

In a further embodiment, it is also possible for the user to leave all normal degrees of freedom of the camera control and merely give him instructions in which direction the slave camera 602 must be controlled in order to detect the visual beam or, if the visual beam already detected by the slave camera 602 which direction the slave camera 602 must travel in order to follow the line of sight. The notes can be made, for example, via overlays in the monitor image of the slave camera 602. FIG. 8 shows a camera image 833 of the slave camera 602. The slave camera 602 is still too high to detect the line of sight 603, sd only the Treetops of the objects 613 and 614 are detected. The arrow 809 indicates the direction in which the line of sight is reached on the shortest path. The information could also be displayed on a control panel, eg in which direction a joystick for camera control is to be moved or which of the control keys is to be operated. If the slave camera is moved according to this information, the camera image 823 is reached. The slave camera has detected the visual beam, the camera image 823 is identical to the camera image 723 FIG. 7 , The arrows 804 shown indicate the direction in which the camera must be moved to start the line of sight. Instead of the arrows, the line of sight can also be displayed as a line or in another suitable way.

FIG. 9 shows a further particularly advantageous embodiment. To clarify the concrete task is given to capture the face of a person in high resolution. FIG. 9 shows a scenario with a master camera 901, a slave camera 902 and three persons 982, 983 and 984 located on a line of sight 903 of the master camera 901 at different positions. In the in the FIGS. 6 and 7 described embodiment, the zoom is not adjusted during the process of the slave camera 602, indicated by the fields of view 922, 923 and 924. The corresponding camera images are in FIG. 10 to see. Due to the different distances from the slave camera 902, the person 982 in the camera image 1022 becomes too large (face cut off), the person 984 in the camera image 1024 is detected too small. The person 983 happens to be at a suitable distance (camera image 1023) for the set zoom.

However, by knowing the external calibration and the visual ray 903, the distance of the slave camera to the visual ray 903 for each orientation can be easily calculated and the zoom adjusted accordingly. This results in the camera images 1122, 1123 and 1124 of the slave camera 902 in FIG FIG. 11 , Over the whole range of the sight ray 903 a suitable resolution is given now. The distance information can also be used to control the focus when moving the camera so that a clear image is obtained at all times. This has the advantage over a normal autofocus that the latter is slow, on the one hand, and moving camera and / or one on the other moving objects does not have to work reliably. For example, the possibility of snapshots of the searched object is particularly advantageous for this embodiment. In the master camera 901, as an example, a suspicious person is detected in low resolution. By means of the described embodiment, a high-resolution snapshot of the person's face can be made very efficiently with the aid of the slave camera 902. The snapshot can be triggered by a user, by an automatic detector (here a face detector) or semi-automatically.

In the case of a large zoom, i. a large range sensitivity of the focus, the auto focus of the focus on the visual beam can also support the detection of the object in the slave camera. Often the arrangement is such that the line of sight, before hitting the selected object, is far away from other objects (through the air). The objects that are projected into the camera image of the slave camera along the line of sight then have the 'wrong' distance from the slave camera and are not focused. Starting from the master camera, the first sharply imaged object is in this case the searched object.

Generally, in other embodiments, for any position on the line of sight 303, 603 or 903, the distance from the slave camera 302, 602 or 902 can be easily calculated by triangulation. If the object has been detected by the slave camera on the visual beam, the distance of the object from the master camera 301, 601 or 901 can also be easily determined by triangulation. This information is generally useful for automatically adjusting zoom and focus and for other tasks.

In all embodiments, the detection or tracking of a moving object in the camera image of the master camera 301, 601 or 901 can also take place continuously. The line of sight 303, 603 or 903 and the control of the slave camera 302, 602 or 902 are then automatically adjusted continuously. In this case, one can further simplify the search for a user as well as for automatic or semi-automatic detectors, since it is no longer necessary to search the entire visual beam, but to search through the previous position and speed of the object the search is highly restrictable.

It is particularly advantageous if, during the movement of the second camera 02 along the line of sight 03, the focus is automatically adjusted such that objects are sharply imaged on the line of sight.
In addition, it is advantageous if, during the process of the second camera 02 along the line of sight 03, the searched object is thereby automatically detected, whereby starting from the first camera 01 with automatically focused on the line of sight 03 zoom the first sharply imaged or the sharpest pictured object is.

Furthermore, it is advantageous if the one-degree-of-freedom controller is implemented as a slider or joystick or in the form of keys, wherein a deflection of the joystick moves the respective camera on the line of sight and the input medium used in normal operation for controlling the second camera 02 can be switched for the object detection task in such a way that it fulfills the function for object detection.

In a further advantageous embodiment of the invention, the master camera, as in FIG. 3 shown equipped with a focused light source 350, eg a laser. The light source is adjustable in its orientation, the spectrum does not have to be in the visible range. The focused light source 350 is mounted as close as possible to the master camera 301 and can thus be used for a selected point 406 (FIG. FIG. 4 ) along the associated line of sight 303. The beam then hits the selected object and illuminates it. If the slave camera 302 is sensitive to the radiation used, this may be used to detect the object by the slave camera 302. This applies in particular if the radiation leads via its spectrum, its intensity, via a pulsed operation, via combinations of these or other features, to a clearly detectable signal in the camera image of the slave camera. As a result, the detection of the object in the camera image of the slave camera can be facilitated for a detector, in particular if a plurality of similar objects are visible along the displayed visual beam 504. Normally, only one of these objects is located actually on the world's visual ray 303 and is therefore marked by the light source. Other objects are only in the projection of the camera image on the visual beam, such as the objects 712-714 in FIG. 7 , They are therefore not marked by the light source. This procedure is general, ie also in the in FIGS. 6 and 9 described embodiments, applicable. A special feature arises if the light source is equipped with a distance measurement (laser scanner). In this case, the object distance is known directly and the slave camera can be aimed directly at the object. It is particularly advantageous if the focused light source is coupled to a distance measuring device, in particular in the function of a laser scanner, and the distance information thus obtained is used to direct the second camera 02 directly to an object on the line of sight 03 at this distance.

Another embodiment is in FIG. 12 shown. The figure shows as well FIG. 4 a camera image of the master camera 301 off FIG. 3 , The superimposed visual beam 1208 is the visual beam 307 belonging to the camera vector of the slave camera FIG. 3 (or the straight line defined by the camera vector in the general case). This allows a user to instantly recognize the current tilt of the slave camera. For handling multi-camera systems, the user is often offered location maps of the area on a monitor in which the cameras are shown. Since the site plans are top views, it is also easy to display the current pan of a pan-tilt-zoom camera by a corresponding symbol. This facilitates the user's orientation in controlling the camera. It is also more difficult to display the tilt of the camera in a way that facilitates the user's orientation in the scene. By inserting the camera sector of the slave camera or the associated visual beam into the camera image of the master camera, this is easily possible. In FIG. 12 For example, it can clearly be seen that the slave camera is looking too deep, since the camera sector's line of sight is below the searched object 1206. In the case of a known terrain shape, even the point of impact of the visual ray 307, ie the position detected by the slave camera 302, can be marked in the camera image of the master camera.

LIST OF REFERENCE NUMBERS

01

erste Kamera (Masterkamera)

02

zweite Kamera (Slavekamera)

03

ein Sehstrahl der Masterkamera in der Welt

04

ein Sehstrahl der Masterkamera, abgebildet in einem Kamerabild

05

Weltpunkt

06

Bildpunkt

07

Kameravektor (bzw. dessen Sehstrahl) der Slavekamera in der Welt

08

Kameravektor (bzw. dessen Sehstrahl) der Slavekamera abgebildet in einem Kamerabild

09

Pfeileinblendung im Kamerabild

11-16

Objekte in der Welt bzw. deren Abbildung in einem Kamerabild

22-24,33

Gesichtsfelder der Slavekamera oder dazugehörige Kamerabilder

43

40-42 Steuereinheiten Eingabevorrichtung

43

Eingabeeinheiten

44,45

Monitor

50

gerichtete Lichtquelle

60

optisches Zentrum der Kamera

61

Kameravektor

62

Bildebene der Kamera

70,71

Begrenzungslinien des Überwachungsbereichs

82-84

Personen vor der Kamera

The first or the first two digits of the reference numerals designate the figure number. The last two digits in all figures mean:

01

first camera (master camera)

02

second camera (slave camera)

03

a line of sight of the master camera in the world

04

a line of sight of the master camera, shown in a camera image

05

world point

06

pixel

07

Camera vector (or its line of sight) of the slave camera in the world

08

Camera vector (or its visual beam) of the slave camera imaged in a camera image

09

Arrow display in the camera image

11-16

Objects in the world or their image in a camera image

22 to 24.33

Fields of view of the slave camera or associated camera images

43

40-42 control units input device

43

input units

44.45

monitor

50

directed light source

60

optical center of the camera

61

camera vector

62

Image plane of the camera

70.71

Boundary lines of the surveillance area

82-84

People in front of the camera

Claims (19)

1. Method for detecting an object in a scene, particularly for monitoring thereof, comprising at least one camera (01) and at least one second, orientatable camera (02), wherein a image element (06) and thus an associated global point (05) in the scene observed by the first camera (01) is selected in the camera image of the first camera (01), characterised in that a line of sight (03) is determined by the selected image element (06) and the second camera (02) is aligned with the global point (05) in that an orientation of the second camera (02) to the line of sight (03) is carried out.
2. Method according to claim 1, characterised in that the line of sight (03) in the camera image of the second camera (02) is merged in in suitable manner, for example as a continuous line, or as a dashed line and/or is illustrated in semi-transparent or coloured form.
3. Method according to claim 1 or 2, characterised in that the line of sight (03) is recalculated by the control unit (41) of the second camera (2) on the basis of the data supplied by the control unit (40) of the first camera (01) and the second camera (02) is aligned to the global point (05) on the basis of these data from the control unit (41) of the second camera (02) or the line of sight (03) is calculated by the central control unit (42) and merged in.
4. Method according to one of claims 1 to 3, characterised in that the line of sight (S) is defined as the line which is defined by the selected image element (06) and the optical centre (60) of the first camera (01).
5. Method according to one of claims 1 to 4, characterised in that the first camera (01) is managed as a master camera and the second camera (02) as a slave camera, wherein the first camera (01) is a stationary camera or a pan-camera and/or tilt-camera and/or zoom-camera and/or the first camera (01) is equipped with a fish-eye objective, a wide-angle objective, a zoom objective or mirrors or is constructed as a catadioptric camera and/or the second camera (02) is a pan-camera and/or tilt-camera and/or zoom-camera and/or the second camera (02) is equipped with a movement unit which enables setting of the orientation of the viewing field of the second camera (02) so that a region of the scene to be monitored can be covered by the second camera (02).
6. Method according to one of claims 1 to 5, characterised in that the distance of the global point (05) from the first camera (01) and/or the second camera (02) is calculated by way of the setting data of the first camera (01) and the image element (06) and the setting data of the second camera (02).
7. Method according to one of claims 1 to 6, characterised in that the image element (06) is selected by way of a detector, which can be a user, an automatic detector or a semi-automatic detector, and in the case of an automatic detector the representation on a monitor can be dispensed with and/or the object detector is realised as a motion detection unit, coloured feature recognition unit, person, motor-vehicle or face recognition unit and/or the object detector is realised in the control unit (40) of the first camera (01) or in the central control unit (42) or is distributed between the two control units.
8. Method according to one of claims 1 to 7, characterised in that the zoom and the orientation and, if needed, also the focus of the second camera (02) are selected in such a manner that after marking of the image element (06) in the camera image of the first camera (01) the associated line of sight (03) is detected in the camera image of the second camera (02) approximately centrally and as sharply as possible and the predetermined monitoring region is detected along the line of sight over the entire length thereof and/or a warning signal is generated when the associated line of sight (03), which belongs to the selected image element (06), cannot be detected by the second camera (02) over the entire predetermined monitoring region.
9. Method according to one of claims 1 to 8, characterised in that the line of sight (07) belonging to the camera vector of the second camera (02) is merged into the camera image of the scene detected by the first camera (01) so as to indicate the orientation of the second camera (02) and/or to assist control of the second camera (02) and/or the point of incidence of the line of sight (07) belonging to the camera vector of the second camera (02) is marked in the camera image of the first camera (01) if the topography of the monitored scene is known.
10. Method according to one of claims 1 to 9, characterised in that merged into the camera image of the second camera (02) is a directional indication, particularly in the form of an arrow, of the direction in which the second camera (02) must be moved so that the line of sight (03) is detected.
11. Method according to one of claims 1 to 10, characterised in that if the line of sight (03) is already in the camera image of the second camera (02) the line of sight (03) is merged into this camera image, particularly in the form of two arrows of opposite direction or of a line, and it is thus indicated in which direction the second camera (02) is to be moved along the line of sight (03).
12. Method according to one of claims 1 to 11, characterised in that a regulator with one degree of freedom is provided, which controls the orientation of the second camera (02) along the line of sight (03).
13. Method according to one of claims 1 to 12, characterised in that an object size is predetermined and the zoom of the second camera (02) during movement along the line of sight (03) is so automatically set that objects of this size are detected on the line of sight in matching size in the camera image of the second camera (02) and/or during movement of the second camera (02) along the line of sight (03) the focus is so automatically set that objects on the line of sight are sharply imaged.
14. Method according to one of claims 1 to 13, characterised in that during movement of the second camera (02) along the line of sight (03) the sought object is thereby automatically detected, wherein starting from the first camera (01) with zoom automatically set sharply on the line of sight (03) it is the first sharply imaged or the most sharply imaged object and/or the regulator with one degree of freedom is realised as a shift regulator or joystick or in the form of buttons, wherein a deflection of the joystick moves the respective camera on the line of sight.
15. Method according to one of claims 1 to 14, characterised in that an input medium, which is used in normal operation for control of the second camera (02), for the object detection task can be converted in the manner that it fulfils the function for object detection and/or an object in the second camera (02) is detected in that a focused light source with settable orientation is mounted at the first camera (01), which light source is aligned along the line of sight (03) belonging to the image element (06) and by which the object is illuminated and thus marked, and/or an unambiguous detection is facilitated by the wavelength of the light of the light source and/or a pulsed operation of the light source and/or the focussed light source is coupled with a distance measuring device, particularly in the function of a laser scanner, and the thereby obtained distance information is used in order to direct the second camera (02) directly onto an object on the line of sight (03) at this distance.
16. Device for detection of an object in a scene, consisting of at least one first camera (01) and at least one second, orientatable camera (02), wherein a first control unit (40) is associated with the first camera (01) and a second control unit (41) with the second, orientatable camera (02) and in the camera image of the first camera (01) an image element (06) and thus an associated global point (05) is selectable in the scene, which is detected by the first camera (01), by way of a central control unit (42) with associated input units (43), characterised in that the central control unit (42) calculates a line of sight (03) beginning from the first camera (01) to the global point (05) and on the basis of this line of sight (03) the second control unit (41) orientates the second camera (02) and aligns it to the global point (05) in that the second control unit (41) aligns the second camera (02) to the line of sight (03).
17. Device according to claim 16, characterised in that the second control unit (14) of the second camera (02) merges the line of sight (03) into the scene detected by the second camera (02) and represents the line of sight (03) as a continuous line or dashed line and/or in semi-transparent or coloured form and the line of sight (03) is calculated by the second control unit (41) of the second camera (02) on the basis of the data supplied by a first control unit (40) of the first camera (01).
18. Device according to claim 16 or 17, characterised in that the first camera (01) is a stationary camera and/or orientatable camera and/or the first camera (01) is a master camera and the second camera (02) a slave camera associated with the master camera and/or the first camera (01) is a pan-camera and/or a tilt-camera and/or a zoom-camera and/or the first camera (01) is equipped with a fish-eye objective, a wide-angle objective, a zoom objective or mirrors or is constructed as a catadioptric camera and/or the second camera (02) is a pan-camera and/or tilt-camera and/or zoom-camera and/or the second camera (02) is a movement unit which enables setting of the orientation of the viewing field of the second camera (02) so that the second camera (02) can cover a region of the scene to be monitored and/or the central control unit (42) calculates the distance of the global point (05) from the first camera (01) and/or the second camera (02) on the basis of the setting data of the first camera (01) and the second camera (02).
19. Device according to one of claims 16 to 18, characterised in that the central control unit (42) illustrates the scene, which is recorded by the first camera (01), on a display device (44) associated with the first camera (01), the global point (05) is selected by a detector and/or the central control unit (42) merges the line of sight (03) in the scene, which is detected by the second camera (02), on a display device (45) associated with the second camera (02) and/or the selection of the image element (06) is carried out automatically by way of an object detector and/or the object detector is a motion detection unit, a coloured feature recognition unit or a face recognition unit and/or the object detector is realised in the first control unit (40) of the first camera (01).

Images