The invention relates to a video surveillance system with a number of video cameras whose video images can be displayed on a display device to monitor an object located in an observation room, each video camera being stationary, but the orientation being controllable by a connected video camera control device. The video camera controller is connected to a user controller. By means of the user control can manually a
Video camera to be selected, which appears suitable for monitoring the object.
State of the art
Video surveillance systems are used, for example, in railway stations or airports to increase security and to reduce vandalism.
Even with major sporting events, such. in a stadium, video cameras are used to detect possible crimes and
To be able to recognize panic situations in advance and to be able to better manage visitor flows. The evaluation of the monitor images takes place in a control room by one or more operators. Systems are known in which the orientation of the individual video cameras can be preset manually by means of an operating device by the operator. In the video observation of an object in motion, so far has been moved so that the operator tracks the movement of the object until the object threatens to disappear from the picture. At this point, the operator switches to a suitable second camera. In order not to lose sight of the object, this switching must be carried out quickly.
However, this requires on the part of the operator always knowledge of local conditions of the observation room, as well as the location and the detection range of the individual video cameras. Only in this way can she select a camera that is best suited for tracking the object. In addition, with the second camera, the target object must be targeted as quickly as possible in order to focus on it. Due to stress can be such a manual
Handover lost valuable time. It may happen that when switching important image information can not be recorded in the desired quality or not at all.
This is unsatisfactory in the case of legal prosecution or in the attempted foiling of an offense.
Systems are also known that can identify and (for example with a mark) track objects in video images through image processing algorithms. Also known are systems that can use this information to guide other cameras (AW Senior, A. Hampapur, M. Lu (IBM TJ Watson Research Center): Acquiring Multi-Scale Images by Pan-Tilt Zoom Control and Automatic Multi-Camera Calibration ). The image processing algorithms used in such systems are very expensive and require powerful computing units and are therefore rarely in use.
On the other hand, the present invention is based on a user manually guiding a guidance camera, and aims to enable the use of other cameras to observe the selected object with simpler means.
Presentation of the invention
It is an object of the present invention to provide a video surveillance system such that the risk of losing image information when switching between video cameras is minimized.
The invention is based on the assumption that video monitoring of an object in motion can take place in a particularly favorable manner if the object is tracked by a manually operated first video camera (guiding camera) and this movement is automatically assigned at least one further second video camera (tracking camera). which also visualizes the environment of the object in real time.
In addition to the video image of the manually operated guidance camera, video images of one or more tracking cameras, which are already focused and focused on the object, are displayed to the operator at the same time. The auto-managed tracking cameras also make the object real-time images, but from a different perspective. The operator can now use this enhanced visual representation to quickly select a video camera other than a guide camera as the object moves out of the image. Once he has chosen and focused on a new guidance camera through a manual operator action, a new set of pre-aligned and focused tracking cameras will automatically follow. The operator no longer has to make a complicated choice.
This lower load reduces the risk of losing important imagery. At the same time, the operator gains a better overview of the observation area with otherwise potentially hidden details due to the different perspectives offered. This is a decisive advantage in the investigation of crimes. When monitoring a stadium, an approaching deviant visitor behavior can be detected very quickly and, if necessary, suitable countermeasures can be taken early.
In a preferred embodiment, the
Video camera control device in the selection of at least one other video camera, a three-dimensional model of the observation room, which is kept in a memory of the video camera control device. The 3-D model contains the coordinates of the locations of each video camera.
The orientation information (spatial and optical) of the guidance camera is also provided by the video camera control device, so that the location information of the object can be determined from the sighting of the guidance camera and by computational linkage of the above information. If the position of the object is fixed, the selection of additional cameras can be done automatically and quickly. A complex image processing (image processing) is not required for the selection.
With regard to the accuracy of the computationally determined object position, it is favorable if, in addition to the target orientation information, the actual orientation of the guidance camera is taken into account. Advantageously, this alignment information is measured locally on the camera and transmitted to the video control device.
The measurement is performed by transducers attached to each actuator of each video camera. The orientation information is provided by the tilt angle, the tilt angle, and the positioning of the vision system's optical system (aperture, focus, and zoom).
In a particularly preferred embodiment of the invention, the operating device of the video surveillance system on a joystick, a so-called. Joystick on.
The joystick makes it easy to sight and manually follow a guide camera.
Advantageously, the joystick hereby assigned a switching device by means of which the joystick can be used alternately either for spatial orientation of the Fuhrungskamera or for alignment of the optical system of the Fuhrungskamera.
Of particular advantage here is the coupling of the joystick with a trained in the control device force generating device.
Thereby, a force counteracting the manual actuation force can be generated, which enables the user to tactually perceive positioning limits.
To signal a limit range of an actuating movement to the user as early as possible, it is favorable if the counterforce increases with manual deflection.
Has proven to be particularly favorable when the
Relationship between the generated counterforce and the manual deflection of the joystick is non-linear.
The counterforce to be overcome becomes increasingly greater the closer the user comes to the limit.
It has proven to be expedient if the video image of the guide camera and the video image of tracking cameras are each displayed on separate monitors.
For archiving the image material are particularly suitable video servers, as they are already used in the monitoring of public buildings.
Brief description of the drawings
To further explain the invention, reference is made in the following part of the description to the drawings, from which further advantageous embodiments, details and further developments of the invention can be found.
Figure 1 shows an exemplary embodiment of the invention
Video surveillance system in a simplified block diagram;
Figure 2 is a flow chart for an automatic
Object tracking based on camera position data and a 3D model of the surveillance room;
Figure 3 is a diagram illustrating the counter force operation of the joystick;
Figure 4 is a schematic representation of a video camera in a position prior to alignment with a target object, with the target object located on a tribune of a stadium.
Exemplary embodiment of the invention
In the figure 1 an embodiment of the invention is shown as it is for the observation of visitors to a sporting event, e.g. a football stadium. Reference numeral 13 denotes the video surveillance system throughout.
The video surveillance system 13 essentially consists of several video cameras in operation, of which only the video cameras 1, 2, 3 can be seen in FIG. 1 for clarity, a video camera control device 15 and an operating and display device 9, 11. Observed is the target object 4 which is located in an observation room 5 and is freely movable there. The observation space 5 is sketched in FIG. 1 as a delimited spatial area by a dashed line.
The video cameras 1, 2, 3 are so-called pan tilt cameras (pan-tilt-zoom cameras), that is to say they are stationary with respect to the coordinates X, Y, Z on a mast 20, but with regard to the orientation of their optical axes 14 on the object to be observed 4 pivotally.
Within predefined limits, therefore, the viewing direction of each of these video cameras 1, 2, 3 can be aligned. The pivoting movement is predetermined by in Fig. 1 is not shown closer and known per se positioning drives an adjusting device. As drives, for example, stepper motors or electric motors can be used in a position control loop. The optical setting (focus and zoom) is also specified by a corresponding electric positioning drive. Each positioning drive is coupled to a transmitter to metrologically detect the position. This actual quantity is reported back to the video camera selection and control device 6 of the video camera control device 15 as an electrical signal via the bus connections 141, 142 143.
The illustrated for the optical adjustment can also relate to the value of the aperture, in other known embodiments, the aperture is fixed to a known value or manually adjustable mechanically.
In this case, the current aperture value of the video camera controller 15 must be known as a configuration parameter. In a large number of the known embodiments, the values for focus are determined either manually or automatically by the camera and adjusted (so-called autofocus mode). The automatic setting is done, for example, by contrast measurements in the image determined by the image recorder not shown individually. This relieves the operator of such setting operations.
The determined adjustment values are available to the video camera control device 15 as described above.
The operation of the video surveillance system 13 is carried out by means of a user control 8 by an operator. The user controller 8 is connected to the video camera selection and control device 6 via a signal-conducting connection 146. The user control 8 has a camera control dome, a so-called joystick 9 and a switching device 10. By means of the joystick 9, a selected video camera can manually with respect to their
Orientation and focusing aligned or the movement of the object 4 nachgefuhrt. The control device 10, among other things, allows the assignment of the joystick 9 to certain positioning drives (spatial orientation, focus and zoom) of a video camera.
The switching device 10 consists of several switches and cubicles, which are arranged in the immediate vicinity of the joystick 9. Through this adjacent arrangement of joystick 9 and
Switching device 10, the position of the hand of the operator can be maintained during operation, which facilitates handling.
The image material supplied by the video cameras 1, 2, 3 passes via the signal-conducting connection 145 to a display device 11. The display is effected by means of the monitors 12.
The video camera control device 15 consists essentially of a video camera selection and control device 6 and a computing device 7, which are connected via the 144. The computing device 7 is a conventional personal computer (PC). The PC 7 has a memory device 100.
The video camera control device 15 generates for controlling the video cameras 1,2,3 control signals by which each of the video cameras 1,2,3 can be focused in their spatial orientation and in terms of their optical system on the object to be observed 4. The memory device 100 contains so-called 3D data, which images the observation space 5 as a three-dimensional model. This includes the coordinates of the installation location of each video camera 1,2,3.
As in the procedure below, using a
Flowchart explained in more detail, selects the operator via the user control 8, one of the video cameras 1,2,3 as a guide camera, which is also referred to below as a lead camera. The selection of the guide camera is freely selectable. The operator positions this lead camera by means of the joystick 9.
As already mentioned, the joystick 9 can be used universally: depending on the specification of the switching device 10, the joystick 9 can be used either for the spatial orientation of the guide camera or for adjusting the optical system of the guide camera. The assignment depends on a switch position of the switching device 10. According to one aspect of the present invention, the joystick 9 is designed to counteract manual design with increasing joystick force feedback.
This joystick counterforce is generated by actuators in the user control 8.
Since the control information for the video cameras 1, 2, 3 are usually transmitted as vector information relative to the current setting with a control value that is often unactuated in advance due to the functional principle or certain values are automatically determined by the camera depending on the operating mode used (autofocus ), the actual orientation or the actual control value for the optical properties (aperture, focus, zoom) is detected metrologically and transmitted via the signal-conducting connection 141, 142, 143 back to the video camera control device 15.
As a result of this readback of actual values of a guide camera (pivot angle 16, tilt angle 17, aperture, focus and zoom) directed onto the object 4, the respective object position of the object 4 to be observed can be calculated on the basis of the three-dimensional model by means of the computing device 7. If this object position is known, a tracking camera, also referred to below as a slave camera, can be selected on the basis of the 3-D model. This selection of a slave camera is based on the 3D data stored in the memory 100: from each eligible video camera, the viewing area is described in the form of a cone, in the top of the video camera and in the center of the
Floor surface the targeted object 4 is located. Between the top and the center of the bottom surface, a location vector 18 to the object 4 is calculated.
Is the view cone or the location vector 18 an obstacle against (that is, the object 4 is partially or completely in one
Shadow area), this video camera can not be used as a slave camera. If, on the other hand, the view of the object 4 is unhindered, then this video camera is eligible for selection as a slave camera. In this way, the suitability for each operational video camera is checked successively.
FIG. 2 shows a flowchart with reference to which the automatic object tracking according to the invention is explained in more detail by program steps:
First, it is assumed in the process description that an operator has manually selected a first camera as a lead camera and has directed this by means of the joystick 9 to the object 4 to be observed.
In case of movement of the object 4, the operator traces the master camera of movement of the object 4.
In a first program step (1), positioning data (alignment information) are determined by the lead camera. This determination is made by reading back the respective actual values from the periphery, ie tilt angle 16, tilt angle 17 as well as aperture, focus and zoom. These positioning data are transmitted via the bidirectional bus 141, 142, 143 to the video camera evaluation and control device 15 and subsequently via the bus 144 to the computing device 7.
In the subsequent second program step (2), the targeted object position is calculated by the computing device 7.
For this purpose, the positioning data of the guide camera determined in program step (1) are used together with 3D data which are stored in the memory 100 of the computing device 7. In conjunction with the 3D data, a verification can be performed so that the object position can be displayed accurately. The result of this calculation is a location vector 18, which points from the guidance camera to the instantaneous object position.
Following this, in the program step (3), at least one further video camera is selected as a tracking camera from the number of installed video cameras. This selection is likewise carried out by the computing device 7 on the basis of the object position determined in the second program step (2) and the coordinates of the respective installation location of a video camera.
The result is at the end of program step
(3) Determine which video cameras are suitable as a tracking camera (s) (slave cameras).
In a subsequent fourth program step
(4) the computing device 7 calculates positioning information that is suitable for aligning the tracking camera (s) selected in program step (3) so that they point to the object position (calculated in program step (2)) and to the object 4 are focused. This control information is transmitted via the signal-conducting connection 144 to the video camera selection and control device 6 and from there via the connection 141 or 142 or 143 to the selected or selected Nachfuhrkamera (s).
In program step (5), the automatic alignment of the tracking cameras takes place.
As can be seen from the flowchart of FIG. 2, this program loop is closed by a jump back to program step (1).
As shown in the flow chart of Figure 2, is in
Program step (6) offered to the operator to change the association between Nachfuhrkamera and Fuhrungskamera: He decides on the basis of the monitor images displayed on the display device 11, whether for further monitoring of the object 4 is not a displayed monitor image of a
Nachfuhrkamera proves to be cheaper as a new guide camera.
If this is the case, he selects in the program step (7) by appropriate input to the user control 8 this video camera as a new guide camera and the process shown starts again.
FIG. 3 shows in a diagram the force curve of the joystick counter force F (force feedback strength) as a function of a camera angle (tilt angle or tilt angle). The counterforce is generated in the user control 8 by suitable actuators. The size of the counterforce can be specified by the user control 8. As can be seen from FIG. 3, no counterforce is at all effective at a specific interval around the zero point of the camera angle. After this interval, the counterforce to be overcome by the operator progressively increases with the camera angle (camera angle).
Thereby, the near end of a parking area (for example, the panning operation of a video camera) becomes noticeable by the operator's hand. This means that the operator not only has visual control, but tactile information as well. When the joystick 9 is used for tilting and tilting the guidance camera, the motor-driven feedback means that the operator notices the approaching end of the camera movement early, that is, even before the camera has reached a mechanical stop. This makes it possible for him to select another camera as a new guide camera in good time. The same applies to the use of the joystick 9 when focusing and / or when zooming the guide camera.
This is particularly advantageous when one considers that a digital transmission of the video data, especially in the case of high compression (MPEG4), there is a significant delay between image acquisition on the camera and the visual display. Due to this delay, an operator concentrating exclusively on the visual presentation when focusing and zooming may notice too late the end of an optical adjustment possibility. The tactile signaling of the "Force Feedback" makes the guidance camera more efficient to handle. The operation is significantly improved.
As already shown above, the selection of a
Fuhrungskamera always by a manual switching action of the operator.
It may be that during the observation of a scenario several times between guide camera and Nachfuhrkamera is changed. The selection of a suitable Nachfuhrkamera is always automatic, the selection of a Fuhrungskamera, however, always manually. In the present application, in which a plurality of video cameras is present and these are operated by multiple operators, it may be advantageous to regulate the authorization of the individual operators in the form of a priority system. The prioritization system grants hierarchical tiered permissions to the operators and also takes into account the distance of each video camera to the current location of the target object 4.
FIG. 4 shows, in a schematic representation, a video surveillance camera 1 which is aimed at a viewer 4 who is located on a tribune 19 of a stadium.
The grandstand 19 is shown in Figure 4 by the Cartesian coordinates 0,0, 0; 100.0, 0; 100,20,100 and 0, 20,100 models. The video-surveillance camera 1 is fixedly arranged on a mast 20 and rotatably mounted about a ball joint 21. Your Aufstellkoordinaten, that is, the coordinates of the ball joint 21 are -50, 100, -50. In order to align its optical axis 14 (dashed line in Figure 4) on the visitor 4, the camera 1 is pivoted about the pivot angle 16 and inclined by the tilt angle 17. Thus, the location vector 18 (solid line in FIG. 4) of the camera 1 points to the target person 4 on the platform 19. The location vector 18 thus contains spatial direction information (the spatial orientation of the video camera 1) as well as a
Distance information (the length of the location vector 18, i.e. the distance between the video camera 1 and the viewer 4).
When the optical system of the video camera 1 (focus and zoom) is focused on the object 4, this optical adjustment corresponds to the spatial distance between the video camera 1 and the object 4. When this video camera 1 is aligned and focused on the visitor 4 the position data from the periphery back to the video camera controller 15, that is transmitted to the computing device 7. As already mentioned, in the memory device 100 of the
Computing device 7 both the spatial coordinates of the video cameras (in Figure 4, the video camera 1 with the Cartesian coordinates -50, 100, -50 and 3D data, here the Schragflache 22 of the grandstand 19 illustrated by data technology.
The calculation of the location coordinates (65, 10.75) of the target object 4 (visitor 4 on the platform 19) results from the intersection of the location vector 18 (optical axis 14 of the video surveillance camera 1) with the oblique surface 22 of the grandstand 19. From this location information with the help of the 3D model suitable Nachfuhrkameras (slave cameras) selected. The selection is done by computer based on the 3D data. If the field of view of a video camera in question to the visitor 4 is not affected, this comes as a tracking camera (slave camera) in question. When the selection of the tracking cameras is completed, the calculation information for these slave cameras is calculated so that they can be aligned with the visitor 4 and focused on him.
Each video image of a selected Nachfuhrkamera is placed in the control room on a monitor 12 for display. Is it in the sports stadium under the
Viewers to tumult or panic situations are e.g. Preparations for throwing fireworks are suspected, so they can be observed quickly and from different perspectives. As a result, any necessary countermeasures can be taken early on. If an offense is observed, relevant evidence can be recorded in a short amount of time and temporarily activated in a video server.
Compilation of the reference numbers used
1 first video camera
2 second video camera 3 third video camera
4 target object
5 observation room
6 video camera selection and control device
7 computing device 8 user control
10 switching device
11 display device
12 monitor 13 video surveillance system
14 optical axis
15 video camera control device
16 swivel angle
17 tilt angle 18 position vector
19 Tribune (Schragflache) 20 Camera mast
21 ball joint
22 oblique surface 100 memory
141 Connection between 6 and 1
142 Connection between 6 and 2
143 Connection between 6 and 3
144 Connection between 6 and 7 145 Connection between 6 and 11
146 connection between 6 and 8
F counterforce (Force Feedback)
(1) to (7) process steps in the flowchart