EP0663132A1

EP0663132A1 - Method and apparatus for controlling a camera

Info

Publication number: EP0663132A1
Application number: EP93921877A
Authority: EP
Inventors: Peter Sander; Ralf Steinmetz
Original assignee: International Business Machines Corp
Current assignee: International Business Machines Corp
Priority date: 1992-10-02
Filing date: 1993-09-29
Publication date: 1995-07-19
Also published as: DE4233137C1; JPH08504543A; WO1994008424A1

Abstract

A method is described by which a camera (10) can be controlled easily via a computer (20) or a computer network (60). The movement of the camera can be angular, translational, or a combination of these. The particular improvement lies in the fact that the camera movement can be actuated by selecting points on the display screen of the computer, using a cursor (24, 92), which is depicted on the screen in addition to the image generated by the camera, positionable with a mouse (26).

Description

- 1 -

D E S C R I P T I O N

METHOD AND APPARATUS FOR CONTROLLING A CAMERA

The invention relates to a method and an apparatus for controlling a camera, for example, a video camera, as stated by the independent part of Claim 1.

Apparatus for controlling cameras, particularly video cameras, which are suited to transmitting continuously images of a specified process to a monitor, have, in and of itself, been disclosed in various embodiments. Particularly for monitoring purposes, the cameras are panned according to a specified sequencing scheme. For certain applications - for example, monitoring of processes in factory halls - actions by the observer with respect to the camera direction, for example, to interrupt the pan function, are required to ensure a suitable camera angle.

User-operated camera controls in various embodiments have also been disclosed. In particular, implementations exist with buttons for panning up, down, right, and left, as well as combinations thereof (e.g., left and up simultaneously) . The camera then pans in the prescribed direction until the button is released, or, in an alternative embodiment in which the appropriate button must be actuated only once, until the appropriate button or a stop button is actuated. The button function is occasionally replaced with a joystick, with which the desired direction (indicated by the direction of joystick movement) and in part also the desired panning speed - based on the force of the joystick movement - are selected. The panning direction is selected either by successively executed panning movements (e.g., up right and then up for one direction, corresponding to the display of the time at 1:30 or the compass direction NW) or by a corresponding position of the joystick.

Embodiments have also been disclosed in which the monitoring and operating functions are coupled to a computer. In this case, the camera image is previously digitized - preferably in the camera itself - and fed to the computer. The computer monitor screen or a portion thereof then serves as a monitor for the camera. In this embodiment, the previously mentioned buttons can be keys designated by the computer software or the operator.

A further disclosed embodiment employing a computer pans the camera using scroll bars displayed on the monitor and activated by a positioning device, e.g., a so-called mouse or a touch-sensitive screen. With this control method too, issuing commands are possible only in a predetermined direction, usually limited to the four points of the compass and sup plemented by at most one intermediate direction (corresponding to the compass point NW, for example) .

All of these methods have the disadvantage that a data link between the entry station for the user commands and the actuating camera control must be maintained for the entire control process (so-called on-line operation), or the delay-free transmission of a command to stop camera movement must be ensured to avoid panning beyond the target.

Furthermore, the disclosed methods have the disadvantage that entry of the control commands is complicated and frequently successful only after several attempts. The object of the invention is to provide a method and an apparatus for controlling a camera, by which simpler pan¬ ning of the camera is possible.

The object is met by the method described in Claim 1 and with the apparatus proposed in Claim 5. The results of the method described in Claim 1 are first that, after entry of the new target position (target selection) by the signal generated by the signal means, all data for panning the camera are available in the computer, which is possibly a considerable distance away from the camera, and in an abstract sense can be passed to an actuator system associated with the camera. This transmission is in no way time-critical, that is, a transmission delay cannot influence the new camera position.

The method described in Claim 2 employs pointer- controlled selection of a movement direction. On the other hand, as disclosed by older methods, a stop signal must be generated to enable termination of camera movement when desired by the user. This is realized by requiring transmission of at least the stop signal to be instantaneous. This method variant is thus of advantage particularly for local applications in which the new camera position can be selected in a simple manner, at least with respect to the direction in which the camera is to be panned.

In Claim 3, the method is combined with the useful embodiment feature by which the speed can be easily selected.

The embodiment of the inventive method, in which the screen center represents the reference point, is advantageous. The apparatus of Claim 5 is, in and of itself, suited to implementing the basic concepts of the invention. The means for determining the direction and the distance the camera is to be panned are, according to the invention, capable of activating the desired camera control function by output of a single command.

The benefit of the invention will be explained using a sample application. In aircraft operation, ground inspections are necessary between two flights, particularly in unusual situations. An inspection method is particularly favorable when it is carried out with a pan-able video camera, whereby the camera is installed at the aircraft location while evaluation occurs in a central maintenance office, e.g., at the airline's headquarters. The two locations can be separated from one another by any distance. The focal point of the inspection is often a specific detail on the aircraft, e.g., the landing gear. The headquarters operator therefore knows which objects he wants to inspect and in which sequence.

Over large distances, the use of a camera-linked computer in addition to the computer at the user's location is beneficial (Claim 6), whereby the link between the computers is realized advantageously with a computer network, possibly public (Claim 7).

The inventive concept can be implemented even if panning comprises only one angle (e.g., only azimuth). The most advantageous embodiment, however, is with panning through two angles (Claim 8) which are orthogonal to each other and represent elevation and direction (Claim 9).

Especially useful among the various possible implementa¬ tions of the means for camera movement are commercial, digitally controlled stepping motors, since this approach requires little technical expense (Claim 10). If an autonomous stabilization is necessary, however, an independent actuator system can be used in place of the stepping motors, preferably digitally controlled by the computer, e.g., a gyro-stabilized platform.

For other applications, the camera control can also be used to advantage for two-dimensional translational movements (Claims 11 and 12).

Especially favorable is the embodiment of the camera control to carry out not only panning but also translational movements. In this case, the means for entry of commands at the monitor and for determining the control-related parameters for transmission to the actuators, which in the end carry out the camera panning, must be implemented twice.

The invention-related components, devices, and means or process steps previously described, claimed, or described in the following embodiments are not subject to any par¬ ticular exceptions with respect to implementation, design, selection, technical architecture, or methodology, so that the known selection criteria in each respective application area can be applied without restriction.

In particular, an implementation is not limited to a pro¬ gram or software package in a freely-programmable computer. Rather, it is left to the individual application whether an electrical circuit, a special-pur¬ pose computer, or a pro gram in a freely-programmable digital data processing system is used. Further details, characteristics, and advantages of the subject of the invention can be derived from the following description of the related drawings, in which - for purposes of example - an apparatus in accordance with the invention and the inventive method are represented.

The drawings are as follows:

Fig. 1 shows a block diagram of an apparatus in which two computers are linked by a computer network.

Fig. 2 shows a block diagram of an apparatus according to the invention with a single computer.

Fig. 3 shows a sequencing scheme of the inventive method.

Fig. 4. shows a block schematic of an apparatus for translational control in accordance with the invention.

Fig. 5 shows an apparatus with two pointers depicted on the screen, in accordance with the invention.

With the apparatus designated in its entirety as 100 in Figure 1, images of the objects to be depicted are captured with the camera 10 and appear at a data output 16 of the camera 10. In the embodiment, this data is passed to an input interface 52 of the second computer 50. The output signal of the camera 10 is an RGB (red, green, blue) video signal. This second computer 50 passes the data to a computer network 60, which forwards it to the computer 20 for depiction on its monitor 22. In the embodiment, the stated process is conducted with a frame refresh rate of 30 Hz, so that a continuous image sequence is shown on the monitor 22 of the computer 20.

In addition, the monitor 22 of the computer 20 displays a pointer (cursor) 24 in the shape of an arrow, which can be positioned over the entire screen independently of the screen contents. The positioning device 26 in the embodi¬ ment is a so-called computer mouse. The pointer 24 can be moved by displacing the mouse. In the embodiment, upon a double-click of the left mouse button (signal means 28) within 300 milliseconds (signal means 28), the current position of the pointer 24 is used to define the new image center 82 of the camera 10, and the camera 10 is to be panned until this target is reached.

In the embodiment, the image center 80 is the method's standard reference point, but this can be assigned to any other desired point on the screen.

In the computer 20, angle-determining means 74 are imple¬ mented with a computer program, which determines the pan¬ ning angle of the camera 10. The computation uses a key- activated zoom setting of the camera 10, possible in the embodiment and realizable in a known manner, and image segment magnification (pseudo-zoom) , which also is activated by keys (image magnify and reduce). Computation of the angle in the embodiment is thus carried out using the formula

vert, distance of pointer 24 from center 80 A 1 elevation angle = * - * - vert, distance from screen edge to center 80 Z P horiz. distance of pointer 24 from center 80 A 1 azimuth angle = * - * - horiz. distance from screen edge to center 80 Z P where A is the half aperture of the camera 10, Z is the current zoom magnification of the camera 10, and P is the current image segment magnification on the monitor 22.

These two angles are in turn passed via the computer net¬ work 60 to the second computer 50, from which a number of pulses, proportional to each respective angle, are applied to stepping motors 34 and 36 of an actuator system 30 for panning the camera 10.

Another embodiment with only one computer 20, which is connected to the camera 10 as well as to the monitor 22 and the positioning device 26, is depicted in its entirety as 200 in Figure 2.

In an alternative embodiment of the inventive apparatus, it is not the angle, computed as previously noted, which is transmitted to the control device of the camera 10 but rather a time interval, in which the camera 10 is to be moved in the selected direction at a prescribed speed. This time interval is, at constant speed, proportional to the panning angle. In this alternative embodiment, however, the angle through which the camera is to be panned is divided into a series of time intervals, in which the camera 10 is moved, starting at a slow speed and increasing stepwise to the maximum speed permitted by the camera panning device, and then, for terminating the move operation and again with stepwise decreasing speed, stopped from the slowest speed level. The angle through which the camera is to be panned is thus divided into a series of time intervals, allowing smooth acceleration of the camera in the panning direction. In the device representing the means 74 for determining data, correction data is included to compensate for nonlin- earity of the acceleration and braking processes. This correction data has been determined experimentally. In a further embodiment, depicted in its entirety as 300 in Figure 4, the camera 10 is not panned but rather moved translationally on a rail 84. The rail 84 itself is in turn attached to a rail pair 86, such that the rail 84 and thus the camera 10 can be moved orthogonal to the movement direction first mentioned. This embodiment is especially suited to applications in which the distance of the camera 10 from an observation plane (reference plane) is known. The location to which the camera 10 is to be moved, determined by positioning and subsequently fixing the pointer 24 by the positioning device 26 and the signal means 28, corresponds to the tangent of the angle, computed as in the previously discussed embodiment, multiplied by the distance from the camera to the reference plane.

A further embodiment, in which the possibilities of both previously mentioned embodiments are combined, comprises means 90 for translational movement, represented by the combination of the rails 84 and 86, as well as the panning device. On the monitor 22 of the computer 20, a second pointer 92 is shown, in addition to the pointer 24 and the image of the camera 10, which differs symbolically from the pointer 24 (Figure 5). In this embodiment, the second pointer 92 is moved using the same computer mouse as for the pointer 24, but the right button of the computer mouse, with which the means 26 for positioning the pointer 24 as well as the means 94 for positioning the second pointer 92 are implemented, is held depressed in order to move the second pointer. The means 96 for outputting a second control signal are implemented by double-clicking the right button of the computer mouse in an interval of up to 300 milliseconds. The translational movement is therefore actuated with means similar to those for panning. The inventive apparatus described in this embodiment is especially suited for monitoring factory halls from above, whereby the ceiling of the hall represents and comprises the two translational directions, and the camera can observe the objects in the hall by translational positioning to a desired observation point and subsequent panning to the object. The reference plane is either a plane of machinery operations in the hall or the floor itself. Selection of the reference plane is possible by keyboard entry using the computer 20 (e.g., input of a number). A further implementation of this embodiment is possible and useful for the previously mentioned application of monitoring the condition of an aircraft by a service manager, where the service manager must observe individual objects (e.g., the landing gear) from a remote location. In this embodiment, however, this should be viewable from various angles. For this purpose, the two translational movement directions of the camera 10 are selected such that a vertical observation wall, on which the camera 10 can be moved, is positioned in front of the aircraft. The reference plane is selected as a cross- section through the middle of the aircraft, parallel to the observation wall. Control errors of the camera 10, caused by the observed object's not lying exactly on the reference plane, are insignificant in this embodiment, since during panning this same observed object (e.g., the landing gear) can be positioned to the image center of the camera 10. This approach is therefore especially advantageous in the mentioned embodiment, since the selected object in the selected observation direction can be compared with an archived reference image available to the service manager, and any damage and necessary repair work can thus be determined with high accuracy.

The basic principle of the inventive method is depicted in Figure 3 as a sequence diagram.

Claims

C L A I M S

1. Method for controlling a camera (10), comprising the following steps

the image generated by the camera (10) is depicted on the display screen of a computer (20);

a new reference point of the camera (10) is selected using a pointer (24) which is depicted on the screen in addition to the image and position- able with a positioning device (26); and

the distance of the pointer position from the old reference point on the screen represents the amount by which the camera should be panned.

2. Method for controlling a camera (10), comprising the following steps:

the image generated by the camera is depicted on a display screen of a computer; and

the direction in which the camera (10) is to be moved is selected using a pointer (24) which is depicted on the screen in addition to the image and positionable with a positioning device (26) corresponding to the direction of the pointer with respect to a reference point.

Method according to Claim 2, characterized in that the distance of the position of the positioning device from the reference point on the screen represents the speed of the camera movement.

Method according to any one of the Claims 1 to 3, characterized in that the reference point is the image center.

Apparatus for controlling a camera (10) with

at least one computer (20) with a monitor (22) for depicting images captured by the camera;

means for depicting a pointer (24) on the monitor (22) in addition to each image being displayed;

means (26) for positioning the pointer (24) and for generating a signal for terminating the positioning operation (28);

characterized i n t h a t the apparatus comprises means (74) by which the data required for controlling the camera is determined based on the distance of the pointer (24) from a reference point after fixing the pointer (24) by the means (28) for generating a signal for terminating the positioning operation.

Apparatus according to Claim 5, characterized in that at least two interconnected computers (20,50) are employed and that one computer (50) receives the images of camera (10) while the other computer (20) comprises the monitor (22) for depicting the images captured by camera (10), the means for depicting the pointer (24), the means for positioning the pointer (26) and for generating the signal for terminating the positioning operation (28).

7. Apparatus according to Claim 6, characterized in that the interconnection of the computers (20,50) is implemented by a computer network (60).

8. Apparatus according to any one of the Claims 5 to 7, characterized in that the camera (10) comprises means (30) for panning in two angular directions.

9. Apparatus according to Claim 8, characterized in that the means (30) for panning the camera pan the camera (10) independently in an elevation angle (70) and an azimuth angle (72).

10. Apparatus according to any one of the Claims 5 to 9, characterized in that the means (30) for panning the camera (10) comprise stepping motors (34,36) actuated by the computer (20,50) connected to the camera (10) .

11. Apparatus according to any one of the Claims 5 to 7, characterized in that the camera (10) is connected to the means (90) for panning in at least one translational direction.

12. Apparatus according to Claim 11, characterized in that the camera (10) is connected to the means (90), which move it in two translational directions substantially orthogonal to one another.

13. Apparatus according to any one of the Claims 5 to 9, characterized in that

the apparatus comprises means by which the camera can be moved in its direction as well as in its translational position; the apparatus comprises means for depicting a second pointer (92) on the monitor (22) in addition to the pointer (24) and to each image being displayed;

the apparatus comprises means for positioning the second pointer (92) and for generating a second signal (94) for terminating the positioning of the second pointer (92); and

the apparatus comprises, in addition to the means (74) for determining the control signal, means (94) for determining a second control signal, by which the data required for controlling the camera (10) is determined based on the distance of the second pointer (92) from a reference point after fixing of the second pointer (92) by the means (94) for generating a signal for terminat¬ ing the positioning of the second pointer (92) .

14. Method for controlling a camera, comprising the fol¬ lowing steps: - 15 -

an image generated by the camera is depicted on a display screen of a computer;

a pointer, positionable using a positioning device, is generated on the screen;

the camera is centered on the position of the pointer in response to a signal which can be activated by an actuation device.