WO2009156745A1

WO2009156745A1 - Methods of stabilising recorded or projected images in optical apparatus

Info

Publication number: WO2009156745A1
Application number: PCT/GB2009/001622
Authority: WO
Inventors: Richard Arthur Lindsay
Original assignee: The Vitec Group Plc
Priority date: 2008-06-26
Filing date: 2009-06-26
Publication date: 2009-12-30
Also published as: GB0811740D0

Abstract

The disclosure relates to a method of stabilising a recorded or projected image in an optical apparatus supported for damped movement in pan and tilt axes. The method comprises, for each axis, monitoring actual movement of the camera using a dynamic sensor; monitoring intended movement input by the operator across the damper using an opto-electronic transducer and determining at time intervals the difference between the actual and intended apparatus movement which represents unwanted movement and applying a correction to counter the unwanted movement. Movement due to-electronic drift in the dynamic sensor and opto-electronic transducers is determined from the residue of the aggregate of said differences between actual and intended movement over a period of time and is used to predict a value for electronic drift for the next difference between actual and intended movement which is taken into account in the difference.

Description

METHODS OF STABILISING RECORDED OR PROJECTED IMAGES IN OPTICAL APPARATUS

This invention relates to stabilising recorded or projected images and is particularly applicable to TV or video cameras, light directing devices and sighting devices.

In WO 00/39498 we describe and illustrate an image stabilisation system in a mounting head for an optical apparatus such as a TV/video camera providing damped pan and tilt movement. A dynamic sensor in the form of an electronic gyroscope is mounted on the head to determine actual movement of the head and optoelectronic transducers in the form of encoders are disposed across the dampers for the pan and tilt movement of the head to determine intended movement in each axis. Extraneous forces on the camera/mounting can cause unwanted movement to the camera. The difference between the actual and intended movements of the head representing unwanted movement is used to correct the position of the head, the optical axis or the image system for recording an image in the camera.

Dynamic sensors such as electronic gyroscopes, optoelectronic encoders and other electronic systems including signal processors are subject to drift in their output which can result in a false measurement of unwanted movement when used in an image stabilisation system as outlined above. An object of the present invention is to minimise the effect of such drift in the correction of the image .

This invention provides a method of stabilising a recorded or projected image in an optical apparatus supported for damped movement in pan and tilt axes, the method comprising, for each axis, monitoring actual movement of the camera using a dynamic sensor; monitoring intended movement input by the operator across the damper using a displacement transducer, determining at time intervals the difference between the actual and intended apparatus movement which represents unwanted movement, applying a correction to counter the unwanted movement, identifying electronic drift in the dynamic sensor and displacement transducers as the residue of the aggregate of said differences between actual and intended movement over a period of time and compensating in the difference between actual and intended movement to be next applied for the value predicted for electronic drift in the next difference. For example a function may be derived which represents the low frequency variation between the actual and intended movement -of the apparatus over time. That -function can then be used to predict compensation required for future electronic drift.

By way of example the function derived to represent said low frequency variation in the difference between the actual and intended movement may provide a curve fit (e.g. least squares fit) or may be a rolling average which matches said differences over time. In a further method said differences over time may be filtered to extract low frequency data from which future drift can be predicted.

The invention also provides a method of stabilising an image generated by a camera mounted for pivotal movement with damping in one or more axes, the method comprising using a dynamic sensor to determine actual camera movement in each axis and displacement transducers to determine intended movement measured across the damper in the or each axis to provide a measure of unwanted movement being the difference between actual and intended movement in each axis for correction of the camera image, and deriving compensation for the actual and intended movement of the camera in the or each axis as measured by said sensor/transducers to allow for drift in said measurement in accordance with any of the above methods .

For example the dynamic sensor for measuring actual camera movement may comprise electronic gyroscope means and the displacement transducers for measuring intended movement across the damper in the or each axis may comprise encoders such as optoelectronic encoders.

Low frequency changes in the differences between actual and intended movement can also be produced by wind up in che mounting for the apparatus (due to the elasticity of the components of the mounting) which is a function of change of torque ^: applied to the apparatus- measured by acceleration determined across the damping in the head. During acceleration these differences are assumed to be constant for the purpose of determining drift compensation. In any of the above methods the image may be corrected by adjusting the camera optics in relation to the camera recording medium, adjusting the camera image recording medium in relation to the optics or by bodily movement of the camera or by subsequent processing of the image.

The following is a description of some specific embodiments of the invention, reference being made to the accompanying drawings in which:

Figure 1 is a diagrammatic side view of a TV/video camera supported on a pan and tilt head and incorporating a control system for determining unwanted camera movement and generating a corresponding correction; Figure 2 is a plan view of the pan and tilt head of Figure 1 with the camera removed;

Figure 3 is a perspective view of a CCD array of the camera for creating electronic data corresponding to the image received, the CCD array being adjustable with respect to the optical axis of the camera to correct the image for said unwanted movement for image correction;

Figure 4 is an illustration of an alternative CCD array with a control system for adjusting the effective part of the array with respect to the optical axis on which an image is incident to correct the image for said unwanted movement;

Figure 5 is a graph of unwanted movement as measured in the pan or tilt axis plotted against time; and

Figure 6 is a graph showing the effect of tripod wind on the difference between actual and intended movement.

Referring firstly to Figures 1 and 2 of the drawings, there is shown a TV/video camera indicated generally at 10 having an optical axis 11, a lens system 12 with a lens axis 13 and an image plane 14 in which a CCD array 15 is mounted for converting the image focused by the lens onto the array into electronic data in known manner. The lens system has an adjustment arrangement 16 for adjusting the lens system . transversely of its axis 13 to vary the location where the image is focused on the CCD array. This adjustment is used to correct for unwanted or unintended movement of the camera during recording. Alternatively the CCD array may have an adjustable mounting which will be described later for adjusting the CCD array in relation to the lens axis as another way of correcting for unwanted camera movement.

The camera is mounted on a pan and tilt head indicated generally at 20 located on a tripod 21 having a head 22 and three divergent legs 23. The pan and tilt head has a platform 24 on which the camera is located and a base 25 which is mounted on the head 22 of the tripod.

The pan and tilt head has a body 26 mounted on the base 25 for rotation about a vertical axis indicated at 27 for pan movement of the camera. Rotation of the body about the base is controlled by a multi-plate annular damper 28 of the construction described in our EP-B-0850382 and an optoelectronic transducer in the form of an encoder 29 is coupled between the base and body across the damper to measure rotation of the body with respect to the base as input by the camera operator.

The platform 24 of the pan and tilt head is mounted on " the body 26 for rotation about a horizontal axis 30 to provide tilt movement for the camera. Again an annular multi-plate- damper 31 is provided for damping ti-lt: movement of the platform with respect to the body and an optoelectronic encoder 32 measures rotary movement of the platform 24 with respect to the body 26 across the damper. A two axis electronic gyroscope 40 such as a MEMS

(Micro-electronic Mechanical System) gyroscope or a fibre optic or laser gyroscope is mounted on the underside of the platform 11 to measure total movement of the camera in the pan and tilt axes whether it be intended movement imparted by the camera operator or unintended movement such as vibration transmitted through the ground into the tripod or wind action or other extraneous forces acting on the camera and causing inadvertent movement of the camera.

The pan and tilt encoders and the gyroscope are coupled to a microprocessor 45 in the camera which is programmed to calculate unwanted movement of a camera in each axis by taking from the actual movement of the camera in each axis - S -

as measured by the electronic gyroscope the movement as measured by the pan and tilt encoders. It will be appreciated that by measuring the movement in the mounting about the pan and tilt axes across the dampers, a representative value for the intended movement as applied to the camera by the camera operator is measured.

The difference between the actual and intended movement in each axis as computed by the microprocessor represents unwanted movement and signals are provided from the microprocessor to the lens adjustment system to provide an adjustment in the lens axis horizontally and/or vertically to compensate for the unwanted movement of the camera in the pan and tilt axes.

'The microprocessor is programmed to calculate repeatedly the unwanted movement. For example calculation may take place every -millisecond..

Figure 3 of the drawings shows a further development in which the CCD array 15 is movable horizontally and vertically by respective pairs of motor units 50, 51 acting on the sides and top and bottom of the array. The motor units are controlled by signals from the microprocessor. Moving the array laterally or vertically as required represents an alternative way of correcting the image as recorded by the camera to adjustment of the lens system. In a further arrangement shown in Figure 4, the microprocessor is coupled to a unit 55 connected to the array of photo diodes to control the effective area in operation at any time. By adjusting the location of the effective area being used to record an image laterally or vertically in the overall array, it is possible to correct the image as recorded in the array for unwanted camera movement . Further arrangements for correcting the unwanted camera movement are described in our WO 00/39498 which include moving the camera bodily. Other arrangements are described in our WO 2006/1175 in which the image and correction information from the optoelectronic transducers are supplied to a separate microprocessor which adjusts the image information to remove the unwanted movement as recorded in the image. This can be done immediately after the image has been recorded or the uncorrected image information and the correction information can be recorded separately and used to adjust the image only when the image is played back subsequently.

The above image stabilising systems rely on accurate measurement of absolute movement of the camera and relative pan and tilt rotation across the damper units. In reality electronic gyroscopes are subject to drift the^" consequence of which is that there will be inaccuracies in the correction of the image.

Figure 5 of the drawings shows a plot of the difference in velocity measured by the gyroscope and the optoelectronic encoder for the pan or tilt axis plotted against time. Over a period of time, without gyroscope drift, the difference in velocity between the gyroscope and the optoelectronic encoder averages to zero. Electronic drift is a low frequency trend in the difference in velocity derived from the gyroscope and optoelectronic encoder and can be extracted by means of a curve fit (e.g. least squares fit) , rolling average or by filtering as indicated earlier. The extracted trend is used to predict future values of drift at specific time intervals to provide compensation for drift in the correction to be applied in the system to eliminate in unwanted movement.

Signals for drift are deducted from the signal for unwanted movement (as measured by the gyroscope and optoelectronic encoder in the microprocessor) and the result applied to the end system, that is the CCD array or other arrangement for correcting the image recorded in the camera.

An additional low frequency effect which would cause errors in the drift estimate arises from tripod elastic deflection during panning and tilting, resulting from torque generated by the damping. Change in tripod deflection is a function of acceleration across the damping units as measured by the second differential of the displacement recorded by the optoelectronic encoders . To eliminate the effect of change of tripod deflection from the calculation of drift, -data collected during damper acceleration/deceleration is ignored by the microprocessor. During this period, difference in velocity measured by the gyroscope and encoder is assumed to remain constant (for the purposes of drift calculation only) .

Figure 6 of the drawings shows the difference between actual and intended velocity of the camera in pan or tilt movement plotted against time showing a typical initial acceleration, a constant velocity period and a final deceleration.

Claims

CLAIMS :

1. A method of stabilising a recorded or projected image in an optical apparatus supported for damped movement in pan and tilt axes, the method comprising, for each axis, monitoring actual movement of the camera using a dynamic sensor; monitoring intended movement input by the operator using a displacement transducer across the damper, determining at time intervals the difference between the actual and intended apparatus movement which represents unwanted movement, applying a correction to counter the unwanted movement, identifying electronic drift in the dynamic sensor and displacement transducers as the residue of the aggregate of said differences between actual and intended movement over a period of time and compensating in the difference between actual and intended movement- to be next applied for the value predicted for electronic drift in the next difference.

2. A method as claimed in claim 1, wherein corrections to the image are derived and applied continuously.

3. A method as claimed in claim 1 or claim 2, wherein a function is derived which represents the low frequency differences between the actual and intended movement of the apparatus, said function being used to predict the compensation for said electronic drift.

4. A method as claimed in claim 3, wherein the function derived to represent the low frequency change in said difference between the actual and intended movement is a curve fit (e.g. least squares fit), a rolling average or by filtering to extract low frequency data from which future drift can be predicted.

5. A method of stabilising an image generated by a camera mounted for pivotal movement with damping in one or more axes, the method comprising using a dynamic sensor to determine actual camera movement in each axis and a displacement transducer to determine intended movement measured across the damper in the or each axis to provide a measure of unwanted movement being the difference between actual and intended movement in each axis for correction of the camera image, and deriving corrections for the unwanted movement of the camera in the or each axis as measured by said sensors/transducers to allow for drift in said measurement in accordance with any of claims 1 to 4.

6. A method as claimed in claim 5, wherein the dynamic sensor for measuring actual camera movement comprises electronic gyroscope means and the displacement transducers for measuring intended movement across the damper in the or each axis comprise encoders.

7. A method as claimed in claim 5 or claim 6, wherein changes in the difference between actual and intended movement are ignored when acceleration across the damper is detected and the previously determined drift correction signal continues to be applied.

8. A method as claimed in any of the preceding claims, wherein the image is corrected by adjusting the camera optics in relation to the camera recording medium, adjusting - li ¬

the camera image recording medium in relation to the optics or by bodily movement of the camera.

9. A method as claimed in any of claims 1 to 4, wherein the optical apparatus is a TV/video camera.

10. A method as claimed in claim 9, wherein said compensating movement is applied to the camera, camera mounting, camera optics, image capture or to image playback.

11. A method as claimed in any of claims 1 to 8, wherein the optical apparatus is a light directing device or a sighting apparatus.