US20140267778A1

US20140267778A1 - Apparatuses and methods for controlling a gimbal and other displacement systems

Info

Publication number: US20140267778A1
Application number: US14/214,637
Authority: US
Inventors: Steve Webb; John Ellison; Tabb Firchau; David Bloomfield
Original assignee: FREEFLY SYSTEMS Inc
Current assignee: FREEFLY SYSTEMS Inc
Priority date: 2013-03-15
Filing date: 2014-03-14
Publication date: 2014-09-18
Also published as: US20140270744A1; US8861947B2; US8934023B2; CA2906997A1; WO2014140354A1; US20160028956A1; AU2014230009A1; WO2014140355A1; EP2974269B1; EP2974269A1; US20140267805A1; WO2014140357A1; AU2014230009B2; US20140270743A1; JP2016515348A; JP6334576B2; US20140267810A1; US10298846B2; US8938161B2; WO2014140356A1

Abstract

Apparatuses and methods for controlling a gimbal and other displacement systems are disclosed herein. In accordance with one or more embodiments of the invention, a pointing angle of a camera attached to a gimbal may be controlled based, at least in part, on one or more control signals provided by a controller. The control signals may be used to compensate for displacement of the camera, to add perceived displacement of the camera, to selectively align a pointing angle of the camera and/or to allow a pointing angle to be manually determined.

Description

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of Provisional Patent Application No. 61/792,878, filed on Mar. 15, 2013, which is hereby incorporated by reference herein in its entirety.

TECHNICAL FIELD

Embodiments of the present invention relate generally to control systems and in particular to control of a gimbal and other displacement systems.

BACKGROUND

Generally, technology trends toward smaller and more efficient devices and sensor technology is not an exception. Sensors, such as those directed to measuring motion, temperature, voltage, pressure, and other quantifiable metrics have become both improved with respect to accuracy and size.
A number of industries, however, such as the motion picture industry, have not yet appreciated the advantages provided by these improvements. For example, gimbals used to orient cameras have remained cumbersome, typically requiring multiple operators for use and presenting difficulties in steady shooting, particularly when the gimbal is carried and/or steered by an operator. Moreover, difficulties are further introduced in circumstances when displacement of a gimbal is intentional (e.g., during a chase scene or an earthquake scene) but reliable control of the orientation of the gimbal is still desired.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram of an apparatus according to an embodiment of the present invention.

FIG. 2 is a schematic block diagram of an apparatus according to an embodiment of the present invention.

FIG. 3 is a schematic block diagram of an apparatus according to an embodiment of the present invention.

FIG. 4 is a schematic block diagram of an apparatus according to an embodiment of the present invention.

FIG. 5 illustrates the effect of translation with constant pointing angle.

FIG. 6 illustrates a vertical translation showing rate of change of pointing direction.

FIG. 7 is a diagram of a gimbal tilt drive control with motor and IMU mounted on a camera.

FIG. 8 is a diagram of a gimbal tilt drive control showing translation compensation of gimbal tilt angle.

FIG. 9 illustrates how translation velocity is reduced for an elevated view-point.

FIG. 10 illustrates direct inference of tilt angle from height and distance.

FIGS. 11A and 11B illustrate normal gimbal stabilization and an artificially implanted rotation to simulate camera operator acceleration and rotation.

FIG. 12 is a diagram of a gimbal controller showing a scheme for acceleration noise addition to tilt pointing angle.

FIG. 13 is a graph illustrating high-pass filtering of an acceleration measurement.

FIG. 14 is a diagram of a gimbal controller showing an introduction of noise optionally from: a pre-recording, remote mounted IMU, artificial generator, or joint angle.

FIG. 15 illustrates two operators filming a scene showing coordination complexity.

FIG. 16 is a diagram of a gimbal controller showing a joint angle zeroing control loop.

FIG. 17 is a diagram of a gimbal controller showing steerage of closed loop tilt angle by an external torque producing force.

FIGS. 18A and 18B respectively illustrate a conventional suspended camera with translation errors and a solution using a feedback control loop.

FIG. 19 is a diagram of a camera height stabilization controller suspended from a support beam.

FIG. 20 is a diagram of a camera height stabilization controller suspended from a support beam showing movement of command height by external force and force threshold detection.

FIG. 21 is a diagram of a camera height stabilization controller suspended from a support beam showing a vario control mode allowing forced height changes by operator pulling down or lifting up the camera.

DETAILED DISCLOSURE

Apparatuses and methods for controlling a gimbal and other displacement systems are disclosed herein. Certain details are set forth below to provide a sufficient understanding of embodiments of the invention. However, it will be clear to one having skill in the art that embodiments of the invention may be practiced without these particular details. Moreover, the particular embodiments of the present invention described herein are provided by way of example and should not be used to limit the scope of the invention to these particular embodiments. In other instances, well-known circuits, control signals, timing protocols, and software operations have not been shown in detail in order to avoid unnecessarily obscuring the invention.
FIG. 1 is a schematic block diagram of an apparatus 100 according to an embodiment of the invention. The apparatus 100 may include an inertial measurement unit (IMU) 110, a controller 120, a driver 130, and a gimbal 140. A camera CAM is shown in FIG. 1 as being attached to the gimbal 140. The camera CAM may comprise any camera known in the art, now or in the future.
The gimbal 140 may be used to control the orientation (e.g., tilt, pan, and/or roll) of the camera CAM. One or more components of the apparatus 100 may be physically or electrically coupled to the gimbal 140 or the camera CAM including, but not limited to, the controller 120 and the IMU 110. In one embodiment for example, one or more components of the apparatus 100 may be coupled to the camera CAM such that the coupled components and the camera CAM have fixed relative positions even when the camera CAM is displaced, for instance, by a camera operator.
The gimbal 140 may include a motor 150 that may be configured to control the gimbal 140 such that the camera CAM is oriented in a particular manner. The motor 150 as shown in FIG. 1 may be representative of one or more motors of the gimbal 140. The motor 150 may comprise a direct drive motor or may comprise a geared or belt reduction drive. In some examples, the motor 150 may comprise a plurality of servo motors configured to control the gimbal 140 such that tilt, roll, and/or pan of the camera CAM may be controlled. Moreover, in at least one embodiment, the motor 150 may be configured to provide one or more signals indicative of an angle of displacement (displacement angle). Because the motor 150 may control the orientation of the gimbal 140, the motor 150 may further be configured to determine a displacement angle for one or more of the gimbal axes and provide signals indicative of the displacement angles. By way of example, the motor 150 may determine a vertical displacement angle, a horizontal displacement angle, and/or a rotational displacement angle and provide signals indicative of one or more of the determined displacement angles, for instance, to the controller 120.
The IMU 110 may comprise any IMU known in the art, now or in the future, and may be configured to detect changes in orientation and/or displacement of the camera CAM. The IMU 110 may be attached to the camera CAM and/or the gimbal 150. By detecting changes in orientation and/or displacement of the camera CAM, the IMU 110 also detects changes in orientation and/or displacement of the gimbal 140. For example, the IMU 110 may comprise one or more sensors physically and/or electrically coupled to the camera CAM including, but not limited to, a global positioning system (GPS) sensor, an accelerometer (e.g., 3-axis accelerometer), a gyroscope (e.g., 3-axis gyroscope), a compass (e.g., 3-axis compass), and a barometer. Accordingly, the IMU 110 may be configured to measure orientation of the camera CAM (and thereby the gimbal 140) in three dimensions and/or velocity of displacement of the camera CAM on one or more axes, and may further may be configured to provide one or more measurement signals indicating the orientation, velocity of the displacement, and/or direction of the displacement. In at least one embodiment, the IMU 110 may provide measurement signals at a rate of 160 Hz, though it will be appreciated by those having ordinary skill in the art that any frequency may be used.
The controller 120 may be coupled to the IMU 110 and configured to receive the measurement signals. Based, at least in part, on the measurement signals, the controller 110 may provide (e.g., generate) one or more control signals. The control signals accordingly may be provided to the driver 130, which may in turn provide the control signals to the motor 150 to control the orientation of the gimbal, and thereby, the orientation of the camera CAM that is attached to the gimbal 140.
As will be explained, the control signals provided by the controller 120 may be used to adjust an orientation of the camera CAM by causing the gimbal 140 to adjust a pointing angle of the camera CAM. The controller 120 may, for instance, be configured to adjust the pointing angle of the camera CAM based, at least in part, on one or more measurement signals provided by the IMU 110. For example, the pointing angle of the camera CAM may be measured by the IMU 110 and the controller 120 may be configured to compare the pointing angle (e.g., attitude) of the camera CAM with a desired pointing angle (as indicated by a command attitude). The controller 120 may determine the difference between the pointing angle of the camera CAM and the desired pointing angle, and provide control signals through the driver 130 to the gimbal 140 (e.g., the motor 150) to adjust the orientation of the gimbal, and as a result, to adjust the pointing angle of the camera CAM. By way of example, the controller 120 may provide control signals to match the pointing angle of the camera CAM to the desired pointing angle. In another example, the velocity of displacement may determine how quickly the pointing angle of the camera CAM may be adjusted.
In some examples, the controller 120 may be configured to operate in a closed-loop control circuit to iteratively determine a new pointing angle for the camera CAM using a current pointing angle of the camera CAM. The controller 120 may, for example, employ a proportional-integral-differential (PID) control loop, and may update the current pointing angle at a fixed frequency, such as 400 Hz, although other frequencies may be used. In this manner, the pointing angle of the camera CAM may be repeatedly updated (e.g., by repeatedly updating the orientation of the gimbal 140) using one or more measurement signals provided by the IMU 110 in accordance with examples described herein. As will be described in further detail below, the pointing angle of the camera CAM may be adjusted to compensate for vertical and/or horizontal displacement of the camera CAM, to add perceived displacement of the camera CAM, to selectively align a pointing angle to one or more displacement angles of the gimbal 140 and/or to allow a pointing angle to be manually determined.
In an example operation of the apparatus 100, the gimbal 140 (and thereby the camera CAM) may be displaced in one or more directions. The IMU 110 may sense the displacement and provide measurement signals to the controller 110 indicating the same. Based, at least in part on, on the measurement signals, the controller 120 may provide control signals to the gimbal (e.g., the motor 150) to adjust a pointing angle of the camera CAM to compensate for the displacement of the camera CAM.
In some instances, a camera operator may use the camera CAM to film a particular subject (e.g., actor) and it may be desirable to keep the camera CAM directed toward the subject despite displacement of the camera CAM. Accordingly, the controller 120 may be configured to adjust the orientation of the gimbal 140 in order to adjust the pointing angle of the camera CAM such that the camera CAM stays directed at the subject.
In at least one embodiment, adjusting the orientation of the gimbal 140 in this manner may include determining a pointing angular rate correction value. For example, in instances in which the pointing angle is nominally horizontal, the angular rate for adjusting the pointing angle may be modeled by the following equation:
$ω = \frac{- V}{X}$
Where ω represents an angular rate, V represents a velocity of displacement, and X represents the horizontal distance between the subject and the camera CAM. In at least one embodiment, X may be determined using a range sensor (not shown in FIG. 1), such as an ultrasonic, laser, or infrared sensor. The range sensor may, for instance, be located on the front of the camera CAM and/or aimed at the center of field of view of the camera CAM. The range sensor may in some embodiments be mounted on the gimbal 140. The range sensor may be configured to provide a signal (e.g., digital or analog) to at least one of the IMU 110 or the controller 120 indicating the distance X between the camera CAM and a subject.
As can be seen, the angular rate is assigned a negative value of the velocity V divided by the distance X. This may ensure, for instance, that the pointing angle of the camera CAM is adjusted in a direction opposite the displacement of the camera CAM. In at least one embodiment, angular rates may be determined in both horizontal and vertical directions.
In instances in which the pointing angle is not nominally horizontal, the controller 110 may be configured to determine the pointing angle using a different approach. For example, the controller 110 may be configured to determine a tangential translation velocity in accordance with the following equation:
Vtangential=cos θ*V
where Vtangential represents the tangential translation velocity, θ represents the current pointing angle of the camera CAM, and V represents the velocity of displacement. Once the tangential translation velocity has been determined, the angular rate may be determined with the previously described formula using the tangential translation velocity in lieu of the velocity of the displacement in accordance with the following equation:
$ω = \frac{- Vtangential}{X}$
It will be appreciated by those having skill in the art that for horizontal displacement of the camera CAM, Vtangential may be equal to V.
As described, the controller 110 may update the orientation of the gimbal 140, and thus the pointing angle of the camera CAM, at a particular rate (e.g., 400 Hz) and may update the pointing angle of the camera CAM according to a calculated angular velocity. In at least one embodiment, the controller 110 may update the pointing angle in accordance with the following equation:
θ_t+dt=θ_t+ω_t
where θ_t+dtrepresents an updated pointing angle, θ_trepresents the current pointing angle, and _ffitrepresents the current angular rate. By way of example, the controller 110 may update the pointing angle of the camera CAM at a rate of 400 Hz and may determine the angular rate to be 1°/s. The controller 120 may update the pointing angle of the camera CAM using a rate 0.0025° per cycle. Accordingly, for a current angle of 25°, the updated angle may comprise 25.0025°.
In some cases, the velocity of displacement of the camera CAM may not be linear. Thus, in some embodiments, the controller 110 may be configured to determine the pointing angle using trapezoidal integration in accordance with the following equation:
$θ_{t + dt} = θ_{t} + \frac{(ω_{t} + ω_{t + dt})}{2}$
where θ_t+dtrepresents an updated pointing angle, θ_trepresents the current pointing angle, wt represents the current angular rate and ω_t+dtrepresents the previous angular rate.
In some instances, measurements by the IMU 110 may vary slightly due to drift and/or noise. This may cause the controller 120 to improperly update the pointing angle of the camera CAM. Thus, in at least one embodiment, only displacements above one or more thresholds, such as a particular displacement threshold and/or a velocity threshold, may cause the controller 110 to update the pointing angle of the camera CAM. By way of example, the controller 110 may be configured to update the pointing angle only when displacement of the camera CAM exceeds a rate of 5 cm/s.
While distances have been described herein as being determined by a range sensor, m some embodiments, distances may be determined in other manners as well. For example, distances may be predetermined and/or stored in the controller 110. In another example, distances may be provided to the controller 120, for instance, via wired or wireless communications. Moreover, in at least one embodiment, distance may be determined based, at least in part, on lens focus adjustment performed by the camera CAM. For example, the camera CAM may be configured to focus on a subject, and focusing in this manner may allow one or more components of the apparatus 100, such as the controller 120 to determine a distance between the camera CAM and the subject. In yet another example, horizontal displacement may be detected by the IMU 110, for instance, based on GPS measurements and/or compass measurements, and based, at least in part, on the displacement, a distance may be determined and the pointing angle adjusted accordingly.
In some embodiments, a plurality of range sensors (e.g., 2 range sensors) may be used. While a first range sensor may be used to determine distance between the camera CAM and a subject as previously discussed, one or more additional range sensors may be used to determine a height of the camera CAM relative to one or more surfaces (e.g., a floor). Additionally or alternatively, the IMU 110 may determine height, for instance, using a barometric measurement. Once a height has been determined, a pointing angle of the camera CAM may be determined. This may for instance, be used in addition to or in lieu of other approaches for determining an angle described herein and may be determined in accordance with the following equation:
$θ = \sin^{- 1} \frac{D}{H}$
where θ represents the pointing angle, D represents the distance between the camera CAM and the subject, and H represents the measured height of the camera CAM.
In some examples, height measurements may be volatile due to obstacles, inclines, declines, and/or sensor variation. Accordingly, the controller 110 may be configured to employ an offset correction, for instance, each time a command attitude is provided to the controller 110. In at least one embodiment, an offset correction may be implemented in accordance with the following equation:
OC=H−D*sin TC
where OC represents an offset compensation, H represents a measured height, D represents a distance between the camera CAM and a subject, and TC represents the command attitude. Once the offset correction has been determined, the pointing angle may be determined in accordance with the following equation:
$θ = \sin^{- 1} \frac{D}{H - OC}$
While examples have been described herein with respect to determining a pointing angle m a vertical direction, it will be appreciated that described approaches for determining a pointing angle may be employed in determining a pointing angle in a horizontal direction as well.
A camera operator may desire to carry or otherwise displace the camera CAM while filming. As a result, acceleration forces and rotational movements may be applied to the camera CAM and other elements of the apparatus 100. By way of example, a camera operator may be walking and/or running while filming a subject. In accordance with examples described herein, the image recorded by the camera CAM may be smoothed due to adjustments made to the pointing angle of the camera CAM by the controller 110 to compensate for displacement. In some instances, however, it may be desirable to replicate a perspective, such as that of a person running or driving a car, while filming. Thus, in some examples, relatively small adjustments of the pointing angle may be made to mimic movement of a desired perspective, that is, to provide perceived displacement. By way of example, perceived displacement may be modeled in accordance with the following equation:
Displacement=tan dθ*D
where Displacement represents perceived displacement, dθ represents an angular displacement and D represents the distance between the camera CAM and the subject. By varying dθ, different perceived displacements may be achieved. Displacement may be used to adjust the pointing angle in other manners as well. For example, as will be described in further detail below, acceleration may be used to adjust the pointing angle.
FIG. 2 is a schematic block diagram of an apparatus 200 according to an embodiment of the invention. The apparatus 200 includes elements that have been previously described with respect to the apparatus 100 of FIG. 1. Those elements have been identified in FIG. 2 using the same reference numbers used in FIG. 1 and operation of the common elements is as previously described. Consequently, a detailed description of the operation of these elements will not be repeated in the interest of brevity.
The apparatus 200 may include a controller 122 that may be used to implement the controller 120 of FIG. 1. The controller 122 may include a scalar 170, a summer 175, and a control logic 180. The scaler 170 may be configured to receive a filtered signal, for instance, from a filter 160, and a scale signal. The scaler 170 may further be configured to multiply the filtered signal by a factor indicated by the scale signal to provide a multiplied signal. The summer 175 may receive the multiplied signal and combine the multiplied signal with a received command attitude. The summed signal may be provided to the control logic 180, which may in turn adjust the orientation of the gimbal 140 (and therefore the pointing angle of the camera CAM) as described herein.
The filter 160 may be configured to receive at least one of the supplemental adjustment signal and the measurement signal from the multiplexer 190 and filter the received signals based, at least in part, on a bandwidth signal. The bandwidth signal may, for instance, indicate one or more frequencies by which the filter 160 should pass or block particular frequencies for the received signals. For example, based, at least in part, on the bandwidth signal, the filter 160 may operate as a high pass filter, a low pass filter, a band pass filter, a band block filter, or a combination thereof. In one embodiment and as will be explained in further detail, based, at least in part, on the bandwidth signal, the filter 160 may operate as a high pass filter such that elements of the measurement signals corresponding to acceleration due to gravity are filtered. While the filter 160 is shown independently of the controller 122, it will be appreciated that in some embodiments, the filter 160 may be included in the controller 122.
As described, the IMU 110 may include an accelerometer and accordingly may be configured to provide a measurement signal indicative of acceleration (e.g., vertical and/or horizontal acceleration). In one embodiment, the IMU 110 may be configured to measure acceleration during displacement of the camera CAM, and provide an associated measurement signal that may be used to adjust the pointing angle of the camera CAM. However, because acceleration measured by the IMU 110 includes acceleration due to gravity, the measurement signal may include acceleration other than transitory acceleration applied to the camera CAM as a result of displacement. Accordingly, the measurement signal may be provided to the filter 160, and the filter 160 may remove all acceleration except transitory acceleration, for instance, by operating as a high pass filter, and as described, the manner in which the measurement signal is filtered by the filter 160 may be based, at least in part, on the bandwidth signal. Subsequently, the filtered signal may be scaled by the scaler 170, combined with a command attitude at the summer 175, and provided to the control logic 180 for adjustment of the gimbal orientation (and pointing angle). In at least one embodiment, adjustments to a pointing angle of the camera CAM may be done in a direction opposite a direction of acceleration. By way of example, upward accelerations may cause a downward tilt of the camera CAM.
In other embodiments, other signals may be used to adjust the pointing angle. For example, as illustrated in FIG. 2, supplemental adjustment signals may include, but are not limited to, pre-recorded acceleration signals (e.g., acceleration of an earthquake), artificially generated acceleration signals, for instance, generated by an electronic device, acceleration signals provided from a remotely mounted IMU (e.g., mounted to an animal or to a remote chassis when a camera is mounted to a boom), or displacement angle signals. It will be appreciated that these acceleration signals are intended as examples, and any number and/or type of acceleration signals may be provided to the apparatus 200.
As shown, a number of supplemental adjustment signals may be received and one or more of the signals may be selectively provided to the filter 160. It will be appreciated, however, that in some implementations, a plurality of acceleration signals may be provided to the filter 160 simultaneously and/or sequentially such that the pointing angle of the camera 140 may be adjusted in any desired manner using any number of signals.
FIG. 3 is a schematic diagram of an apparatus 300 according to an embodiment of the invention. The apparatus 300 includes elements that have been previously described with respect to the apparatus 100 of FIG. 1. Those elements have been identified in FIG. 3 using the same reference numbers used in FIG. 1 and operation of the common elements is as previously described. Consequently, a detailed description of the operation of these elements will not be repeated in the interest of brevity.
The apparatus 300 may include a controller 124 that may be used to implement the controller 120 of FIG. 1. The controller 124 may receive a command attitude that may be used as a reference angle as described herein. The apparatus 300 may further include an LED 199 and a handle 155 that may be coupled to the gimbal 140.
As described, the motor 150 may be configured to determine a displacement angle for each gimbal axis and further may provide a signal indicating the same. In at least one embodiment, the handle 155 may comprise part of a frame to which the gimbal 140 is attached. Accordingly, a camera operator may carry the gimbal 140 and steer the gimbal 140 to direct the camera CAM at a subject. Because the handle comprises part of the frame to which the gimbal 140 is attached, in at least one embodiment, by determining the displacement angle of the gimbal 140, the motor 150 may determine an angle at which the handle has been rotated relative to the camera CAM. In this manner, the controller 124 may be configured to adjust the pointing angle of the camera CAM based, at least in part, on an angle of the handle.
In operation, a camera operator may steer the gimbal 140 to point the camera CAM in an intended direction. As described, in steering the gimbal 140, the handle 155 may be rotated, and the rotation may be detected by the controller 124 based, at least in part, on displacement angle signals. If the displacement angle, or angle indicated by the displacement angle signals, is different than the reference angle, the controller 124 may adjust the pointing angle of the camera CAM to match the angles. Once matched, the updated pointing angle may be used as the reference angle. In at least one embodiment, the controller 124 may be configured to adjust the pointing angle at a rate based, at least in part, on the magnitude of the difference between the angles. For example, the further the pointing angle from the displacement angle, the more quickly the pointing angle of the camera CAM may be adjusted. In some instances, the controller 124 may further be configured to wait a delay before adjusting the pointing angle. This may, for instance, provide smoother operation of the camera CAM.
Moreover, the controller 124 may be configured to adjust the pointing angle only when the difference between the reference angle and the displacement angle exceeds a threshold. For a difference not exceeding the threshold, the pointing angle may not be adjusted. For a difference exceeding the threshold, the updated pointing angle may be determined using the following pseudo code:
if (angle_measured > angle_threshold) {

angle_out = angle_measured − angle_threshold

}

if (angle_measured < angle_threshold) {

angle_out = angle_measured + angle_threshold

}

where angle_measured represents the displacement angle, angle_threshold represents the threshold angle, and angle_out represents an updated pointing angle of the camera 140.
In at least one embodiment, during instances in which the controller 124 determines that the angle does not exceed the threshold, the controller 124 may enable the LED 199 to indicate that the pointing angle of the camera CAM is in an “aligned” state such that no adjustments are being made. Alternatively, in other embodiments, the controller 124 may be configured to enable the LED 199 when the controller 124 is adjusting the pointing angle of the camera CAM. The controller 124 may further be configured to receive a joint lock signal. Responsive, at least in part, to assertion of the joint lock signal, the controller 124 may lock the current displacement angles, thereby assigning each displacement angle as a respective reference angle, as described above. In this manner, an operator may freely direct the camera without adjustment of the pointing angle until the joint lock signal is no longer asserted. Because the joint lock signal may be asserted using a switch, an operator may selectively assert the joint lock signal to selectively enable a “locked” state for the pointing angle of the camera CAM. For example, a trigger switch positioned on the handle may be pulled to enable the locked state and released to disable the locked state. In some embodiments, the locked state may be enabled by releasing the trigger and disabled by pulling the trigger.
In some instances, a camera operator may desire to manually adjust a pointing angle of a camera, for example, to direct a camera toward a subject. In doing so, the camera operator may apply a force to the camera CAM and/or the gimbal 140. Accordingly, the external force may be measured, in direction and/or in magnitude, and based, at least in part, on the measured force, a controller, such as the controller 124 of FIG. 3 may allow the gimbal 150 to adjust such that the camera CAM is adjusted as desired.
By way of example, the camera CAM may be maintained at a first pointing angle, and a camera operator may wish to adjust the pointing angle. In manually adjusting the camera CAM and/or the gimbal 140, the camera operator may apply a force (e.g., torque) that may be measured, for instance, using control signals provided by a controller to control the gimbal 150. The control signals may, for instance, be indicative of a reaction torque resulting from the applied force. Additionally or alternatively, the force may be measured by a pressure sensor that may be configured to provide a signal to the controller 124 indicating a direction and/or magnitude of an applied force. In response to the applied force, the controller 124 may allow the camera CAM and/or the gimbal 140 to be adjusted manually. The controller 124 may, for instance, cease any stabilization of the gimbal 140 while the force is applied. When the force is no longer applied, the controller 124 may cease allowing the manual adjustment and begin to stabilize the camera CAM at the new pointing angle in accordance with examples described herein, e.g., new displacement angle may be used as a reference angle. In this manner, an operator may adjust a pointing angle any number of times and in each case, the controller 124 may adjust the gimbal such that the camera CAM remains directed in desired direction.
In some examples, the controller 124 may be configured to allow the camera CAM and/or the gimbal 140 to be adjusted manually only when an applied force exceeds a particular threshold. If a force does not exceed a threshold, the controller 124 may stabilize the gimbal 140 to compensate for the force as described herein. In at least one embodiment, the threshold may be set by an operator.
In some examples, the controller 124 may be configured to operate as if the camera has inertia. That is, the controller 124 may decreasingly resist an adjustment of the pointing angle when an operator initially performs a manual adjustment and to increasingly resist an adjustment of the pointing angle when an operator is completing the manual adjustment. The controller may, for instance, resist adjustments in accordance with any differential equation known in the art, including but not limited to one or more decay equations (e.g., equations directed to viscosity). Dampening changes in the rate of change of the pointing angle of the camera CAM may provide a smoother change in pointing angle.
In some examples, the camera CAM may be adjusted using approaches that do not include a gimbal. Instead, the camera CAM may be adjusted using, for example, lifts, winches, balloon systems, telescopic mast systems, booms, long rope systems, pulleys, or any other type of support structure that may support a camera.
FIG. 4 is a schematic block diagram of an apparatus 400 according to an embodiment of the present invention. The apparatus 400 includes elements that have been previously described with respect to the apparatus 100 of FIG. 1. Those elements have been identified in FIG. 4 using the same reference numbers used in FIG. 1 and operation of the common elements is as previously described. Consequently, a detailed description of the operation of these elements will not be repeated in the interest of brevity.
The apparatus 400 may include a controller 126 that may be used to implement the controller 120 of FIG. 1, and may further include a winch 202 and a motor 250. The winch 250 may be coupled to the camera CAM and the motor 250 may be configured to adjust the height of the camera CAM using the winch 202. In at least one embodiment, the motor 250 may comprise the motor 150 of FIG. 1, or may comprise a different motor.
The controller 126 may be configured to receive measurement signals from the IMU 110 indicating a height of the camera CAM, a velocity of displacement, acceleration of displacement, and/or direction of displacement. In some examples, the height may be determined in other manners as described herein. The controller 126 may further receive a commanded height, for instance, from a user, and may compare the commanded height to the height of the camera CAM. Once any difference between the commanded height and height of the camera CAM is determined, the controller 126 may be configured to provide control signals, using the driver 130, to the motor 250 to control the winch 202 such that the height of the camera CAM is adjusted to the commanded height.
In some instances, a camera may be displaced over uneven ground. To compensate, the controller 126 may further be configured to receive a rate control signal indicating the rate at which the height of the camera CAM may be adjusted. By way of example, a user may use a remote control device, such as a joystick, to increase or decrease the height of the camera CAM, for instance, by pushing or pulling the joystick, respectively. In some instances, the controller 126 may be configured to operate in a “vario mode” wherein the controller may adjust the height of the camera CAM only in response to receipt of a rate control signal. When operating in the vario mode, the controller may measure displacement of the camera CAM.
In some examples, the controller 126 may be configured to receive one or more control signals (not shown in FIG. 4) from the winch 202. For example, the controller 126 may receive one or more control signals indicating rotations of one or more pulleys on the winch 202. The controller 126 may set a maximum number of rotations that pulleys may make in a particular direction to prevent jamming or grounding of the camera CAM. In another example, the controller 126 may receive one more signals from limit switches indicating that the winch is at risk of jamming or grounding the camera CAM, and the controller 126 may prevent further height adjustment accordingly.
The winch 202 may further be configured to include a counterbalance weight. This may, for instance, reduce power consumption by reducing power required to maintain a particular height of the camera CAM. Additionally or alternatively, the winch 202 may include a gas spring. Both counterbalance weights and gas springs used in this manner may be configured to maintain tension in the cable of the winch.
While examples described herein are directed to a winch, in some instances, the height of the camera CAM may be adjusted using other devices. For example, a telescopic pole may be used to adjust the height of the camera CAM, and may be hydraulic and/or pneumatic.
As described with respect to FIG. 3, in some cases, a camera operator may desire to adjust the pointing angle of the camera CAM. Similarly, a camera operator may desire to adjust the height of the camera CAM. Accordingly, the controller 126 may be configured to measure a force applied to the camera CAM, allowing the height of the camera to be adjusted, and maintain the camera CAM at the adjusted height. As described, adjustment of the camera CAM in this manner may include a threshold for the applied force and/or may be dampened.
Examples have been described herein with respect to adjustment of a pointing angle of a camera. It will be appreciated that while several features have been described separately, that some or all examples described herein may be implemented simultaneously and/or may be selectively enabled such that any combination of features may be used. Moreover, in many examples, adjustment of a pointing angle is described with respect to a single axis, however, it will be appreciated that examples described herein may be applied to a plurality of axes such that a pointing angle of a camera may be adjusted in any desirable manner.
Moreover, particular embodiments of described examples may further be described in the following examples. The following examples are provided merely to highlight various aspects of described examples and are not intended to be limiting in any manner.

Example A

As described in Example A, the camera pointing angle may be automatically adjusted to compensate for translational movements, which otherwise uncorrected would take the subject outside the camera field of view. By measuring subject distance using a simple range sensor and combining with a high performance IMU that accurately reports 3D velocity measurements, it is possible to correct pointing pan and tilt angles to compensate for movements. The correction rates are geometrically related displacement velocity and pointing angles.
Referring to FIG. 5, an actively stabilized camera gimbal will keep a constant pointing angle by means of gyroscopic feedback. For close-in shots, and in particular where the gimbal is a hand-held device, the effects of translation are noticeable with the subject matter moving out of the camera FOV (field of view). This may be corrected by changing pointing angle; however, thus far this is a manual control process performed by the camera operator or a remote tele-operator.
For fast translation movements and close-in shooting, it is an impossibility to achieve a perfect in frame shot using manual methods. Typically, the filming subject will be an actor, where it is desirable to keep them in frame. This approach allows for an automatic compensation of pointing angle based on a feedback control loop using a combination of position/speed and distance sensors.
The crux of the approach in Example A is to either actively measure or pre-set an approximate distance from the filming target, and to use this in combination with translational measurements to correct a camera pointing angle.
Active distance measurement can be achieved by ultrasonic, laser or infra-red (IR) sensors and would typically work over distances where translation effects are noticeable. An example of an Ultrasonic sensor is MB1200 marketed by MaxBotix Inc. An example of an IR sensor is GP2Y0A02YK0F marketed by Sharp Electronics. The type of sensor and detection profile can be tailored for specific applications. Ultrasonic sensors have a good range and adjustable beam profile suitable for a person target.
Modern developments on IMU's (inertial measurement units) mean it is now possible to actively measure position and velocity with centimeter (cm) and cm/s accuracy. Such IMU's typically incorporate GPS, 3 axis accelerometer, 3 axis gyroscopes, 3 axis compass and a barometer. By complex and novel proprietary algorithms, it is possible to fuse the sensor data in order to derive accurate readings for 3D position and velocity. Such a device has been marketed by Webb Consultancy Ltd. and typically gives cm and cm/s resolution readings at 160 Hz update rate.
In order to correct for a translational velocity, a pointing angular rate correction value can be calculated. For example, if the pointing angle is nominally horizontal then the two parameters are related by a standard equation similar to that describing a rotating body around an origin:
ω=−ν_up/χ
Where ω is angular rate, ν_upis velocity and χ is the horizontal distance of the camera from the subject (see FIG. 6). So here upward movement requires a downward correction in pointing angle. The same equation applies to horizontal displacements where the displacement velocity is that measured tangentially for the radius connecting the camera lens and the subject, by knowing the actual camera pointing direction and local compass heading in combination with GPS measurements.
A gimbal tilt drive control can include a brushless motor and an IMU mounted on the camera body. Referring to FIG. 7, the gimbal controller uses a closed loop electro-mechanical feedback control system to update its pointing angle at a regular rate. Typically the control system is a digital controller based on a software program using techniques such as PID (proportional-integral-differential) control. The digital controller normally has a fixed update rate whereby discrete control decisions are made—typically at 400 Hz, but may be slower of faster dependent on the specific design. The IMU measures the tilt angle and the controller compares this with the commanded angle and then this is used to update a power driver section, and in turn operate an actuator in order that camera remains stabilized at the desired setting. Preferably the IMU also measures tilt rate, and an inner control loops performs stabilization of this motion. The actuator may be some form of motor I gearbox I belt reduction drive; however, it is preferable to use a direct drive brushless DC motor (BLDC) to increase actuation bandwidth and precision. A BLDC driver outputs 3 phase currents to the motor and usually requires an angle resolver to form a local control loop in order to control those currents accurately dependent on motor phase angle.
Referring to FIG. 8, range sensor information is also passed to the tilt compensation controller, either as a digital or analogue signal, and is used to process pointing angle corrections. As an example, if a correction command of 1°/s is required, then the controller will be required to make small integral steps in pointing angle of 1/400=0.0025° every 400 Hz update cycle leading to the illusion of continuous motion. This motion would approximately correct 5 cm/s tangential movement at 3 m.
θ_t+dt=θ_t+ω_t
The correction command is likely to be variable in speed with time. The simple integration method above (Simpson's method) would not perform well except for a constant speed. It is preferable to use a more accurate numerical integration technique in order to reduce integrated errors in the pointing angle. One such technique is so called trapezoidal integration. Here the current angular rate is averaged with the previous angular rate, and that value is used for the angle step update:
θ_t+dt=θ_t+(ω_t+ω_t+dt)/2
where θ is the pointing angle.
Measurement of velocity using an IMU is not an exacting capability; the IMU is subject to some drift and noise which could cause a low random walk of pointing angle when nominally no pointing direction change is required. Macroscopic changes in velocity are very easy to detect. It is preferable to set some threshold whereby only velocities above a certain level contribute to a change in pointing angle. A typical threshold of 5 cm/s has been found to work well for cinematography applications. It would fall back to a manually operated pointing correction method for displacement velocities slower than this threshold.
Further complication may be added where movement is not normal to the pointing angle. For instance, if the camera is already elevated with respect to the subject it would be in a look-downward mode. Translation pointing correction may require less adjustment the further this elevation—tending towards zero for almost above the subject. This effect may be factored into the equation by registering the current pointing angle in addition to subject horizontal distance (in practice the displacement is likely to be no greater than the horizontal distance).
Referring to FIG. 9, a simple modification to the pointing rate equation may be made by calculating the tangential translation velocity for a certain pointing angle. Here the translational velocity is related to viewing angle by:
ν_tangential=cos(θ)·ν_up
In this case the tangential velocity may be used in conjunction with the subject line of sight distance in order to work out the angular correction rate. As expected the ω=−v_up/x for a horizontally displaced subject or zero for a vertically displaced subject.
It is preferable that pointing distance is measured from the camera head to the subject for best effect. Thus, the measurement sensor would be mounted on the camera head and be aimed at the center of the camera field of view.
If no range measurement sensor is used then the distance can be pre-input into the gimbal controller to give an approximate effect. This could be useable on a rehearsed or planned shoot.
Further to this direct or pre-input method of ranging, an inference of subject range may be derived from lens focus adjustment. By calibrating the focus control with real distance, a measure of range may be deduced. The actual camera focus may be automatically controlled or manually adjusted.
By similar means, horizontal translations may be compensated for non-tangential movements. Here the pointing angle would be related by compass and GPS derived Eastward and Northward velocities.
An extension to tilt compensation by measuring actual height also presented. Here a second range sensor can be placed to interrogate the vertical height or the IMU may report measured barometric altitude as shown in FIG. 8. Provided this height is reset with a reference of zero for horizontal pointing then the tilt angle is directly given by:
θ=α sin(distance/height)
This solution may be preferred for some situations, but relies on an accurate height measurement without baseline wander (see FIG. 10, which illustrates direct inference of tilt angle from height and distance). Additionally, the camera operator commanded tilt has to be correlated with the unobstructed height measurement. For instance, if the height measurement is inaccurate due to an obstacle in the path of the height sensor, or sloping ground, then this height error would corrupt the tilt compensation. In practice, an offset correction would be required to be updated every time the camera operator finished changing the tilt command. The offset correction would be the difference in the measured height and the calculated height from tilt angle and distance:
Offset_correction=height_measured·distance.sin(tilt command)
Now:
θ=α sin(distance/(height_measured−offset_correction))
A similar process can be applied to pointing direction; however, in practice conventional GPS position measurements typically have an accuracy of ˜2.5 m CEP over a long time period. For close in shooting, this would be perceived as a constant pointing angle wander and not give the desired effect. It is possible to employ greater GPS positioning accuracy using DGPS techniques (differential GPS).

Example B

Example B describes a method that introduces a specifiable camera shake into a camera stabilization control system to make for more realistic cinematography using an actively controlled gimbal. Measurement of lateral acceleration or joint angles, in conjunction with a high-pass filter provide a means to introduce a camera tilt or pan disturbance giving the illusion of a vibration or translational event. Pre-recorded, synthesized or remote mounted IMU may be used to provide the seed noise signal.
An actively stabilized camera gimbal will keep a constant pointing angle by means of gyroscopic feedback. Used in the movie industry this can provide a useful means to aid a camera operator keeping the subject in shot for an action scene.
However, the stabilization can be too good and lead to a non-realistic result for a watching audience. Without stabilization a scene could be difficult to film and most likely lead to unusable results.
Embodiments of this invention describe a method of introducing realistic disturbance on camera pointing angle whilst maintaining a stabilized pointing direction.
According to Example B, an active electro-mechanical gimbal controller for camera stabilization is based on 3 servo motors in the tilt, roll and pan axis' operating in conjunction with an IMU (inertial measurement unit) and integral control system for closed loop feedback. Disturbance of pointing angle is corrected by the control loop to maintain a constant pointing angle. FIG. 7 is a diagram of a gimbal tilt control with a drive motor and IMU mounted on a camera. Here, a simple diagram is provided for the tilt axis and this control is normally replicated across the 3 degrees of angular freedom. Typically, a PID (proportional-integral-differential) controller normally implemented by digital means such as a software program, is used to output a driver control signal based on the error between the IMU and commanded tilt angle. Preferably, an inner PID loop is used to control tilt rate and by comparing to IMU derived tilt rate for greater response bandwidth—in this case the outer angle loop would be proportional only. The power driver output preferably controls phase currents to a connected direct drive BLDC motor (brush less DC motor). Some other form of actuator may be used, such as a normal DC motor, pulley system or gearbox; however, the use of a direct drive BLDC allows greater control precision and faster response times.
The IMU sub-system typically consists of a 3-axis accelerometer, 3-axis gyroscopes, barometer, compass and GPS. Such a device has been marketed by Webb Consultancy Ltd. and incorporates proprietary sensor fusion algorithms to accurately derive pointing angles, velocities and acceleration both in body frame and reference frame with a high update rate—typically 160 Hz.
Referring to FIG. 11A, during a filming sequence the gimbal may be carried by a camera operator or a vehicle and be subject to acceleration forces and also rotational movements of the support structure. For instance, the camera operator may be walking or running whilst tracking the subject. These acceleration forces will be registered by the IMU and output as a combination of up/down, left/right and forwards/backwards accelerations, and pan/tilt/roll rates, all of which are resolved.
The control system will seek to eliminate all pan/tilt/roll rate fluctuation and maintain a perfect constant attitude—within the limits of the system bandwidth leading to a perfect steady image at the camera. The crux of the approach in Example B is a means to apply a controlled corruption to this stabilization to provide the illusion it was filmed in first-person-view, immersing the audience into the scene with more realism, whilst maintaining accurate stabilization to allow the filming to take place.
The illusion of real movement can be mimicked by altering the camera pointing angle by a small amount, such as illustrated in FIG. 11B. For angular changes, the perceived displacement is approximately
displacement=tan(dθ).distance
Where dθ is the angular displacement and distance is the measurement to the filming subject. By scaling the value of dθ, different perceived displacements may be achieved irrespective of real filming distance and indeed will be related to the zoom set-up on the lens system.
Actual camera displacement is a transitory event returning to the original position, it is like an acceleration phenomena. By measuring IMU body accelerations, an approximation camera operator noise may be achieved. Furthermore, this acceleration may be resolved into up/down and left/right.
Using this acceleration information, an addition can be made to the pointing angle. For instance up/down accelerations are arranged to provide transitory up/down tilt by some adjustable scaling factor. Referring to FIG. 12, raw acceleration measurements are resolved by the IMU into the actual FOV (field of view) irrespective of tilt angle. Up/down would be on a line from bottom to top of the camera. Due to the Earth's gravitational field, there will be a constant acceleration registered that will be resolved depended on tilt angle. Since transitory acceleration is the aspect of interest, this constant offset needs to be subtracted by an adjustable high pass filter. FIG. 13 s a graph illustrating high-pass filtering of an acceleration measurement. An adjustable scale factor is then used to multiply this acceleration measurement and add it into the commanded set angle. An adjustable filter bandwidth can further alter the strength and nature of the effect. Thus, a tilt motion is instructed in sympathy with transitory accelerations and they may be dialed in for consistent behavior. It would be normal to adjust the scale factor such that upward acceleration provided a downward tilt and vice-versa. It is possible to invert this factor and provide upward tilt for upward acceleration for a particular illusion as desired.
Lateral accelerations would be treated with the same scheme, but this time controlling pan axis.
By linking the real measured acceleration to the image a realistic result may be achieved at the point of filming. Other solutions may be use pre-recorded accelerations for a planned sequence (see FIG. 14). Indeed the data could be synthesized by a computer program, perhaps to simulate an earthquake. Accelerations measured in other parts of the camera system might be preferable. For instance, a camera gimbal supported on a boom from a camera car is unlikely to measure appreciable high frequency acceleration noise. As also illustrated in FIG. 14, by mounting a remote chassis based accelerometer more realistic measurements may be made.
Camera shake can also be emulated by monitoring gimbal joint angles. This would be akin to measuring rotational movements if the camera was hard-mounted or held by an operator. Slight modification to the show how this may be achieved using the BLDC resolver shown in FIG. 14. By high pass filtering the measured joint angles a symmetrical jitter or shake signal is derived. The signal may be introduced at a controlled level in a similar way to the acceleration derived shake.

Example C

In Example C, camera pointing angles may be adjusted by moving the gimbal base in a pan, tilt or roll motion. By measuring the angle of the gimbal base in relation to the camera, it is possible to allow the user to control gimbal pan, tilt, and slew rates. Examples of the invention allow seamless pointing and translation control of a camera as a single operator.
An actively stabilized camera gimbal typically requires a remote operator to control the pan and tilt slew rates. With a handheld gimbal this requires two operators to translate and point the gimbal simultaneously as illustrated in FIG. 15.
Successful filming requires careful collaboration between the parties controlling the translation route and pointing plan of the camera. There is a complexity of multiple radio transmitters together with extra equipment and resources.
Examples of the invention allow a single operator translating the camera to also intuitively control the camera pointing angle without sacrificing the benefits of active stabilization.
Stabilized gimbals typically require human operators to control the pointing angle via a remote monitor and joystick. Additionally, because of the lack of lightweight, handheld, and easily moveable gimbals, it was impossible for the person operating the gimbal to also carry the gimbal. With the advent of small, lightweight, handheld gimbals, it is now possible for a human to carry a stabilized gimbal. Accordingly, a way to intuitively control the camera angles for the person carrying the gimbal is desirable.
The crux of the approach in Example C is a method of using the gimbal stabilization controller to also steer the camera pointing direction by the operator using support base rotation and tilting.
An active gimbal stabilization controller is detailed in FIG. 7. An IMU attached to the camera senses camera tilt and a digital controller based on PID techniques (proportional-integral-differential) compares camera attitude with desired attitude. Attitude error is amplified and fed to a BLDC (brushless DC motor) via a power driver stage to form a closed loop feedback control circuit. Preferably a direct drive BLDC is used for increased control bandwidth and resolution however other solutions may be used such as geared or belt reduction DC drives. The digital controller typically runs at a fixed update rate of some 400 Hz. The same set-up is repeated for pan and roll axis' and a suitable mechanical arrangement.
The scheme as drawn in FIG. 7 requires a command attitude to be fed into the controller as the “set-point”. Conventionally this may be via the second operator and a remote link or possible a small thumb joystick that the camera operator may have access to. Both are not an ideal solution and can compromise gimbal maneuvering.
The subject of the approach of Example C uses a measurement of the actual gimbal joint angles to provide a control signal indicating the camera operators pointing intentions. The handheld gimbal has a support base formed of lifting and steering handles which the operator will always be holding. In order to control the steerage, the support handles are rotated in the intended direction and the gimbal will track this motion. For an example, in a nominally horizontal stance the gimbal remains horizontal. If the operator tilts the gimbal handles forwards then the gimbal will start to tilt downwards at a rate proportional to the estimated handle angle. Returning the handles to the level will stop further movement, and indeed bringing them above the horizontal will cause an upwards movement in tilt. In this way by tilting the handles one way or another, the camera operator has direct control of the pointing angle.
The same idea can be associated with the pan axis. The pan process would stop once the gimbal has been steered to a point where the handles are centralized. The simplest solution is to set-up an outer control loop that effectively zeroes the joint angle in both pan and tilt as shown in FIG. 16. In FIG. 16, the desired tilt is down. When the camera angle is effectively the same as the joint angle, the controller would be stable and converged. The control loop would be arranged to be PI where the P (proportional) control term allows for a stronger or faster response for larger errors. The I (integral) term allows for a time-constant which may be tuned to give a slow and fluid response if a sufficiently large value is chosen.
For the operator there would be some issue that he would exactly have to hold a zero joint angle to prevent the gimbal from moving. This would be difficult to achieve in practice and there would be small errors constantly being corrected. A solution to this issue is to add some threshholding function to the joint angle measurement. For angles within a certain limit the angle is registered as zero, outside those angles the angle is registered with this pseudo code:
if(angle_measured>angle_threshold)angle_out=angle_measured−angle threshold
if(angle_measured<−angle_threshold)angle_out=angle_measured+angle threshold
Effectively, a deadband is introduced that the camera operator may not worry about accurate pointing within this region. As soon as he exceeds it then slow movement would be achieved proportional to the angle_out value.
To make this dead-banding more obvious, some form of visible indicator such as an LED could be lit when inside the dead-band giving a direct pointing lock indication.
A further optional enhancement to allow desirable pointing control would be the use of a trigger button to lock the current joint angles via a process of sampling the current joint angles. The angle control loop then uses the sensed joint angle for feedback rather than the attitude from the IMU. With this scheme, the camera operator would be responsible for pointing direction changes until the trigger was released. Thus, the operator may easily switch between stabilized and non-stabilized pointing at the press of a button.

Example D

Example D describes a method of active camera gimbal steerage by sensing an externally applied torque to register a new pointing direction. For external torques below some threshold the gimbal reverts to a normal closed loop stabilization mode.
An actively stabilized camera gimbal typically requires a remote operator to control the pan and tilt slew rates. With a handheld gimbal this requires two operators to translate and point the gimbal simultaneously as illustrated in FIG. 15.
Successful filming requires careful collaboration between the parties controlling the translation route and pointing plan of the camera. There is complexity of multiple radio transmitters together with extra equipment and resources.
Examples of the invention allow a single operator translating the camera to also directly control the camera pointing angle without sacrificing the benefits of active stabilization. The pointing control is intuitive and emulates a camera fixed to a normal tripod.
Stabilized gimbals typically require human operators to control the pointing angle via a remote monitor and joystick. Additionally, because of the lack of lightweight, handheld, and easily moveable gimbals, it was impossible for the person operating the gimbal to also carry the gimbal. With the advent of small, lightweight, handheld gimbals, it is now possible for a human to carry a stabilized gimbal. Accordingly, a way to intuitively control the camera angles for the person carrying the gimbal is desirable.
The crux of the approach of Example D is a method of using the gimbal stabilization controller to also allow an at will steering capability of the camera pointing direction by the operator applying an external disturbing force to the gimbal.
The force is in the direction of rotation required—for instance down to tilt down and left to pan left. The force may be applied to the gimbal structure or more naturally to the camera lens body. If the disturbance force is removed, then the control loop is designed to maintain the last pointing angle. The gimbal is naturally balanced and there are nominally only low forces for the stabilizer to react against.
A single axis active gimbal stabilization controller is detailed in FIG. 7. An IMU attached to the camera senses camera tilt and tilt rate. A digital controller based on PID techniques (proportional-integral-differential) compares camera tilt with desired tilt. The attitude error is amplified by a proportional gain constant (Kp), termed a P loop, and fed into an inner angular rate control loop. Here the IMU tilt rate is compared with the output of the attitude P loop to give a control error. The control error is amplified using proportional, integral and differential constants (Kp, Ki & Kd) to form a PID loop. The output of this loop is fed to a BLDC (brushless DC motor) via a power driver stage to form an overall closed loop feedback control circuit. Preferably, a direct drive BLDC is used for increased control bandwidth and resolution; however, other solutions may be used such as geared or belt reduction DC drives. The digital controller typically runs at a fixed update rate of some 400 Hz. The same set-up is repeated for pan and roll axis' using a suitable mechanical arrangement.
Disturbances to the gimbal pointing direction are automatically corrected using the above control loops. The scheme as drawn in FIG. 7 requires a command attitude to be fed into the controller as the “set-point”. Conventionally, this may be via the second operator and a remote link or possibly a small thumb joystick that the camera operator may have access to. Both are not an ideal solution and can compromise gimbal maneuvering.
For the approach of Example D, a method to register the control force applied externally is used. One such method is to measure the reaction torque or the control signal applied to the driver motor. This torque is a direct result of the external force and a threshold may be applied.
For forces below a threshold, the controller treats these as normal stabilization requirements and simply keeps the control loop in closed loop.
Forces above this threshold allow the gimbal to move in the required direction.
If a force is applied to a control axis, there will be some small displacement due to the natural control loop response. The combination of registering the force above a certain level and this displacement may be used to update a new holding angle. As soon as the new holding angle is registered, the control loop will back off and settle into the new position. Consequently, the torque will also reduce and, unless the force is re-applied, motion will stop.
In the stopped state, the gimbal will provide normal closed loop attitude stabilization.
To continuously slew the gimbal, the operator has to maintain a force in the required direction. Effectively, the algorithm self-stops when this force falls below the threshold point.
Diagrammatically the algorithm is illustrated in FIG. 17. The sample and hold element register the current joint angle whenever the output control signal exceeds a torque threshold. The threshold measurement element also resets the integral accumulator in the PID rate loop to effectively set the current control position with a zero error. The net result is a smooth movement with little discontinuity or pointing hysteresis. The setting of a particular threshold value adjusts the amount of torque required before movement is allowed. This may be tuned for operator preference.
The algorithm may alternatively use a direct reading of external force via some form of external force pressure sensor. This would require more complexity.
The algorithm may be enhanced by mathematically treating the external torque as a force acting on an inertia. The inertia would be represented by an equation and allow some steady increase in pointing direction like a heavy flywheel. On release of the torque, the pointing direction would continue to change but slowly decay in rate as if the flywheel was coming to rest. In this way, a smooth pointed movement is possible, which is desirable for filming. The decay equation can be represented by the physics of viscosity where force is a function of velocity squared and could represent a real oil damper. Thus, by the use of an electro-mechanical system and some equations of motion, a large inertia camera and gimbal may be emulated with a light weight system. The system could be tuned for a particular response as required.

Example E

Example E describes a method of feedback control to stabilize and control the height of a supported camera using an IMU and PID control loops. Camera height is held constant when the support structure may be moving up and down, for instance due to terrain whilst a translation move is performed.
The solution of Example E eradicates vertical translation errors and simplifies filming tasks for a camera operator.
Filming a subject often involves translation of a camera from one position to another. It is desirable to perform such translation with a stabilized gimbal in order to obtain good results. It is further desirable to perform the translation at a constant height without any vertical translation errors.
Often cameras are hand carried by operators or possibly strung from a cable system. An example of a support system would be “Easy-rig,” illustrated in FIG. 18A, with which a camera operator will find it difficult to not induce vertical translation errors.
The subject of this Example E is to provide an active control loop to remove all or some of the vertical translation errors using an electromechanical system which may change the camera suspension height as illustrated in FIG. 18B.
It is now possible to measure vertical position and velocity in real time and to cm and cm/s accuracy. Such devices that report this information are typically based on a sensor fusion of accelerometer, gyroscopes and barometer, and may give update rates typically at 160 Hz. One example has been marketed by Webb Consultancy Ltd. It is an IMU (inertial measurement unit) intended for autopilot control of VTOL flight systems and also has integrated control capabilities.
By using this information, it is possible to construct a digital control loop using PID (proportional, integral, differential) techniques and to then feedback control errors to a servo system that may correct the camera height and maintain a constant value.
Referring to FIG. 19, the control preferably consists of three input signals from the IMU-height, climb rate and climb acceleration and a nested PID structure. An outer P loop registers the actual height and compares to a commanded height to derive a command climb rate. Errors in height will change the command velocity +Ve thru −Ve. The next inner P loop compares real climb rate with the commanded climb rate and produces and acceleration command output. The inner-most PID loop does similar for the measured acceleration and preferably includes differential and integral terms to improve overall accuracy and speed. Deflection in height, velocity or acceleration will produce a restoring control signal at the control output.
The controller drives a winch system which preferably consists of a sinusoidally commutated BLDC motor (brushless DC) and using an angle resolver and driver circuit. Other systems may be employed such as a DC motor, with or without gearbox, hydraulic or compressed gas. The use of a BLDC motor provides convenience, low torque ripple and high torque capabilities. The overall control system has high bandwidth and resolution.
So far, the system has been described for a fixed height command and the feedback ensures that effects of any support beam translations are mitigated by the closed loop. In practice, there would be a need to change the camera height such as when filming on sloped ground. In this instance, the command height may be manipulated using a ‘vario’-like control input. A joystick or thumb wheel may be used to command a climb rate where central stick would be stationary and +Ve or −Ve would be ascend or descend.
Further to this, it is possible to introduce a central deadband on the control to force height hold in the approximately central position. Alternatively, a joystick or knob may be sprung loaded to force a return to the center and zero rate command.
It is possible for the height control command to be derived from an assistant camera operator over a radio command link.
It is desirable to have extension limit clamps on the control system to prevent grounding out or jamming. These may be introduced by rotation counting or limit switches on the winch cable.
As described, the weight of the camera system has to be supported by the winch and, consequently, there is a constant torque applied by the motor. This is wasteful of power and will cause unnecessary motor heating. It is preferable to have some form of counter-balance weight such as used in a passenger lift. Alternatively, rather than a weight, a gas-spring may be used. Gas springs have the advantage they may be designed to be used in both compression and extension, and also the restoring force is approximately constant versus displacement. Such a spring could be matched to the camera weight in the same way a counter-weight would be. The pressure in the spring may be manipulated to improve this match and possibly settable by the camera operator to allow for different camera equipment to be interchanged. By using a spring, the control system is centralized and may provide both +Ve and −Ve torques as required.
In this Example, it is required that the spring or counter-balance weight is placed in a way to always keep the support cable in tension.
It is possible to use other means to change the camera height. For instance the support pole may be a telescoping design with a geared rack and pinion drive mechanism. As previously mentioned, it could also be hydraulic or pneumatic based.
A camera operator can find the work load of both pointing and translating the camera to be quite high. The addition of the command height vario control does not improve matters. It is possible to introduce a control loop detection method that recognizes they wish to change the height. By sensing the motor torque command, if the operator tries to lift the camera up or pull it down, there is knowledge they are wishing to perform this move. The control may be crafted to allow this action by use of a threshold detector and a sample and hold switch such as illustrated in the diagram of FIG. 20. Here, once the force and resulting motor control signal exceeds some threshold, a new height measurement is captured and the PID integrator is reset. This allows a fluid-like displacement of the camera and returns to a solid height hold afterwards.
Alternatively, command climb rate may be forced to be zero and operated in a purely vario mode. Here, the height may slowly drift with time, especially if the weight is unbalanced; however, the control system will still provide high bandwidth feedback control for transient events, as shown in FIG. 21. The operator may force the height to be changed and will feel a natural resistance in doing so, but will not be prevented.
Thus-far, the Example E has been targeted at a single operator carrying a camera on something like an Easy-rig support system. The Example E doesn't preclude applications outside this regime and may include: Crane systems, Balloon systems, Telescopic mast systems, Extension boom systems that are commonly installed on camera cars, Long rope systems for instance to descend into a deep cave, or any other form of camera support structure.
From the foregoing it will be appreciated that, although specific embodiments of the invention have been described herein for purposes of illustration, various modifications may be made without deviating from the spirit and scope of the invention. Accordingly, the invention is not limited except as by the appended claims.

Claims

1-3. (canceled)

4. A method for correcting a pointing direction of a camera, actively stabilized by an active stabilization system in accordance with a commanded pointing angle, to compensate for translational movements of the camera, the method comprising:

determining a distance from the camera to a filming target;

deriving one or more translational measurements associated with a translational movement of the camera;

calculating a correction update as a function of at least the distance and the one or more translational measurements; and

adjusting the commanded pointing angle of the camera based on the correction update to retain the filming target within a field of view of the camera.

5. A method according to claim 4, further comprising:

stabilizing the pointing direction of camera based on the adjusted commanded pointing angle.

6. A method according to claim 4, wherein the calculating a correction update step comprises:

calculating a correction update as a function of at least the distance, the one or more measurements, and a current pointing angle of the camera;

wherein the function for calculating the correction update rate provides correction updates of lower values for the camera having a higher elevation in relation to the filming target than for the camera having a lower elevation in relation to the filming target for translational movements of a same magnitude.

7. A method according to claim 4, wherein:

the commanded pointing angle is adjusted incrementally, using an incremental correction update determined based on the correction update; and

the distance, the one or more translational measurements, and the correction update are being updated with each incremental step.

8. A method according to claim 4, wherein:

the one or more translational measurements comprise a vertical height of the camera in relation to the filming target; and

the calculating a correction update step comprises:

calculating, based on the distance and the vertical height, a desired pointing angle for retaining the filming target within the field of view of the camera; and

calculating the correction update based on the desired pointing angle and a current pointing angle of the camera.

9. A method for introducing controlled disturbance into an active stabilization system executing a stabilization process to stabilize a pointing angle of a camera housed by the active stabilization system in accordance with a commanded angle, the method comprising:

acquiring a measurement associated with a movement of the active stabilization system;

determining a noise value based on the acquired measurement; and

injecting the noise value into the stabilization process causing the process to adjust the pointing angle of the camera in a direction away from the commanded angle of the camera.

10. A method according to claim 9, wherein the injecting step comprises:

executing an angle-based control loop of the stabilization process based on the commanded angle to calculate a commanded rate;

adjusting the commanded angle rate using the noise value; and

executing a rate-base loop of the stabilization process based on the adjusted commanded angle.

11. A method according to claim 9, wherein the determining step comprises:

filtering the measurement to derive a transitory component of the measurement; and

scaling the transitory component to determine the noise value.

12. A method according to claim 9, wherein:

the method is performed for one of a pan axis, a tilt axis, and a roll axis; and

the pointing angle of the camera is adjusted for the one axis.

13. A method according to claim 9, wherein the noise value is one of an angle, an angular rate, a control torque, and a drive current.

14. A method for adjusting a pointing angle of a camera housed by an active stabilization system configured to stabilize the camera in accordance with a commanded pointing angle, the system comprising a steering member, the steering member rotatable around one or more of a pan axis, tilt axis, and roll axis of the system, the method comprising:

deriving a joint angle measurement of the steering member associated with a rotational movement of the steering member; and

adjusting the pointing angle of the camera, based on the derived joint angle measurement, in a direction of the rotational movement of the steering member, if the joint angle measurement exceeds the threshold window.

15. A method according to claim 14, further comprising:

actively stabilizing the pointing angle of the camera in accordance with the commanded pointing angle, if the joint angle measurement is within the threshold window.

16. A method according to claim 14, further comprising:

indicating, by the active stabilization system, a pointing angle locked state, if the joint angle measurement is within the threshold window.

17. A method according to claim 14, further comprising:

updating the joint angle measurement, wherein the updating step comprises:

reducing the joint angle measurement by a threshold value of the threshold window, if the joint angle measurement exceeds the threshold window, and

setting the joint angle measurement to zero, if the joint angle measurement is within the threshold window; and

deriving a control command for adjusting the pointing angle of the camera based on the updated joint angle measurement.

18. A method according to claim 17, further comprising:

executing an angle-based control loop to derive a commanded angle rate; and

executing a stabilization control loop update based on the updated joint angle measurement and a zero commanded angle to derive the control command for adjusting the pointing angle.

19. A method for adjusting a pointing direction of a camera housed by an active stabilization system, the active stabilization system executing a stabilization process to stabilize the pointing direction of the camera, the method comprising:

detecting an externally applied force;

disabling the stabilization process upon detecting a manual adjustment condition;

adjusting the pointing angle of the camera in a direction of the externally applied force;

measuring a pointing angle of the camera; and

re-enabling the stabilization process to stabilize the pointing direction of the camera based on the measured pointing angle of the camera in response to detecting that the detected manual adjustment condition failed.

20. The method according to claim 19, wherein:

the detecting the manual adjustment condition comprises determining that the external force exceeds a pre-set threshold; and

the manual adjustment condition fails when the external force falls below the threshold.

21. The method according to claim 19, wherein the external force is detected based on a control signal issued by the active stabilization system for controlling movement of a motor of the active stabilization system.

22. The method according to claim 19, wherein the disabling step comprises:

disabling an angle-based control loop of the stabilization process; and

resetting an integral accumulator of a rate-based control loop of the stabilization process.

23. The method according to claim 19, executed for one or more of a pan axis, a tilt axis, and a roll axis, wherein the externally applied force is detected in relation to the one or more axes.