WO2022151473A1

WO2022151473A1 - Photographing control method, photographing control apparatus and gimbal assembly

Info

Publication number: WO2022151473A1
Application number: PCT/CN2021/072468
Authority: WO
Inventors: 殷汇鹏
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2021-01-18
Filing date: 2021-01-18
Publication date: 2022-07-21

Abstract

A photographing control method, a photographing control apparatus and a gimbal assembly. The photographing control method comprises: determining, by means of measurement information of an accelerometer in an inertial measurement unit (IMU), a position change amount of a photographing apparatus in a specific direction (S302), the IMU being provided in the photographing apparatus, or the IMU being provided in a gimbal, and the IMU being used for measuring posture information of the photographing apparatus; and controlling, according to the position change amount, the photographing apparatus to follow focus or zoom (S304).

Description

Shooting control method, shooting control device and pan/tilt assembly

technical field

The present application relates to the field of control technology, and in particular, to a photographing control method, a photographing control device, and a pan/tilt assembly.

Background technique

In many situations, such as filming, a more professional manual lens will be required for shooting. For example, Dolly Zoom is also known as "Hitchcock-style zoom". When shooting, the camera moves forward and backward relative to the target object at a constant speed, and at the same time, the FOV of the camera is evenly reduced (zoomed in), so that the proportion of the target object in the frame remains unchanged. The objects on the background of the screen will be continuously enlarged (reduced), which can achieve the shooting effect of highlighting the main position of the target object. For another example, using a manual lens requires the lens operator to manually adjust the lens in real time according to the distance between the subject and the camera to achieve follow focus. The above-mentioned scenes require a high level of operating experience for the photographer.

SUMMARY OF THE INVENTION

In view of this, the embodiments of the present application provide a shooting control method, a shooting control device, and a pan/tilt assembly, so as to reduce the dependence of the manual lens on operating experience when dealing with the above-mentioned scenarios.

In a first aspect, an embodiment of the present application provides a photographing control method, the method comprising: determining a position change amount of a photographing device in a specific direction through measurement information of an accelerometer in an inertial measurement unit (Inertial Measurement Unit, IMU for short) , the inertial measurement unit is arranged on the photographing device, or the inertial measuring unit is arranged on the pan/tilt carrying the photographing device, and the inertial measuring unit is used to measure the attitude information of the photographing device; control the focusing or zooming of the photographing device according to the position change amount.

In a second aspect, an embodiment of the present application provides a shooting control device, the device includes: at least one processor and a memory; the memory stores computer-executed instructions; at least one processor executes the computer-executed instructions stored in the memory, so that the computer-executed instructions are executed. The following steps are implemented when the measurement information of the accelerometer in the inertial measurement unit is used to determine the position change amount of the shooting device in a specific direction, the inertial measurement unit is set on the shooting device, or the inertial measurement unit is set on the pan/tilt carrying the shooting device, the inertial The measuring unit is used to measure the attitude information of the photographing device; control the photographing device to follow focus or zoom according to the position change amount.

In a third aspect, an embodiment of the present application provides a pan-tilt assembly, the pan-tilt assembly includes a pan-tilt and a motor detachable from the pan-tilt, the pan-tilt assembly further includes a shooting control device, and the shooting control device is arranged in the pan-tilt and the motor In one of the above, the shooting control device includes: at least one processor and a memory; the memory stores computer-executed instructions; at least one processor executes the computer-executed instructions stored in the memory, so that the following steps are implemented when executing the computer-executed instructions: Acceleration in the inertial measurement unit The measurement information of the meter is used to determine the position change amount of the photographing device in a specific direction, the inertial measurement unit is arranged on the photographing device, or the inertial measurement unit is arranged on the pan-tilt carrying the photographing device, and the inertial measuring unit is used to measure the attitude information of the photographing device; The motor is controlled according to the amount of position change to make the camera follow focus or zoom.

In a fourth aspect, an embodiment of the present application provides a readable storage medium on which a computer program is stored; when the computer program is executed, the shooting control method of the embodiment of the present application according to the first aspect is implemented.

In a fifth aspect, embodiments of the present application provide a computer program, including executable instructions, which, when executed, implement the above method.

In the embodiment of the present application, the position change amount of the photographing device in a specific direction is determined based on the measurement information of the accelerometer in the inertial measurement unit, so that the photographing device can be driven to follow focus or zoom based on the position change amount, reducing the user's High reliance on operating experience when implementing follow focus operation or zoom operation in special scenarios. In addition, since the inertial measurement unit can be configured to measure the attitude information of the photographing device, there is no need to configure an additional costly sensor to detect the distance change between the photographing device and the photographing object, thereby realizing the multiplexing of the inertial measurement unit.

Advantages of additional aspects of the present application will be set forth in part in the following description, in part will be apparent from the following description, or learned by practice of the present application.

Description of drawings

The above and other objects, features and advantages of embodiments of the present application will become more readily understood from the following detailed description with reference to the accompanying drawings. In the accompanying drawings, various embodiments of the present application will be illustrated by way of example and not limitation, wherein:

1 is an application scenario of a shooting control method, a shooting control device, and a pan-tilt assembly provided by an embodiment of the present application;

FIG. 2 is an application scenario of a shooting control method, a shooting control device, and a pan-tilt assembly provided by another embodiment of the present application;

3 is a schematic flowchart of a shooting control method provided by an embodiment of the present application;

4 is a schematic diagram of adjusting the focal length by changing the distance between lenses according to an embodiment of the present application;

5 is a schematic diagram of a lens provided by an embodiment of the present application;

6 is a schematic diagram of a flange provided in an embodiment of the present application;

Figure 7 is a schematic diagram of the implementation principle of dolly zoom;

8 is a schematic structural diagram of a PTZ provided by an embodiment of the present application;

9 is a schematic diagram of a specific direction provided by an embodiment of the present application;

10 is a schematic diagram of a zoom motor provided by an embodiment of the application;

11 is a flowchart of a calibration process for a zoom motor provided by an embodiment of the present application;

12 is a schematic diagram of a calibration process for a zoom motor provided by an embodiment of the present application;

13 is a schematic diagram of a follow focus motor provided by an embodiment of the application;

14 is a flowchart of a calibration process for a follow focus motor provided by an embodiment of the present application;

15 is a schematic diagram of a calibration process for a follow focus motor provided by an embodiment of the application;

16 is a schematic diagram of an included angle satisfying a simplified condition provided by an embodiment of the present application;

FIG. 17 is a block diagram of a PTZ provided by an embodiment of the present application;

FIG. 18 is a block diagram of a pan/tilt assembly provided by an embodiment of the present application; and

FIG. 19 is a schematic diagram of a motor provided by an embodiment of the present application.

Detailed ways

Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to be used to explain the present application, but should not be construed as a limitation to the present application.

It should be noted that when a component is referred to as being "fixed to" another component, it can be directly on the other component or there may also be a centered component. When a component is said to "carry" another component, the other component may be carried on a surface of the component, or disposed within the component, or disposed within or on the surface of other components that the component carries. When a component is considered to be "connected" to another component, it may be directly connected to the other component or there may also be an intervening component.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. The terms used herein in the specification of the application are for the purpose of describing specific embodiments only, and are not intended to limit the application. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items. The terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second" may expressly or implicitly include one or more features.

Taking third-party shooting devices as an example, the lenses of SLR and mirrorless cameras have three parameters: follow focus, zoom, and aperture. The follow focus function can be achieved by an automatic follow focus lens, or a manual follow focus lens plus a follow focus device (follow focus wheel and follow focus motor, etc.). Most of the zoom lenses are manually adjusted, and there are many prime lenses.

For example, when shooting Dolly Zoom, the camera moves at a constant speed forward and backward relative to the target object, and at the same time evenly reduces (enlarges) the camera's Field Of View (FOV), so that the target object occupies the same proportion of the screen, while the screen The objects in the background will be continuously enlarged (reduced), which can achieve shooting effects such as highlighting the main position of the target object.

In order to achieve this shooting effect, it is necessary to achieve focus and continuous and smooth zooming. Generally, professionals use slide rails to dolly and continuous manual zooming (zoom) to shoot, and then use image post-processing software for post-production. It is difficult to achieve similar effects for novice users and single-person operations. For example, in the field of single-lens reflex camera and micro-single lens, there is no automatic realization scheme of optical zoom.

The shooting control method, the shooting control device, and the pan/tilt assembly provided by the embodiments of the present application can effectively reduce the high requirements on the user's operating experience when performing the above-mentioned lens movement shooting, and improve the user experience.

It should be noted that the above shooting scenes may be shooting scenes for people shooting, still scene shooting, animal shooting, video shooting, movie shooting, TV drama shooting, and the like. The above scenarios are only exemplary descriptions and should not be construed as limitations on the present application. For example, it can be applied to various operations that require at least one of a follow focus operation and a zoom operation.

In order to facilitate a better understanding of the embodiments of the present application, an exemplary description of the gimbal and the tracking technology based on the gimbal in the related art will be given first.

The appearance of the gimbal not only provides a stable shooting environment for taking pictures and video recordings, but also provides a wealth of possibilities for photographers to move the mirror. For example, a hand-held gimbal can cause the camera set on it to adjust the pose to track the target object.

The pan/tilt provided by the embodiments of the present application can make the camera set on it change the focal length to achieve specific visual effects, such as follow focus and Dolly Zoom effects. In addition, the gimbal can make the camera arranged on it rotate around at least one of the yaw axis, the pitch axis and the roll axis, so as to achieve a specific lens movement shooting effect.

However, using the gimbal to achieve follow focus and Dolly Zoom effects requires a high level of user experience. Beginner users often cannot get the desired picture, and cannot keep the subject stably during the movement of the gimbal. in the designated area of the screen.

For the shooting control method, shooting control device, and pan/tilt assembly provided by the embodiments of the present application, for a shooting device equipped with a manual zoom lens, an inertial measurement unit (Inertial Measurement Unit, IMU for short) (such as the original IMU on the pan/tilt or the external The measurement information output by the IMU) estimates the distance change between the lens and the target object, and obtains continuous zoom information according to the distance change. The motor-driven manual zoom lens performs zoom or follow focus based on the zoom information, to achieve such as dolly zoom, etc. Corresponding mirror movement method. It allows novice users to shoot film-level shooting effects, enriching shooting techniques and creative expression effects.

In order to facilitate the understanding of the technical solutions of the present application, detailed descriptions are given below with reference to FIGS. 1 to 19 .

FIG. 1 is an application scenario of a shooting control method, a shooting control device, and a pan/tilt assembly provided by an embodiment of the present application.

In the related art, in order to realize that the proportion of the image of the hero in the photographed image remains unchanged (the position may also remain unchanged) during the process of the photographing device gradually approaching the hero as shown in FIG. 1, the background of the photographed image remains unchanged. A change occurs that requires two or more users to do so by means of continuous zooming operations such as track push and through the photographer.

As shown in FIG. 1 , in the embodiment of the present application, it is not necessary to cooperate with multiple users to achieve effects such as Dolly Zoom. The user can achieve the above shooting effect with the help of the gimbal and the motor set on the gimbal. For example, while controlling the gimbal to approach the male protagonist, the user controls the motor carried by the gimbal to drive the lens to zoom synchronously. Wherein, the focal length change amount of the zoom may be determined based on the position change amount of the gimbal, and the corresponding relationship between the focal length change amount and the position change amount of the gimbal may be determined by calculation and/or calibration.

It should be noted that the pan/tilt in FIG. 1 may be a handheld pan/tilt or a pan/tilt mounted on a movable platform, and the pan/tilt carries a photographing device, and the photographing device may have a focus adjustment component.

For example, a handheld pan/tilt head may include a stand, a handle, and the like. Wherein, the bracket may include a matching motor and a shaft arm, and the motor is used to drive the shaft arm to rotate, so as to drive the movement of the photographing device.

The pan/tilt head may include, but is not limited to, a single-axis or multi-axis attitude-adjustable structure for fixing the photographing device on the hand-held part. For example, the gimbal allows the camera to be displaced relative to the handpiece, or rotated along one or more axes, such as the gimbal allows the camera to move along one or more of the pitch, pan, and roll axes combined translational motion. As another example, the pan/tilt head may allow the camera to rotate about one or more of a pitch axis, a pan axis, and a roll axis. There may be a linkage conversion relationship between the gimbal and the hand-held part, for example, the first movement (such as movement or rotation) of the hand-held part can be converted into the second movement of the gimbal. vice versa.

In addition, a sensor system may also be included on the gimbal. The sensing system may include one or more sensors to sense spatial orientation, velocity, and/or acceleration (eg, rotation and translation with respect to up to three degrees of freedom). The one or more sensors include, but are not limited to, GPS sensors, motion sensors, inertial measurement units, or image sensors. The sensing data provided by the sensing system can be used to control the pose, speed and/or acceleration of the photographing device. Optionally, a sensing system may be used to detect data about the environment of the gimbal, such as climatic conditions, the location of man-made structures, and the like.

In addition, the PTZ may also include a communication system. The communication system can realize the communication between the PTZ and the control terminal with the communication system through wired or wireless signals sent and received. A communication system may include any number of transmitters, receivers, and/or transceivers for wireless communication. Communication can be one-way communication, so that data can be sent from one direction. For example, one-way communication may include that only the pan/tilt transmits data to the camera, or vice versa. One or more transmitters of the communication system may transmit data to one or more receivers of the communication system, and vice versa. Optionally, the communication may be two-way communication, so that data can be transmitted in both directions between the pan/tilt and the camera. Two-way communication includes that one or more transmitters of the communication system can send data to one or more receivers of the communication system, and vice versa. Control data from the pan/tilt can control the operation of the camera or other image capture device it carries (capture still or moving images, zoom, turn on or off, switch imaging modes, change image resolution, change focus, change depth of field, change exposure time, changing the viewing angle or field of view).

In the scenario where the movable platform is equipped with a PTZ, the movable pan can also be connected to the control terminal. In some embodiments, the control terminal can be connected to the PTZ or the photographing device, and the control terminal can connect to one of the PTZ and the photographing device. or more provide control commands, and receive information from one or more of the pan/tilt and the camera (such as the position and/or motion information of the carrier or the camera, data sensed by the camera, such as the image data of the camera) . In some embodiments, the control data of the control terminal may include instructions on position, motion, braking, or control of the pan/tilt and/or camera. For example, the control data may cause a change in the position and/or orientation of the carrier. The control data of the control terminal can control the operation of the camera or other image capture devices (capturing still or moving images, zooming, turning on or off, switching imaging modes, changing image resolution, changing focal length, changing depth of field, changing exposure time, changing viewing angle or field of view). In some embodiments, communications to the pan/tilt and/or camera may include information from one or more sensors. Communication may include sensory information transmitted from one or more different types of sensors, such as GPS sensors, motion sensors, inertial sensors, proximity sensors, or image sensors. The sensing information is about the position (eg, orientation, position), motion, or acceleration of the gimbal and/or the camera. The sensing information transmitted from the camera includes data captured by the camera or the status of the camera. The control data transmitted and provided by the control terminal can be used to control the state of one or more of the pan/tilt or the photographing device. Optionally, one or more of the pan/tilt and the photographing device may include a communication module for communicating with the control terminal, so that the control terminal can communicate or control the pan/tilt and the photographing device individually. The control terminal may be a remote controller of the PTZ, or may be an intelligent electronic device such as a mobile phone, an iPad, a wearable electronic device, etc., which can be used to control the PTZ.

It should be noted that the control terminal can be far away from the gimbal to realize remote control of the gimbal, and can be fixed or detachable on the gimbal, and can be set as required.

In some embodiments, the pan-tilt can communicate with other remote devices other than the control terminal, or with remote devices other than the control terminal. The control terminal can also communicate with another remote device and PTZ. For example, the pan/tilt and/or the control terminal may communicate with another movable platform or a carrier or a camera of another movable platform. When desired, the additional remote device may be a second terminal or other computing device (eg, a computer, desktop, tablet, smartphone, or other mobile device). The remote device may transmit data to the pan/tilt, receive data from the pan/tilt, transmit data to the control terminal, and/or receive data from the control terminal. Optionally, the remote device may be connected to the Internet or other telecommunication network to allow data received from the pan-tilt and/or control terminal to be uploaded to a website or server.

FIG. 2 is an application scenario of a shooting control method, a shooting control device, and a pan/tilt assembly provided by another embodiment of the present application. As shown in FIG. 2 , a gimbal capable of carrying a third-party photographing device is taken as an example for description. It should be noted that the photographing device and the motor may also be integrated with the gimbal, which is not limited here.

The novice photographer on the right in Figure 2 shoots the image of the user on the left in Figure 2 by holding the gimbal. Because the novice photographer is not skilled enough in shooting, it is easy to shake involuntarily during the shooting process. This easily results in an unclear image of the user on the left in the captured image, which fails to achieve the desired effect of the user.

As shown in FIG. 2 , in the embodiment of the present application, a novice shooting user can drive the lens to follow focus with the help of a gimbal and a motor set on the gimbal, such as determining the focal length change based on the position change detected by the gimbal, The drive motor compensates for the change in the focal length to achieve a follow focus effect. For example, the position change amount of the photographing device detected by the gimbal may be determined through measurement information output by the IMU. The amount of change in focus may be determined based on the amount of change in the position of the photographing device. The corresponding relationship between the focal length change amount and the position change amount of the pan/tilt head may be determined by calculation and/or calibration.

FIG. 3 is a schematic flowchart of a shooting control method provided by an embodiment of the present application.

As shown in FIG. 3 , the photographing control method may include operations S302 to S304.

In operation S302, the amount of position change of the photographing device in a specific direction is determined through the measurement information of the accelerometer in the inertial measurement unit.

Wherein, the photographing control method can be used in a pan/tilt head, where the pan/tilt head carries a photographing device and an inertial measurement unit, and the inertial measurement unit (IMU) is used to measure the attitude information of the photographing device. The shooting control method can be used in a shooting device, such as an IMU in the shooting device. The shooting control method can be used for a motor, for example, the motor has an IMU or the motor is set on the gimbal (to receive measurement data from the IMU on the gimbal). IMU is a device that measures the three-axis attitude angle (or angular rate) and acceleration of an object. A three-axis gyroscope and a three-axis accelerometer can be set in the IMU, and the attitude of the object can be calculated by measuring the angular velocity and acceleration of the object in three-dimensional space.

For example, the inertial measurement unit may include a gyroscope and/or an accelerometer, the gyroscope may be used to determine the angular acceleration information of the gimbal, and the accelerometer may be used to determine the acceleration information of the gimbal. Since the photographing device and the gimbal can be fixed to each other, the angular acceleration information and acceleration information of the gimbal can be used to represent the attitude information and acceleration information of the photographing device.

Taking the hand-held gimbal as an example, the IMU is used to calculate the posture of the camera mounted on it, and to perform stabilization and follow-up control. In addition to solving the attitude of the camera, by integrating the acceleration twice, the original IMU of the gimbal can also be used to calculate the displacement.

In one embodiment, the specific direction may refer to the direction of the line connecting the photographing device and the photographing object. In this case, the position change amount of the photographing device in the specific direction may be used as the position changing amount between the photographing device and the photographing object. In one embodiment, the specific direction may refer to the direction of the line connecting the photographing device and the photographing object at a specific historical moment, for example, after the specific historical moment, the photographing device has undergone a lateral displacement relative to the photographing object. If moving, the moving distance of the photographing device can be projected to the direction of the connection line between the photographing device and the photographing object at the certain historical moment, to obtain the position change amount of the photographing device relative to the photographing object in a specific direction.

In operation S304, the photographing apparatus is controlled to perform a follow focus or zoom operation according to the position change amount.

In this embodiment, the position change amount can be used as the object distance change amount of the shooting object relative to a certain lens, and the image distance change amount for the certain lens can be determined based on the object distance change amount, and then based on the distance change amount Control the photographing device to perform a follow focus operation to achieve a follow focus effect; or control the photographing device to perform a zoom operation based on the distance change amount to achieve an effect such as Dolly Zoom.

It should be noted that the photographing object may be stationary. For example, when the photographing object is stationary, the change amount of the distance between the photographing object relative to the photographing device may be determined only by relying on the IMU to calculate the position change of the photographing device. In addition, the photographing object may also be moving. In this case, the change amount of the photographing object relative to the distance between the photographing devices determined in other ways may be optimized based on the position change amount of the photographing device calculated by the IMU.

For example, in the related art, the amount of change of the distance between the photographing object relative to the photographing device can also be determined in the following manner. Taking the focus scene as an example, the target ranging can be measured by Time of Flight (TOF), the focus position can be searched by screen contrast (CDAF), or the diagonal position can be measured and estimated by PDAF pixels. , to achieve focus control.

An exemplary description is made by taking a scene in which the subject is moving as an example.

TOF needs to be equipped with a dedicated distance detection device, which is costly; some PTZs are not equipped with this device, and a corresponding distance detection device needs to be added; the frame rate of TOF is low, such as 10fps, which may cause some key frames to be missing from the camera The relative distance change to the target cannot meet the needs of follow focus or zoom scenarios.

The CDAF method has the same refresh rate as the screen, but consumes more computing resources; requires multi-frame convergence; and has obvious breathing effects, which cannot well meet the operational requirements for follow focus or zooming in some scenarios.

The PDAF method has the same refresh rate as the screen, but has certain requirements for light intensity, which cannot meet the needs of follow-focus or zoom scenarios when the light is poor.

The embodiments of the present application can optimize the ranging effect in some scenarios, such as TOF, CDAF, or PDAF, based on the position change amount of the photographing device calculated by the IMU.

For example, first, the focusing system achieves accurate focusing through TOF or CDAF or the like. On the basis of successful focusing, it is assumed that the target does not move, or the movement of the photographer is the main moving factor. At this time, use IMU or IMU and Visual Inertial Odometry (Visual Inertial Odometry, referred to as VIO) to obtain high frame rate camera position changes quantity.

The calculated camera position change is used to estimate the relative distance change between the camera and the target, and used to fine-tune the results of the focusing system (such as the upper part of the key frame that lacks the relative distance change between the camera and the target). A frame or the latest relative distance change is fine-tuned to predict the relative distance change between the camera and the target corresponding to the key frame.

This makes it possible to predict and control the diagonal position between keyframes without distance observations.

In one embodiment, controlling the camera to follow focus or zoom according to the position change amount includes: controlling a motor engaged with the lens of the camera device to drive the lens to follow focus or zoom according to the position change amount.

In order to facilitate the understanding of the embodiments of the present application, the principle of lens zoom is exemplarily described below.

FIG. 4 is a schematic diagram of adjusting the focal length by changing the distance between lenses according to an embodiment of the present application.

As shown in Figure 4, a zoom lens is generally composed of 3-4 groups of lens groups (such as L1, L2 and L3). In this embodiment of the present application, the zoom ring or the follow focus ring can be rotated through the motor output torque to drive the movement of the lens groups. A change in position will change the equivalent focal length of the lens as a whole. Depending on the optical design of the lens, the moving lens groups may be different, and the relationship between the moving distance of the lens group and the change of the focal length may be different. In this case, only the mapping relationship needs to be changed, and this embodiment is also applicable. As shown in FIG. 4 , by moving the middle concave lens L2 to change the focal length f, the focal length f and the position x of the lens L2 can be considered as a linear relationship, as shown in formula (1).

x=k ₁ *f+b ₁ Formula (1)

Among them, k ₁ and b ₁ are undetermined constants related to the mechanical structure and lens group parameters. The motor drives the zoom ring to rotate or drives the follow focus ring to rotate through gear meshing.

The above-mentioned mapping relationship may be obtained by calibration and/or calculation.

In one embodiment, controlling the motor engaged with the lens of the photographing device to drive the lens to change the focal length according to the position change amount includes: controlling the motor engaged with the lens of the photographing device to drive the lens to change the focal length according to the position change amount and the mapping relationship, wherein, The mapping relationship represents the relationship between the amount of position change and the rotor position of the motor.

FIG. 5 is a schematic diagram of a lens provided by an embodiment of the present application.

As shown in Figure 5, the lens has threads inside, and is provided with a focus ring and a zoom ring to adjust the front and rear distances of the lens group of the lens. The focus or follow focus of the lens is generally to keep the position of the photographed object and the position of the focal plane of the camera body of the photographing device unchanged, and the follow focus ring or the zoom ring of the lens of the photographing device is rotated by a motor. In the process of follow focus, the thread on the inner wall of the lens converts the rotation angle of rotation into the front and rear translation distance of the lens group of the lens, which is equivalent to using follow focus when the real-time distance between the shooting device and the subject remains unchanged. The camera motor drives the lens to rotate, and adjusts the object distance d and image distance f to make the shooting picture in focus. That is, during the focusing process, there is a one-to-one correspondence between the real-time distance between the shooting device and the subject and the target rotational position of the rotor of the follow focus motor, and the target rotational position of the rotor of the follow focus motor and the lens of the lens. The position of the group also has a one-to-one correspondence. According to the distance change between the shooting device and the object, the target rotation position of the rotor of the motor corresponding to the distance change can be determined, and then the rotor of the motor can be rotated. To the target rotation position, the lens is moved to the target position to achieve focusing, so that the subject is in a state of focus in the shooting screen of the shooting device. During the zooming process, the process of adjusting the zoom ring by the motor is similar to the above-mentioned focusing process, and will not be repeated here.

In addition, the lens may also include an aperture ring for adjusting the aperture. It should be noted that the motor for driving the follow focus ring and the motor for driving the zoom ring can be set at different positions on the gimbal, for example, on both sides of the photographing device, so as to achieve an even distribution of the load.

FIG. 6 is a schematic diagram of a flange provided in an embodiment of the present application.

As shown in FIG. 6 , taking the follow focus ring as an example, the manner in which the follow focus ring drives the movement of the lens may be the cam barrel structure as shown in FIG. 6 .

The focus motor rotor position motor_position and the focal length f have a linear relationship, as shown in formula (2).

motor_position=k ₂ *f+b ₂ Formula (2)

Among them, k ₂ and b ₂ are constants related to the structural parameters of the cam cylinder and the gear reduction ratio of the motor. The focal length of the lens can be controlled by controlling the motor rotor position motor_position.

In theory, the velocity can be obtained by integrating the acceleration data, and the displacement can be obtained by integrating again. If an IMU with good performance is used, an accurate displacement can be directly obtained for zoom control, while low-cost accelerometers inevitably have problems such as drift error and bias wandering, and the displacement calculation will be inaccurate.

In this embodiment, there is a linear relationship between the preset focal length f, the motor rotor position motor_position, and the distance d from the camera to the shooting object, and the coefficient of the linear relationship is determined by calibration before shooting. The process will be scaled to the coefficients of the linear map. The distance obtained by integrating the acceleration twice does not need to be completely accurate, but only needs to change linearly, and the tolerance to the sensor signal is high.

The principle of distance calculation is given below. Through vector rotation, the acceleration value measured in the sensor coordinate system is rotated to the geographic coordinate system, as shown in formula (3).

in,

is the rotation matrix of the sensor system to the geographic system.

is the sensor measurement vector under the sensor system.

is the sensor measurement vector under the geographic system. When the motion only includes translation, subtract the gravitational acceleration g to obtain the translational acceleration.

When the accelerometer starts to integrate from rest, the initial velocity vel(0)=0, and the moving velocity is shown in equation (4).

Among them, acc is the acceleration of translation, vel is the moving speed of the camera, and the distance from the camera to the object can be expressed as shown in formula (5).

d=Δd+d ₀ Formula (5)

Among them, Δd is the position change,

d ₀ is the distance from the camera to the subject when the integration starts, and d is the distance from the camera to the subject. Δd can be calculated in real time through acceleration data, but the initial distance is related to the shooting scene.

Figure 7 is a schematic diagram of the implementation principle of dolly zoom.

As shown in Figure 7, on the basis of ensuring real-time and accurate focus, dolly zoom smoothly zooms according to the change in the distance between the camera and the subject, so that the subject is fixed in the picture, but the background can be continuously zoomed. On the basis of accurate follow focus, the implementation principle of dolly zoom in this embodiment is shown in FIG. 7 .

As shown in Figure 7, in the process of continuous change of the object distance d, on the basis of ensuring the accurate follow-focus of the picture, real-time zooming is performed according to the object distance d, so that the size of the object image width frame_width in the camera sensor remains unchanged, while the two edges The background is continuously scaled.

The object distance d can be obtained by integrating the acceleration obtained by the IMU twice.

For example, the expression of the image distance f that can be obtained from similar triangles can be as shown in Equation (6).

Therefore, the motor rotor position motor_position can be expressed as shown in equation (7).

Since, d=Δd+d ₀ , the motor rotor position motor_position can be expressed as shown in formula (8).

In a specific shooting, the two terms in the parentheses of the above formula are constants, denoted as k ₃ , b ₃ , as shown in formula (9).

Based on the above calculation process, the motor rotor position motor_position can be expressed as shown in equation (10).

motor_position=k ₃ *Δd+b ₃ Formula (10)

Among them, Δd is obtained by real-time double integration according to the acceleration data.

Based on the above-mentioned implementation principle of dolly zoom, the embodiment of the present application provides a calibration process as shown below.

In one embodiment, there is a linear relationship between the amount of position change and the focal length of the lens. Correspondingly, determining the mapping relationship includes at least one of the following.

First, the first mapping relationship between the rotor position of the motor and the position change amount is determined. For example, after calibrating the first mapping relationship between the rotor position of the motor and the position change amount, the first mapping relationship is stored.

Second, determine the second mapping relationship between the rotor position of the motor and the focal length change amount, and then calculate the rotor position of the motor corresponding to each position change amount based on the second mapping relationship, the image size of the photographed object, and the size of the photographed object.

In order to facilitate the understanding of the embodiments of the present application, an example will be given below for the calibration process of the motor. Since the motor can be arranged on the pan/tilt head, the structure of the pan/tilt head will be exemplified first.

FIG. 8 is a schematic structural diagram of a pan/tilt according to an embodiment of the present application.

As shown in FIG. 8 , the pan/tilt head may include: a bracket assembly and at least one motor. Wherein, the bracket assembly may include at least two relatively movable bracket parts, and the bracket assembly is used for supporting the photographing device. The at least one motor is respectively used to drive the corresponding bracket parts to move, so as to adjust the posture of the photographing device.

For example, a pitch axis motor and a pitch axis arm cooperate to drive the camera to rotate about the pitch axis. The roll axis motor cooperates with the roll axis arm to drive the photographing device to rotate around the roll axis. The yaw axis motor cooperates with the yaw axis arm to drive the photographing device to rotate about the yaw axis.

Among them, the pitch axis motor can drive the movement of the pitch axis arm, the roll axis motor can drive the movement of the roll axis arm, and the yaw axis motor can drive the movement of the yaw axis arm.

For example, the yaw axis arm may be connected to one end of the roll axis arm, and the other end of the roll axis arm may be connected to the pitch axis arm, but the embodiment of the present application is not limited to this, the yaw axis arm, the roll axis arm and the pitch axis arm The axle arms can also be connected in other sequences.

It should be understood that, the pan/tilt head can also enable the photographing device to rotate around only one, two or four axes, etc., which is not limited herein.

It should be noted that, when the pan/tilt head can also drive the photographing device to move in translation, the embodiments of the present application do not exclude that the power components (such as air cylinders, liquid cylinders, linear motors, etc.) that drive translation correspond to any of the above tracking modes. At least one motor combined embodiment.

In one embodiment, the photographing device is detachably carried on the bearing seat of the gimbal; and/or the inertial measurement unit is provided on the gimbal and/or the photographing device; and/or the motor is detachably carried on the gimbal on the bearing seat.

For example, in order to facilitate the user to take pictures in a horizontal or vertical mode, a photographing device (such as a camera) may be fixed on the pan/tilt horizontally or vertically by a photographing device fixing mechanism. Referring to FIG. 8 , the photographing device is laterally placed on the pan/tilt by the photographing device fixing mechanism.

Wherein, the fixing mechanism of the photographing device can rotate relative to one or more shaft arms. For example, the fixing mechanism of the photographing device includes a rotating arm that can rotate relative to the tilt axis and a fixing portion that can cooperate with the photographing device. For example, the fixing part can move linearly with respect to the rotating arm, so as to fix photographing devices of different sizes or different configurations. It should be noted that the fixing mechanism of the photographing device may be a separate component or a part of a certain axis arm. For example, the fixing mechanism of the photographing device may be a component of the pitch axis arm or the yaw axis arm, which is not limited here.

For example, when the camera needs to be loaded on the platform, the camera can be fixed on the rotating arm first, and the position of the fixed part can be adjusted so that the fixed part can be matched with the positioning part of the camera, and then the camera can be fixed. In a designated position, the photographing device is set on the photographing device fixing mechanism.

Wherein, the IMU can be set anywhere on the gimbal to determine the attitude information of the components supported by the set inertial measurement unit. For example, in order to facilitate the determination of the attitude information of the photographing device, the inertial measurement unit may be arranged on the fixing mechanism of the photographing device, and is used to measure the attitude information of the fixing mechanism. For another example, in order to facilitate the determination of the attitude information of the shaft arm, the inertial measurement unit may be arranged on the shaft arm. The inertial measurement unit may be at least one of an accelerometer or a gyroscope, and may be used to measure the attitude and acceleration of the photographing device.

In one embodiment, the fixing mechanism of the photographing device can also be used to carry a motor, such as a zoom motor and a follow focus motor. For example, the zoom motor and the follow focus motor are respectively arranged on the fixing mechanism of the photographing device, and are located on both sides of the photographing device. For example, the shaft of the zoom motor is engaged with the zoom ring of the lens, and the shaft of the follow focus motor is engaged with the follow focus ring of the lens.

In one embodiment, the photographing device is detachably carried on the bearing seat of the pan/tilt, the pan/tilt is communicatively connected to the photographing device, and a display screen is provided on the hand-held part of the pan/tilt, and the display screen can display the photographing image of the photographing device.

For example, the PTZ may be provided with components such as a display screen, a lever, and a dial to facilitate human-computer interaction between the user and the PTZ. Among them, the human-computer interaction interface can be displayed on the display screen. Track wheels include but are not limited to focus wheels and/or zoom wheels.

FIG. 9 is a schematic diagram of a specific direction provided by an embodiment of the present application.

As shown in FIG. 9 , the specific direction is the direction of the connecting line between the photographing device and the photographing object when the mapping relationship is calibrated.

The following is an exemplary description of the calibration processes of the zoom motor and the follow focus motor, respectively, in the above-mentioned first calibration manner.

In one embodiment, the embodiment of the present application requires the user to calibrate the parameters k ₃ and b ₃ in the formula (10) before implementing the dolly zoom, and needs to calibrate 2 or more different distance points, and the image is displayed in which in the screen Display the scale ruler, adjust the zoom ring of the lens at each distance point, so that the subject at each distance point coincides with the same position of the scale ruler in the screen, so as to ensure that the image width of the subject is unchanged in the camera sensor. . Alternatively, adjust the lens follow focus ring at each distance point to get a sharper image of the subject.

For example, record the rotor position motor_position of the motor at each distance point, and the object distance change Δd obtained by IMU integration, and use the method of parameter fitting to obtain k ₃ and b ₃ . After the calibration is completed, it can be obtained according to the IMU double integration. The object distance change Δd is calculated in real time, and the motor rotor position motor_position required for zooming is calculated in real time, so that the zoom motor position is used to drive the lens in a closed loop to complete the zoom. Combined with real-time follow focus, the shooting effect of dolly zoom can be achieved.

In one embodiment, for a zoom motor, determining the first mapping relationship between the rotor position of the motor and the position change amount may include the following operations.

First, a first preset proportion is determined, and the first preset proportion is the proportion of the image of the photographing object in the photographing screen of the photographing device. Specifically, the proportion of the image of the photographing object in the photographing screen of the photographing device is determined based on the first user operation. For example, the user may input the first user operation by the user through the interactive interface displayed on the display screen. For example, the user may input the first user operation through a lever or a wheel or the like.

Then, when the photographing device moves a first distance in a specific direction, a first zoom adjustment instruction is obtained, where the first zoom adjustment instruction is used to control the zoom motor to drive the lens to change the focal length, so that the image of the photographed object occupies an image of the photographing device. The ratio is at a first preset ratio, resulting in a first rotor position.

Next, when the photographing device moves a second distance in a specific direction, a second zoom adjustment instruction from the user is obtained, and the second zoom adjustment instruction is used to control the zoom motor to drive the lens to change the focal length, so that the image of the photographed object is displayed on the photographing screen of the photographing device. The middle ratio is at the first preset ratio, and the second rotor position is obtained.

A first mapping relationship is determined based on the first distance, the first rotor position, the second distance, and the second rotor position.

FIG. 10 is a schematic diagram of a zoom motor provided by an embodiment of the present application.

As shown in Fig. 10, the zoom motor is arranged on one side of the manual lens of the photographing device, and the zoom motor is engaged with the zoom ring of the lens. The zoom motor can be fixed on the photographing device fixing mechanism.

FIG. 11 is a flowchart of a calibration process for a zoom motor provided by an embodiment of the present application. FIG. 12 is a schematic diagram of a calibration process for a zoom motor provided by an embodiment of the present application.

As shown in FIG. 11 and FIG. 12 , at the position before the camera moves, the zoom-related parameters are read. Then, the camera moves to the position after the camera moves, receives measurement information from the IMU, and can calculate the object distance change amount based on the measurement information. Then, as shown in Figure 12, the FOV of the camera changes. In order to keep the FOV of the camera consistent with the FOV before the camera moves, the focal length of the camera can be adjusted by the motor, so that the information such as the rotor position of the motor can be obtained. Then, the above operations are repeated to obtain at least two sets of correspondences between the object distance change and the rotor position. In order to calculate k ₃ , b ₃ based on the method shown in Equation (10).

In one embodiment, for a follow focus motor, determining the first mapping relationship between the rotor position of the motor and the position change amount includes the following operations.

First, when the photographing device moves a third distance in a specific direction, the user's first focus adjustment instruction is obtained, and the first focus adjustment instruction is used to control the follow focus motor to drive the lens to follow focus, so that the image of the subject is captured during the shooting. The photographed picture of the device is in focus state, and the third rotor position is obtained.

Then, when the photographing device moves a fourth distance in a specific direction, a second focus adjustment instruction from the user is obtained, and the second focus adjustment instruction is used to control the follow focus motor to drive the lens to follow focus, so that the image of the photographed object is captured during the shooting. The photographed picture of the device is in the in-focus state, and the fourth rotor position is obtained.

Next, a first mapping relationship is determined based on the third distance, the third rotor position, the fourth distance, and the fourth rotor position.

FIG. 13 is a schematic diagram of a follow focus motor provided by an embodiment of the present application.

As shown in FIG. 13 , the follow focus motor can be arranged on one side of the manual lens of the photographing device, and the follow focus motor is engaged with the follow focus ring of the lens. The follow focus motor can be fixed on the fixing mechanism of the photographing device.

FIG. 14 is a flowchart of a calibration process for a follow focus motor according to an embodiment of the present application. FIG. 15 is a schematic diagram of a calibration process for a follow focus motor according to an embodiment of the present application.

As shown in FIG. 14 and FIG. 15 , at the position before the camera moves, the focus-related parameters are read. Then, the camera moves to the position after the camera moves, receives measurement information from the IMU, and can calculate the object distance change amount based on the measurement information. Next, as shown in Figure 15, the image clarity of the target object in the image captured by the camera deteriorates. In order to make the image clarity of the target object meet user requirements, the focus of the camera can be adjusted by the focus motor to make the image of the target object clearer. In this way, information such as the rotor position of the follow focus motor can be obtained. Then, the above operations are repeated to obtain at least two sets of correspondences between the object distance change and the rotor position. In order to calculate k ₃ , b ₃ based on the method shown in Equation (10).

It should be noted that the zoom motor and the follow focus motor can also be calibrated at the same time.

For example, after the calibration process is started, the subject is placed in a designated area of the camera, such as a central area, by moving the mirror to determine the current position a of the photographing device.

Next, control the zoom ring by rotating the zoom motor to realize the zooming of the subject. Combined with the scale on the camera liveview, determine the size ratio of the current subject, mark the zoom motor parameters of the current position a, and drive the follow focus ring to rotate through the follow focus motor. , make the focal plane of the current position a clear, and mark the parameters of the follow focus motor at the current position a. Determine the current distance change c measured by the IMU.

Then, move to position b and place the subject in the designated area of the camera by moving the lens.

Then, by rotating the zoom motor rotor to drive the zoom ring, the zooming of the shooting object is realized. Combined with the scale ruler on the live view of the camera, the size ratio of the current shooting object is determined, and the zoom motor parameters of the current position b are marked. The rotor of the follow focus motor drives the follow focus ring, so that the focal plane of the current position b is clear, and the parameters of the follow focus motor at the current position b are marked. Determine the change amount d of the distance measured by the current IMU.

In this way, k ₃ , b ₃ as shown in equation (10) can be obtained according to the above-determined zoom motor parameters, follow focus motor parameters, distance change c and distance change d, etc. combined with the parameter fitting method. After the motor parameter calibration is completed, by moving the mirror between position a and position b, it can ensure that the size of the main body of the picture remains unchanged, and the depth of field changes the dolly zoom shooting effect.

In one embodiment, after the above-mentioned calibration process is completed, controlling a motor engaged with the lens of the photographing device to drive the lens to follow focus or zoom according to the position change amount includes at least one of the following: controlling the lens of the photographing device according to the position change amount The engaged zoom motor drives the zoom of the lens, so that the proportion of the image of the subject in the shooting frame of the shooting device remains stable. Alternatively, a follow focus motor engaged with the lens of the photographing device is controlled according to the position change amount to drive the lens to follow focus, so that the photographing object is in a state of focus in the photographing picture of the photographing device.

In one embodiment, in order to ensure the accuracy of the mapping relationship and ensure user experience, the above method further includes: when a re-calibration condition of the mapping relationship is satisfied, outputting calibration prompt information or forcibly terminating the use of the mapping relationship. Due to the limited detection accuracy of the IMU, or due to the influence of the IMU's own characteristics (for example, the detection accuracy for large acceleration scenarios is lower than the detection accuracy for small acceleration scenarios), the applicable scenarios of the mapping relationship for one calibration are limited. Therefore, the user can be prompted to re-calibrate in time to determine the accuracy of the calibrated mapping relationship.

For example, the recalibration condition includes at least one of the following: the inertial measurement unit has undergone a power-off operation, the gimbal has experienced a power-off operation, the displacement of the gimbal in the power-off state exceeds the recalibration displacement threshold, or the acceleration of the inertial measurement unit is greater than a set value. Fixed acceleration threshold.

In an embodiment, the above-mentioned mapping relationship may also be determined by calculation. For example, there is a linear relationship between the amount of position change and the focal length of the lens. When the mapping relationship is determined by calculation, the calculation parameters involved in the calculation include: the image size of the shooting object, the size of the shooting object, the object distance at the starting position, and the flange distance , among which, the flange distance is related to the structural parameters of the cam barrel and the gear reduction ratio of the motor.

In one embodiment, the above method may further include the following operations. If the angle between the connection direction between the photographing device and the photographed object and the specific direction satisfies the simplified condition, the position change amount determined based on the measurement information of the accelerometer in the inertial measurement unit is used as the position of the photographing device in the specific direction The amount of change.

FIG. 16 is a schematic diagram of an included angle satisfying a simplified condition provided by an embodiment of the present application.

As shown in Figure 16, after the calibration is completed, the user moves the gimbal in the direction of approaching the subject to achieve a shooting effect such as dolly zoom. The user may not be able to move in a specific direction at the time of calibration (eg lack of reference control or the user wishes to change the angle of the shot, etc.). In this scenario, if the angle between the user's moving direction and the specific direction during calibration is small, in order to reduce the consumption of computing resources, etc., the position change determined based on the measurement information can be used as the shooting device in the specific direction. position change amount.

In the shooting control method of the embodiment of the present application, although the zoom optical structure design of different lenses may be different, common lenses can be processed according to the technical solutions of the present application. In addition, the variable distance and the position of the focus ring of some lenses are not necessarily linearly related. It is only necessary to bring in the nonlinear model and fit the corresponding parameters, which has a wide range of applications.

The content related to the human-computer interaction mode is exemplified below.

In one embodiment, the proportion of the image of the target object in the captured image may be input in the following manner.

For example, a display screen is provided on the PTZ, and a user interaction interface is displayed on the display screen. Correspondingly, the first user operation includes an operation for the user to input a proportion in the user interaction interface, wherein the proportion of the user input in the user interaction interface is It is based on the preset shooting control method, shooting control device and pan/tilt assembly. For example, the user can make the image of the photographed object reach a desired proportion in the photographed image through a zoom operation. For example, the user can enter a percentage by entering a percentage value.

For example, the first user operation includes an operation in which the user inputs the percentage and the base point in the user interaction interface. In this way, it is convenient for the user to achieve the required proportion of the image of the photographed object in the photographed image in the designated area.

For example, the first user operation includes an operation of the user inputting at least two base points in the user interaction interface.

In one embodiment, the photographing object of the photographing device is determined based on a second user operation of the user with respect to the user interface. For example, the user changes the posture of the pan/tilt head so that the photographed object is located in the designated area of the photographed image.

In one embodiment, the photographed object of the photographing device is determined through image recognition. For example, after a user inputs a photographic subject, the photographing device automatically recognizes the image of the photographing subject from the photographed image. For example, the user inputs the object to be photographed through a click operation or the like.

In one embodiment, the photographing device is set on a pan/tilt head, and a focus wheel and/or a zoom wheel are provided on the hand-held part of the pan/tilt head. Correspondingly, the above method further includes: receiving a third user operation on the focus wheel and/or the zoom wheel, so as to control the motor engaged with the lens of the photographing device to drive the lens to follow focus or zoom according to the position change amount. For example, the focus wheel and/or the zoom wheel may be arranged on the hand-held part of the pan/tilt head, the control terminal of the movable platform, and the like.

Another aspect of the present application also provides a photographing control device. The photographing control device can be set on any one of the pan/tilt, the motor or the photographing device.

In the following, the shooting control device is set in the PTZ for exemplary illustration.

FIG. 17 is a block diagram of a pan/tilt provided by an embodiment of the present application.

The pan/tilt 1700 carries a camera and an inertial measurement unit, and the inertial measurement unit is used to measure the attitude information of the camera. As shown in FIG. 17 , the pan/tilt 1700 includes: at least one processor 1710 and a memory 1720 . Memory 1720 stores computer-executable instructions.

At least one processor 1710 executes the computer-executable instructions stored in the memory 1720, so that the following steps are implemented when the computer-executable instructions are executed.

For example, first, through the measurement information of the accelerometer in the inertial measurement unit, the amount of position change of the photographing device in a specific direction is determined. Then, the photographing device is controlled to perform a follow focus or zoom operation according to the position change amount.

Wherein, the gimbal may include a handheld gimbal, an airborne gimbal, and the like.

For example, a pan/tilt head can be used to mount on a movable platform with a powered system. The mobile platform is an unmanned aerial vehicle as an example to illustrate. The movable platform may include a powered mechanism, a sensing system. In addition, the movable platform may also include a communication system.

The power mechanism may include one or more rotating bodies, propellers, blades, engines, motors, wheels, bearings, magnets, and nozzles. For example, the rotating body of the powertrain may be a self-tightening rotating body, a rotating body assembly, or other rotating body power unit. The movable platform may have one or more power mechanisms. All powertrains can be of the same type or of different types. The power mechanism enables the movable platform to take off vertically from a surface, or to land vertically on a surface, without any horizontal movement of the movable platform (eg, without taxiing on a runway). For example, the movable platform may have a plurality of horizontal rotating bodies to control the lifting and/or pushing of the movable platform. The sensing system may include one or more sensors to sense surrounding obstacles, spatial orientation, velocity and/or acceleration (eg, rotation and translation with respect to up to three degrees of freedom) of the movable platform. For the communication system, please refer to the relevant part of the communication system of the PTZ, which will not be repeated here.

For another example, the hand-held pan/tilt head further includes: a holding component for supporting the bracket component. In addition to the function of supporting the bracket assembly, the holding assembly can also function such as accommodating batteries, processors, setting input/output components, etc., which are not limited herein.

In order to facilitate the interaction between the user and the PTZ, the PTZ may be provided with an input part and/or an output part.

For example, the input part may be used to input the user's operation instruction on the handheld pan/tilt head, and the input part may include a wheel and a control joystick for realizing human-computer interaction. The control rocker can control the movement of the pivot arm. For example, by turning the control rocker, the pivot arm of the handheld gimbal can be rotated in a corresponding direction.

Specifically, the component for realizing human-computer interaction may include a display screen for displaying an interactive interface, and a user can input control instructions in the interactive interface.

For example, the handheld PTZ may also include a display screen for displaying a user interface. The handle assembly is used to support the stand assembly. Or, the user interface is displayed on the display located on the photographing device; alternatively, the user interface is displayed on the display of the terminal device connected to the PTZ; or, the preset function keys (such as zoom wheel, follow focus wheel) are located on the holding component Above, the handle assembly is used to support the bracket assembly.

In addition, the PTZ may further include a status prompting component. For example, the pan/tilt head may include indicator lights.

For example, during the calibration process, the indicator light representing the calibration state is on. During the dolly zoom shooting process, the indicator light representing the dolly zoom shooting state is on.

It should be understood that, in addition to a dial wheel and a control rocker for realizing human-computer interaction, the input part may also have other components or parts, for example, a switch of a handheld pan/tilt head, etc. may be provided.

A processor may be provided in the input unit for processing input control commands, or sending and receiving signals. Of course, the processor can also be arranged in the handle assembly.

Optionally, the processor may be a central processing unit (Central Processing Unit, referred to as CPU), and the processor may also be other general-purpose processors, digital signal processors (Digital Signal Processor, referred to as DSP), application-specific integrated circuits (application specific integrated circuit, referred to as ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, referred to as FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The processing unit may be connected to non-volatile computer readable storage medium 1720 . The non-volatile computer-readable storage medium 1720 may store logic, code, and/or computer instructions 1721 executed by the processing unit for performing one or more steps. The non-volatile computer readable storage medium 1720 may include one or more storage units (removable media or external memory such as SD card or RAM). In some embodiments, the measurement information of the IMU may be directly transferred and stored in a storage unit of the non-volatile computer-readable storage medium 1720 . The storage units of the non-volatile computer-readable storage medium 1720 may store logic, code, and/or computer instructions 1721 executed by the processing unit to perform various embodiments of the various methods described herein. For example, the processing unit may be operable to execute instructions to cause one or more processors of the processing unit to perform the zoom and/or focus functions described above. In some embodiments, the storage unit of the non-volatile computer-readable storage medium 1720 may store processing results generated by the processing unit.

In some embodiments, the processing unit may be connected to the control module for controlling the state of the motor.

The processing unit may also be connected to the communication module for transmitting and/or receiving data with one or more peripheral devices (eg, terminals, display devices, or other remote control devices). Any suitable communication method may be utilized here, such as wired communication or wireless communication. For example, the communication module may utilize one or more local area networks, wide area networks, infrared, radio, Wi-Fi, peer-to-peer (P2P) networks, telecommunication networks, cloud networks, and the like. Optionally, a relay station, such as a signal tower, a satellite, or a mobile base station, can be used.

The input module on the PTZ may include one or more input mechanisms to obtain input generated by the user by operating the input module. Input mechanisms include one or more joysticks, switches, knobs, slide switches, buttons, dials, touchscreens, keypads, keyboards, mice, voice controls, gesture controls, inertial modules, and the like. The input module may be used to obtain user input for controlling aspects such as a pan/tilt head, motors, cameras, movable platform, or components thereof. Any aspect includes attitude, position, orientation, flight, tracking, etc. For example, the input mechanism may be that the user manually sets one or more positions, each position corresponding to a preset input.

In various embodiments, the input module may be executed by more than one device. For example, the input module can be implemented by a standard remote controller with a joystick. A standard remote controller with a joystick connects to a mobile device (eg, a smartphone) running a suitable application ("app") to generate control commands for the movable platform. The app can be used to get input from the user.

The processing unit may be connected to the memory. Memory includes volatile or non-volatile storage media for storing data, and/or logic, code, and/or program instructions executable by a processing unit for performing one or more rules or functions. The memory may include one or more storage units (removable media or external memory such as SD card or RAM). In some embodiments, the data input to the module may be directly transferred and stored in a storage unit of the memory. The storage units of the memory may store logic, code and/or computer instructions executed by the processing unit to perform various embodiments of the various methods described herein. For example, the processing unit may be configured to execute instructions to cause one or more processors of the processing unit to process and display sensory data (eg, images) obtained from cameras or motors, etc., and control commands, including motion commands, generated based on user input and target object information, and cause the communication module to transmit and/or receive data, etc. The storage unit may store sensed data or other data received from an external device such as a removable platform. In some embodiments, the storage unit of the memory may store the processing result generated by the processing unit.

The communication module of the PTZ can be used to transmit and/or receive data from one or more remote devices (eg, mobile platforms, base stations, etc.). For example, the communication module can transmit control signals (such as motion signals, target object information, and tracking control commands) to peripheral systems or devices, such as the above-mentioned PTZ and/or load. The communication module may include a transmitter and a receiver for receiving data from and transmitting data to the remote device, respectively. In some embodiments, the communication module may include a transceiver that combines the functions of a transmitter and a receiver. In some embodiments, the transmitter and receiver and the processing unit may communicate with each other. Communication may utilize any suitable means of communication, such as wired or wireless communication.

Images captured by the movable platform during motion can be transmitted from the movable platform or imaging device back to a control terminal or other suitable device for display, playback, storage, editing, or other purposes. Such transmission may occur in real-time or near real-time as the imaging device captures the imagery. Optionally, there may be a delay between the capture and transmission of the imagery. In some embodiments, the imagery may be stored in the removable platform's memory without being transferred anywhere else. The user can view these images in real time and, if necessary, adjust the target object information or adjust other aspects of the movable platform or its components. Adjusted target object information may be provided to the movable platform, and an iterative process may continue until a desired image is obtained. In some embodiments, the imagery may be transmitted from the camera and/or the control terminal to a remote server. For example, images can be shared on some social networking platforms, such as WeChat Moments or Weibo.

In one embodiment, controlling the camera to follow focus or zoom according to the position change amount includes: controlling a motor engaged with the lens of the camera device according to the position change amount to drive the lens to follow focus or zoom.

In one embodiment, controlling the motor engaged with the lens of the photographing device to drive the lens to follow focus or zoom according to the position change amount includes:

The motor engaged with the lens of the photographing device is controlled to drive the lens to follow focus or zoom according to the position change amount and a mapping relationship, wherein the mapping relationship represents the relationship between the position change amount and the rotor position of the motor. For example, the gimbal sends the position change amount to the motor, so that the motor can determine the rotor position of the motor according to the position change amount and the mapping relationship, so as to drive the lens to follow focus or zoom by adjusting the rotor position.

In addition, when the at least one processor executes the computer-executed instruction, the following steps may also be implemented: when the re-calibration condition of the mapping relationship is satisfied, output calibration prompt information or forcibly terminate the use of the mapping relationship.

For example, the recalibration conditions include at least one of the following: the inertial measurement unit has undergone a power-off operation; the gimbal has experienced a power-off operation; the displacement of the gimbal in the power-off state exceeds the recalibration displacement threshold; the acceleration of the inertial measurement unit is greater than the set value. Fixed acceleration threshold.

For the specific content, refer to the same part of the previous embodiment, which will not be repeated here.

In one embodiment, the specific direction is the direction of the line connecting the photographing device and the photographing object when the mapping relationship is calibrated.

The mapping relationship is obtained by calibration and/or calculation.

For example, there is a linear relationship between the position change amount and the focal length of the lens; determining the mapping relationship includes at least one of the following: determining a first mapping relationship between the rotor position of the motor and the position change amount; or; determining the rotor position and focal length of the motor A second mapping relationship between changes.

In this way, the rotor position of the motor corresponding to each position change amount can be calculated based on the second mapping relationship, the image size of the subject, and the size of the subject.

For another example, determining the first mapping relationship between the rotor position of the motor and the position change amount includes: for the zoom motor, determining a first preset proportion, where the first preset proportion is the image of the object to be photographed in the photographing screen of the photographing device When the photographing device moves a first distance in a specific direction, a first zoom adjustment instruction is obtained, and the first zoom adjustment instruction is used to control the zoom motor to drive the lens to change the focal length, so that the image of the photographed object can be captured by the photographing device. When the proportion in the picture is at the first preset proportion, the first rotor position is obtained; when the photographing device moves a second distance in a specific direction, a second zoom adjustment instruction from the user is obtained, and the second zoom adjustment instruction is used to control the drive of the zoom motor The lens changes the focal length, so that the proportion of the image of the photographed object in the photographed image of the photographing device is at the first preset proportion, and the second rotor position is obtained; based on the first distance, the first rotor position, the second distance and the second rotor The location determines the first mapping relationship.

For another example, determining the first mapping relationship between the rotor position of the motor and the position change amount includes: for the follow-focus motor, when the photographing device moves a third distance in a specific direction, acquiring the user's first focus-focusing adjustment instruction, the first The in-focus adjustment command is used to control the follow-focus motor to drive the lens to follow-focus, so that the image of the photographed object is in the in-focus state on the shooting screen of the photographing device, and the third rotor position is obtained; when the photographing device moves a fourth distance in a specific direction , obtain the user's second in-focus adjustment instruction, and the second in-focus adjustment instruction is used to control the follow-focus motor to drive the lens to follow-focus, so that the image of the photographed object is in the in-focus state on the shooting screen of the shooting device, and the fourth rotor is obtained. a position; a first mapping relationship is determined based on the third distance, the third rotor position, the fourth distance, and the fourth rotor position.

In one embodiment, there is a linear relationship between the position change amount and the focal length of the lens. When the mapping relationship is determined by calculation, the calculation parameters involved in the calculation include: the image size of the shooting object, the shooting object size, the starting position and the object distance and the flange distance, where the flange distance is related to the structural parameters of the cam barrel and the gear reduction ratio of the motor.

In one embodiment, controlling the motor engaged with the lens of the photographing device to drive the lens to follow focus or zoom according to the position change amount includes at least one of the following: controlling the zoom motor engaged with the lens of the photographing device according to the position change amount to drive the lens to zoom , so that the proportion of the image of the subject in the shooting screen of the shooting device remains stable; the follow focus motor engaged with the lens of the shooting device is controlled according to the position change to drive the lens to follow focus, so that the subject is in the shooting screen of the shooting device. is in focus.

In one embodiment, the proportion of the image of the photographing object in the photographing screen of the photographing device is determined based on the first user operation.

For example, the first user operation includes an operation in which the user inputs a proportion in the user interaction interface, wherein the proportion input by the user in the user interaction interface is based on a preset point; or the first user operation includes the user in the user interaction interface. The operation of inputting the percentage and the base point in the user interface; or the first user operation includes an operation of the user inputting at least two base points in the user interaction interface.

The rotating shaft of the zoom motor can be engaged with the zoom ring of the lens, and the rotating shaft of the follow focus motor can be engaged with the follow focus ring of the lens.

For example, the photographing object of the photographing device is determined based on the user's second user operation on the user interaction interface; or the photographing object of the photographing device is determined through image recognition.

In one embodiment, the photographing device is detachably carried on the bearing seat of the gimbal; and/or the inertial measurement unit is provided on the gimbal and/or the photographing device; and/or the motor is detachably carried on the bearing of the gimbal seat.

For example, the photographing device is detachably carried on the bearing seat of the gimbal, the gimbal is communicatively connected to the photographing device, and a display screen is provided on the hand-held part of the gimbal, and the display screen can display the shooting image of the photographing device.

In one embodiment, when the at least one processor executes the computer-executed instructions, the following steps are implemented: if the included angle between the direction of the line connecting the photographing device and the photographed object and the specific direction satisfies the simplified condition, then based on the inertial measurement unit The position change amount determined by the measurement information of the accelerometer is used as the position change amount of the photographing device in a specific direction.

When the photographing device is set on the gimbal, a focus wheel and/or a zoom wheel are arranged on the hand-held part of the gimbal.

Correspondingly, when the at least one processor executes the computer-executed instructions, the following steps are implemented: receiving a third user operation on the follow focus wheel and/or the zoom wheel, so as to control the camera device to follow focus or zoom based on the third user operation.

In one embodiment, the capture control device is provided in the motor. For example, the motor can be a follow focus motor or a zoom motor. Taking a zoom motor as an example, the zoom motor may include: a control circuit and a driving device. Wherein, the control circuit may include a processor and a storage medium. The storage medium may store computer instructions and mapping relationships (eg, a first mapping relationship, a second mapping relationship), and the like.

Another aspect of the present application also provides a pan/tilt assembly. FIG. 18 is a block diagram of a pan/tilt assembly provided by an embodiment of the present application.

As shown in FIG. 18 , the pan/tilt assembly may include a pan/tilt, a motor detachable from the pan/tilt, and a shooting control device disposed on one of the pan/tilt and the motor. The pan/tilt can be used to carry a photographing device. For example, the photographing device may be detachably arranged on the pan/tilt in a structurally fixed manner (eg, fixed by a photographing device fixing mechanism).

The motor is fixedly connected with the camera and the lens by the fixing mechanism of the shooting device, and the lens is driven by the reduction gear to realize zooming. The positional relationship between the camera, the lens and the IMU is fixed, so the distance change measured by the IMU can be considered to be consistent with the displacement of the camera.

An IMU can be set on the gimbal. For example, the IMU includes an accelerometer, thereby integrating acceleration information collected based on the accelerometer to obtain velocity information, and integrating the velocity information to obtain displacement information. The measurement information collected by the IMU may include the amount of position change of the photographing device corresponding to different sampling moments. The distance difference between the IMU and the photographing device is a fixed value, and the amount of change in the distance between the IMU and the photographing device may be used as the amount of change in the distance between the photographing object and the photographing device. For example, the reference plane of the IMU is located below the camera. The calculation processing of the position change amount may be performed by a controller in the IMU or by a controller in the motor, which is not limited in this embodiment.

Optionally, the camera fixing mechanism of the pan/tilt head can fix the motor to the bottom of the camera body, and after the lens is mounted on the camera body through the bayonet, the motor can drive the lens to rotate.

A follow focus wheel and/or zoom wheel can be set on the gimbal, and the follow focus wheel and/or zoom wheel can push the current rotation position and rotation speed data of the follow focus wheel or zoom wheel to the motor, so that the motor rotates to the same level as the follow focus wheel. the specified position corresponding to the wheel or zoom wheel. In the parameter calibration link in the embodiment of the present application, the focus wheel or the zoom wheel can be used to drive the motor to reach at least two specified focal lengths, so as to complete the change of the distance between the photographing device and the photographed object and the change of the rotor position of the motor. Calibration of the mapping relationship.

A display screen can be set on the gimbal, and during the follow focus or zoom process in the shooting control method provided by the embodiment of the present application, the user can also select a mode by inputting a control command on the display screen, and switch the automatic mode to the manual mode , to achieve follow focus or zoom by using the follow focus wheel or zoom wheel to control the position of the motor.

The motor is an actuator, including a control circuit (such as a position controller) and a driving device. The position controller is used for the control instructions given by the focus wheel or zoom wheel connected to the motor or the control given by the controller connected to the IMU. Command, output torque, drive the lens meshed with the motor to follow focus or zoom through gear transmission. Optionally, the follow focus motor is engaged with the follow focus ring of the lens in the photographing device, and/or the zoom motor is engaged with the zoom ring of the lens in the photographing device.

The photographing device includes a lens and a camera body, and is used for photographing a photographed object to obtain an image after the motor drives the lens to zoom or follow focus. The camera body can be a camera body of a handheld single-lens reflex or mirrorless camera. A lens is one that includes a zoom ring and a follow focus ring, like a manual lens. The lens in this embodiment can also support manual focus and follow focus.

It should be understood that the above naming of the components of the pan/tilt assembly is only for the purpose of identification, and should not be construed as a limitation on the embodiments of the present application.

Wherein, the photographing control device includes: at least one processor and a memory.

The memory stores instructions for execution by the computer.

At least one processor executes computer-implemented instructions stored in the memory such that at least some of the steps described above are performed when the computer-implemented instructions are executed.

It can be understood that, in some embodiments, the above steps may also be completed by the pan/tilt and the motor, for example:

In one embodiment, the gimbal determines the position change amount of the photographing device carried by the gimbal in a specific direction by using the measurement information of the accelerometer in the inertial measurement unit, the inertial measurement unit is arranged in the gimbal, and the inertial measurement unit is used to measure the shooting The attitude information of the device; the gimbal transmits the position change to the motor; the motor drives the camera to follow focus or zoom according to the position change.

For example, the motor is engaged with the lens of the camera, and the motor drives the lens to focus or zoom according to the amount of position change.

Specifically, the motor determines the rotor position according to the position change amount and a mapping relationship to drive the lens to follow focus or zoom, wherein the mapping relationship represents the relationship between the position change amount and the rotor position of the motor.

In one embodiment, when the re-calibration condition of the mapping relationship is satisfied, the PTZ outputs calibration prompt information or forcibly terminates the use of the mapping relationship.

Correspondingly, the mapping relationship is obtained by calibration and/or calculation.

Specifically, there is a linear relationship between the amount of position change and the focal length of the lens. Correspondingly, determining the mapping relationship includes at least one of the following: determining a first mapping relationship between the rotor position of the motor and the position change amount. Alternatively, first, determine the second mapping relationship between the rotor position of the motor and the focal length change amount; then, calculate the rotor position of the motor corresponding to each position change amount based on the second mapping relationship, the image size of the subject and the size of the subject.

For example, for the zoom motor, the gimbal obtains a first preset proportion, which is the proportion of the image of the photographed object in the photographing screen of the photographing device; the gimbal moves a first distance along a specific direction in the photographing device When the first zoom adjustment instruction is obtained, the first zoom adjustment instruction is used to control the zoom motor to drive the lens to change the focal length, so that the proportion of the image of the shooting object in the shooting screen of the shooting device is at the first preset proportion, and the first zoom adjustment instruction is obtained. Rotor position; when the camera moves a second distance in a specific direction, the pan/tilt obtains a second zoom adjustment instruction from the user, and the second zoom adjustment instruction is used to control the zoom motor to drive the lens to change the focal length, so that the image of the object can be captured in the camera. The ratio of the captured images is at the first preset ratio, and the second rotor position is obtained; the motor determines the first mapping relationship based on the first distance, the first rotor position, the second distance and the second rotor position.

For example, for the follow focus motor, when the camera moves a third distance in a specific direction, the gimbal obtains the user's first focus adjustment instruction, and the first focus adjustment instruction is used to control the follow focus motor to drive the lens to follow focus, so as to The image of the subject is in focus on the shooting screen of the shooting device, and the third rotor position is obtained; when the camera moves a fourth distance in a specific direction, the gimbal obtains the user's second focus adjustment instruction, and the second focus is achieved. The adjustment command is used to control the follow focus motor to drive the lens to follow focus, so that the image of the subject is in focus on the shooting screen of the shooting device, and the fourth rotor position is obtained; the motor is based on the third distance, the third rotor position, and the fourth rotor position. The distance and the fourth rotor position determine a first mapping relationship.

For example, there is a linear relationship between the amount of position change and the focal length of the lens. When the mapping relationship is determined by calculation, the calculation parameters involved in the calculation include: image size of the subject, size of the subject, object distance at the starting position, and flange distance , where the flange distance is related to the structural parameters of the cam barrel and the gear reduction ratio of the motor.

In one embodiment, the motor driving the lens to follow focus or zoom according to the position change includes at least one of the following: the motor drives the lens of the photographing device to zoom according to the position change, so that the image of the photographed object occupies a proportion of the photographing frame of the photographing device Keep it stable; or, the motor drives the lens to follow focus according to the amount of position change, so that the subject is in focus in the shooting screen of the shooting device.

For example, the proportion of the image of the photographing object in the photographing screen of the photographing device is determined based on the first user operation on the pan/tilt or the photographing device.

Specifically, the display screen of the gimbal or the photographing device can display the user interaction interface.

Correspondingly, the first user operation includes an operation of the user inputting a proportion in the user interaction interface, wherein the proportion input by the user in the user interaction interface is based on a preset point; or the first user operation includes the user interacting with the user. The operation of inputting the ratio and the base point in the interface; or the first user operation includes the operation of the user inputting at least two base points in the user interaction interface.

For example, the shaft of the zoom motor is engaged with the zoom ring of the lens, and the shaft of the follow focus motor is engaged with the follow focus ring of the lens.

For another example, the photographing object of the photographing device is determined based on the second user operation of the user on the user interaction interface; or the photographing object of the photographing device is determined through image recognition.

In one embodiment, the photographing device is detachably carried on the bearing seat of the gimbal; and/or; the motor is detachably carried on the bearing seat of the gimbal.

For example, if the angle between the connection direction between the photographing device and the photographed object and the specific direction satisfies the simplified condition, the position change amount determined by the motor based on the measurement information of the accelerometer in the inertial measurement unit is used as the photographing device in the specific direction. position change.

In one embodiment, the photographing device is set on a pan/tilt head, and a focus wheel and/or a zoom wheel are provided on the hand-held part of the pan/tilt head. Correspondingly, the pan/tilt head receives a third user operation on the focus wheel and/or the zoom wheel, so as to control the motor engaged with the lens of the photographing device to drive the lens to focus or zoom.

The embodiments of the present application can effectively improve the unevenness and discontinuity of the current camera controller follow focus control. It should be noted that, in this embodiment of the present application, a creative follow focus effect can also be achieved by using a preset focusing curve. In addition, combining existing modes such as fixed-point time-lapse and trajectory time-lapse to create time-lapse photography with focus changes, it provides users with more creative space.

As shown in Figure 19, taking the zoom ring as an example, the zoom control is performed by driving the movement of the manual lens zoom ring through gear meshing. The motor control system adopts a common vector control strategy, and uses a linear Hall sensor to feedback the position of the motor rotor. According to the position and current feedback The signal realizes the current closed loop, realizes the position closed loop control on the basis of the current loop and the speed loop, controls the rotation angle of the motor rotor and then controls the position of the zoom ring, and realizes the control of the focal length of the lens.

The target position in FIG. 19 is determined based on the position change amount of the photographing device in a specific direction and the mapping relationship. The amount of change in the position of the camera in a particular direction may be determined based on measurement information (eg, acceleration information) from the IMU.

The position loop controller determines the target speed based on the difference between the amount of change in position and the position feedback from the drive circuit feedback circuit. The speed loop controller determines the target current based on the difference between the target speed and the speed feedback from the drive circuit feedback circuit. The current loop controller determines a control signal based on the difference between the target current and the current feedback from the drive circuit feedback circuit, the control signal is used to control the motor (such as a permanent magnet synchronous motor) to drive the zoom ring or focus ring of the lens .

The following takes the handheld pan/tilt as an example to illustrate the execution subject of each of the above operations. For example, the execution subject of each of the above operations can be implemented by a handheld pan/tilt or a motor. Specifically, the input part of the handheld pan/tilt, a holding part, a processor arranged in the holding part, a motor carried by the pan/tilt, etc. Function.

For example, the operation of determining the position change amount of the photographing device carried by the gimbal in a specific direction may be implemented by the gimbal. The operation of driving the photographing device to follow focus or zoom according to the amount of position change may be implemented by a motor. The rotor position is determined according to the position change amount and the mapping relationship, and the operation of driving the lens to follow focus or zoom can be realized by a motor. Operations related to displaying information can be performed by the display on the PTZ.

Hereinafter, the execution subject of each of the above operations will be exemplarily described by taking a pan/tilt mounted on the mobile platform as an example.

Determining the position change amount of the photographing device in a specific direction may be determined by a pan/tilt, a motor, a movable platform (eg, a processor) or a control terminal of the movable platform (eg, a control terminal provided on a land robot).

Controlling the motor engaged with the lens of the photographing device to drive the lens to follow focus or zoom according to the position change amount may be performed by a pan/tilt or a motor.

Controlling the motor engaged with the lens of the photographing device to drive the lens to follow focus or zoom according to the position change amount and the mapping relationship may be performed by a motor or a pan/tilt head.

Outputting the calibration prompt information or forcibly terminating the use of the mapping relationship may be performed by the display screen or other output components on the PTZ or the control terminal.

The first user operation, the second user operation, and the third user operation may be received by the control terminal.

The operation of image processing and target object recognition can be determined by the control terminal of the movable platform, the pan/tilt, the load or the movable platform.

It should be noted that the execution subjects of the above operations are only exemplary descriptions, and should not be construed as limitations on this application, and may be independently completed by one of the movable platform, the control terminal, the photographing device, and the PTZ, or several of them. The cooperation is complete. For example, in the case where the movable platform is a land robot, a human-computer interaction module (such as a display for displaying a human-computer interaction interface, etc.) can be set on the land robot, and the user can directly display the interactive interface on the movable platform. Get user actions to generate user commands, determine images of target objects, etc. Wherein, the independent completion includes actively or passively, directly or indirectly acquiring corresponding data from other devices to perform corresponding operations.

The embodiments of the present application also provide a computer program product, which includes a computer program, the computer program includes program codes for executing the methods provided by the embodiments of the present application, when the computer program product runs on an electronic device, the The program code is used to enable the electronic device to implement the image model training method or the image processing method provided by the embodiments of the present application.

When the computer program is executed by the processor, the above-mentioned functions defined in the system/device of the embodiments of the present application are executed. According to the embodiments of the present application, the systems, apparatuses, modules, units, etc. described above may be implemented by computer program modules.

In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed in the form of a signal over a network medium, and downloaded and installed through the communication portion, and/or installed from a removable medium. The program code embodied by the computer program may be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

According to the embodiments of the present application, the program code for executing the computer program provided by the embodiments of the present application may be written in any combination of one or more programming languages. programming language, and/or assembly/machine language to implement these computational programs. Programming languages include, but are not limited to, languages such as Java, C++, python, "C" or similar programming languages. The program code may execute entirely on the user computing device, partly on the user device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (eg, using an Internet service provider business via an Internet connection).

The above are the best embodiments of the present application, and it should be noted that the best embodiments are only used for understanding the present application, and are not used to limit the protection scope of the present application. In addition, the features in the preferred embodiment, unless otherwise specified, are applicable to both the method embodiment and the device embodiment, and the technical features appearing in the same or different embodiments can be used without conflicting with each other. used in combination.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the application, rather than limiting them; although the application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: It is still possible to modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements to some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application. .

Claims

A shooting control method, characterized in that the method comprises:

The amount of position change of the photographing device in a specific direction is determined through the measurement information of the accelerometer in the inertial measurement unit, the inertial measurement unit is arranged on the photographing device, or the inertial measurement unit is arranged on the cloud for carrying the photographing device a stage, the inertial measurement unit is used to measure the attitude information of the photographing device;

The photographing device is controlled to follow focus or zoom according to the position change amount.
The method according to claim 1, wherein the controlling the camera to follow focus or zoom according to the position change amount comprises:

A motor engaged with the lens of the photographing device is controlled to drive the lens to follow focus or zoom according to the position change amount.
The method according to claim 2, wherein the controlling the motor engaged with the lens of the photographing device to drive the lens to follow focus or zoom according to the position change amount comprises:

The motor engaged with the lens of the photographing device is controlled to drive the lens to follow focus or zoom according to the position change amount and a mapping relationship, wherein the mapping relationship represents the relationship between the position change amount and the rotor position of the motor.
The method of claim 3, further comprising:

When the re-calibration condition of the mapping relationship is satisfied, a calibration prompt message is output or the use of the mapping relationship is forcibly terminated.
The method according to claim 4, wherein the recalibration condition comprises at least one of the following:

the inertial measurement unit has undergone a power down operation;

The gimbal has undergone a power-off operation;

The displacement of the gimbal in the power-off state exceeds the re-calibration displacement threshold;

The acceleration of the inertial measurement unit is greater than the set acceleration threshold.
The method according to claim 3, wherein the specific direction is a direction of a line connecting the photographing device and the photographing object when the mapping relationship is calibrated.
The method according to claim 3, wherein the mapping relationship is obtained by calibration and/or calculation.
The method according to claim 7, characterized in that there is a linear relationship between the position change amount and the focal length of the lens;

Determining the mapping relationship includes at least one of the following:

determining a first mapping relationship between the rotor position of the motor and the position change amount;

or

determining a second mapping relationship between the rotor position of the motor and the amount of change in focus;

The rotor position of the motor corresponding to each position change amount is calculated based on the second mapping relationship, the image size of the subject, and the size of the subject.
The method according to claim 7, wherein the motor comprises a zoom motor, and the determining the first mapping relationship between the rotor position of the motor and the position change amount comprises:

determining a first preset proportion, where the first preset proportion is the proportion of the image of the photographing object in the photographing screen of the photographing device;

When the photographing device moves a first distance in the specific direction, a first zoom adjustment instruction is obtained, where the first zoom adjustment instruction is used to control the zoom motor to drive the lens to change the focal length, so that the object is photographed The proportion of the image in the shooting picture of the shooting device is in the first preset proportion, and the first rotor position is obtained;

When the photographing device moves a second distance along the specific direction, obtain a second zoom adjustment instruction from the user, where the second zoom adjustment instruction is used to control the zoom motor to drive the lens to change the focal length, so that the The proportion of the image of the photographing object in the photographing picture of the photographing device is at the first preset proportion, and the second rotor position is obtained;

The first mapping relationship is determined based on the first distance, the first rotor position, the second distance, and the second rotor position.
The method according to claim 7, wherein the motor comprises a follow-focus motor, and the determining the first mapping relationship between the rotor position of the motor and the position change amount comprises:

When the photographing device moves a third distance along the specific direction, a first focus adjustment instruction from the user is acquired, and the first focus adjustment instruction is used to control the follow focus motor to drive the lens to follow focus, The third rotor position is obtained so that the image of the photographed object is in the in-focus state on the photographing screen of the photographing device;

When the photographing device moves a fourth distance along the specific direction, a second focus adjustment instruction from the user is acquired, and the second focus adjustment instruction is used to control the follow focus motor to drive the lens to follow focus, obtaining the fourth rotor position so that the image of the photographed object is in focus on the photographing screen of the photographing device;

The first mapping relationship is determined based on the third distance, the third rotor position, the fourth distance, and the fourth rotor position.
The method according to claim 7, characterized in that there is a linear relationship between the position change amount and the focal length of the lens;

When the mapping relationship is determined by calculation, the calculation parameters involved in the calculation include: the image size of the photographed object, the size of the photographed object, the object distance at the starting position, and the flange distance, wherein the flange distance is related to the cam barrel The structural parameters are related to the gear reduction ratio of the motor.
The method according to claim 2, wherein the controlling the motor engaged with the lens of the photographing device to drive the lens to follow focus or zoom according to the position change amount comprises at least one of the following:

Control the zoom motor engaged with the lens of the photographing device to drive the lens to zoom according to the position change amount, so that the proportion of the image of the photographing object in the photographing picture of the photographing device remains stable;

A follow focus motor engaged with the lens of the photographing device is controlled according to the position change amount to drive the lens to follow focus, so that the photographed object is in a focus state in the photographing picture of the photographing device.
The method according to claim 12, wherein the proportion of the image of the photographing object in the photographing screen of the photographing device is determined based on a first user operation.
The method according to claim 13, wherein the first user operation comprises an operation of a user inputting a proportion in the user interaction interface, wherein the proportion input by the user in the user interaction interface is a preset point as the base point; or

The first user operation includes an operation of the user inputting the percentage and the base point in the user interaction interface; or

The first user operation includes an operation of the user inputting at least two base points in the user interaction interface.
The method according to claim 12, wherein the rotating shaft of the zoom motor is engaged with the zoom ring of the lens, and the rotating shaft of the follow focus motor is engaged with the follow focus ring of the lens.
The method according to any one of claims 2-15, wherein the photographing object of the photographing device is determined based on a second user operation of the user on the user interface; or

The photographing object of the photographing device is determined through image recognition.
The method according to any one of claims 2-15, wherein the photographing device is detachably carried on a bearing seat of the head; and/or

The motor is detachably carried on the bearing seat of the pan/tilt head.
The method according to any one of claims 1-15, wherein the photographing device is detachably carried on a bearing seat of a pan/tilt, the pan/tilt is communicatively connected to the photographing device, and the pan/tilt is communicatively connected. A display screen is provided on the handheld part of the camera, and the display screen can display the shooting picture of the shooting device.
The method according to any one of claims 2-15, wherein the method further comprises:

If the included angle between the connection direction between the photographing device and the photographing object and the specific direction satisfies the simplified condition, the position change amount determined based on the measurement information of the accelerometer in the inertial measurement unit is used as the The amount of change in the position of the camera in the specific direction.
The method according to any one of claims 1-15, wherein the photographing device is arranged on a pan/tilt head, and a focus wheel and/or a zoom wheel are provided on the hand-held part of the pan/tilt head;

The method further includes: receiving a third user operation on the follow focus wheel and/or the zoom wheel, so as to control the camera device to follow focus or zoom based on the third user operation.
A shooting control device, characterized in that the device comprises:

at least one processor and memory;

the memory stores computer-executable instructions;

At least one of the processors executes the computer-executed instructions stored in the memory, so that the following steps are implemented when executing the computer-executed instructions:

The amount of position change of the photographing device in a specific direction is determined by the measurement information of the accelerometer in the inertial measurement unit, the inertial measurement unit is arranged on the photographing device, or the inertial measurement unit is arranged on the pan/tilt that carries the photographing device, The inertial measurement unit is used to measure the attitude information of the photographing device;

The photographing device is controlled to follow focus or zoom according to the position change amount.
The device according to claim 21, wherein the controlling the photographing device to follow focus or zoom according to the position change amount comprises:

A motor engaged with the lens of the photographing device is controlled to drive the lens to follow focus or zoom according to the position change amount.
The device according to claim 22, wherein the controlling the motor engaged with the lens of the photographing device to drive the lens to follow focus or zoom according to the position change amount comprises:

The motor engaged with the lens of the photographing device is controlled to drive the lens to follow focus or zoom according to the position change amount and a mapping relationship, wherein the mapping relationship represents the relationship between the position change amount and the rotor position of the motor.
The apparatus according to claim 23, wherein at least one of the processors implements the following steps when executing the computer-executed instructions:

When the re-calibration condition of the mapping relationship is satisfied, a calibration prompt message is output or the use of the mapping relationship is forcibly terminated.
The device according to claim 24, wherein the recalibration condition comprises at least one of the following:

the inertial measurement unit has undergone a power down operation;

The gimbal has undergone a power-off operation;

The displacement of the gimbal in the power-off state exceeds the re-calibration displacement threshold;

The acceleration of the inertial measurement unit is greater than the set acceleration threshold.
The device according to claim 23, wherein the specific direction is a direction of a line connecting the photographing device and the photographing object when the mapping relationship is calibrated.
The apparatus according to claim 23, wherein the mapping relationship is obtained by calibration and/or calculation.
The device according to claim 27, wherein there is a linear relationship between the position change amount and the focal length of the lens;

Determining the mapping relationship includes at least one of the following:

determining a first mapping relationship between the rotor position of the motor and the position change amount;

or

determining a second mapping relationship between the rotor position of the motor and the amount of change in focus;

The rotor position of the motor corresponding to each position change amount is calculated based on the second mapping relationship, the image size of the subject, and the size of the subject.
The device according to claim 27, wherein the motor comprises a zoom motor, and the determining the first mapping relationship between the rotor position of the motor and the position change amount comprises:

determining a first preset proportion, where the first preset proportion is the proportion of the image of the photographing object in the photographing screen of the photographing device;

When the photographing device moves a first distance in the specific direction, a first zoom adjustment instruction is obtained, where the first zoom adjustment instruction is used to control the zoom motor to drive the lens to change the focal length, so that the object is photographed The proportion of the image in the shooting picture of the shooting device is in the first preset proportion, and the first rotor position is obtained;

When the photographing device moves a second distance along the specific direction, obtain a second zoom adjustment instruction from the user, where the second zoom adjustment instruction is used to control the zoom motor to drive the lens to change the focal length, so that the The proportion of the image of the photographing object in the photographing picture of the photographing device is at the first preset proportion, and the second rotor position is obtained;

The first mapping relationship is determined based on the first distance, the first rotor position, the second distance, and the second rotor position.
The device according to claim 27, wherein the motor comprises a follow focus motor, and the determining the first mapping relationship between the rotor position of the motor and the position change amount comprises:

When the photographing device moves a third distance along the specific direction, a first focus adjustment instruction from the user is acquired, and the first focus adjustment instruction is used to control the follow focus motor to drive the lens to follow focus, The third rotor position is obtained so that the image of the photographed object is in the in-focus state on the photographing screen of the photographing device;

When the photographing device moves a fourth distance along the specific direction, a second focus adjustment instruction from the user is acquired, and the second focus adjustment instruction is used to control the follow focus motor to drive the lens to follow focus, obtaining the fourth rotor position so that the image of the photographed object is in focus on the photographing screen of the photographing device;

The first mapping relationship is determined based on the third distance, the third rotor position, the fourth distance, and the fourth rotor position.
The device according to claim 27, wherein there is a linear relationship between the amount of position change and the focal length of the lens;

When the mapping relationship is determined by calculation, the calculation parameters involved in the calculation include: the image size of the photographed object, the size of the photographed object, the object distance at the starting position, and the flange distance, wherein the flange distance is related to the cam barrel The structural parameters are related to the gear reduction ratio of the motor.
The device according to claim 22, wherein the controlling the motor engaged with the lens of the photographing device to drive the lens to follow focus or zoom according to the position change amount comprises at least one of the following:

Control the zoom motor engaged with the lens of the photographing device to drive the lens to zoom according to the position change amount, so that the proportion of the image of the photographing object in the photographing picture of the photographing device remains stable;

A follow focus motor engaged with the lens of the photographing device is controlled according to the position change amount to drive the lens to follow focus, so that the photographed object is in a focus state in the photographing picture of the photographing device.
The device according to claim 32, wherein the proportion of the image of the photographing object in the photographing screen of the photographing device is determined based on a first user operation.
The device according to claim 33, wherein the first user operation comprises an operation of the user inputting a proportion in the user interaction interface, wherein the proportion input by the user in the user interaction interface is a preset point as the base point; or

The first user operation includes an operation of the user inputting the percentage and the base point in the user interaction interface; or

The first user operation includes an operation of the user inputting at least two base points in the user interaction interface.
The device according to claim 32, wherein the rotating shaft of the zoom motor is engaged with the zoom ring of the lens, and the rotating shaft of the follow focus motor is engaged with the follow focus ring of the lens.
The device according to any one of claims 22 to 35, wherein the photographing object of the photographing device is determined based on a second user operation of the user on the user interface; or

The photographing object of the photographing device is determined through image recognition.
The device according to any one of claims 22 to 35, wherein the photographing device is detachably carried on the bearing seat of the pan/tilt head; and/or

The motor is detachably carried on the bearing seat of the pan/tilt head.
The device according to any one of claims 21 to 35, characterized in that, the photographing device is detachably carried on a bearing seat of a pan/tilt, the pan/tilt is communicatively connected to the photographing device, and the pan/tilt is communicatively connected. A display screen is provided on the handheld part of the camera, and the display screen can display the shooting picture of the shooting device.
The apparatus according to any one of claims 22 to 35, wherein at least one of the processors implements the following steps when executing the computer-executed instructions:

If the included angle between the connection direction between the photographing device and the photographing object and the specific direction satisfies the simplified condition, the position change amount determined based on the measurement information of the accelerometer in the inertial measurement unit is used as the The amount of change in the position of the photographing device in the specific direction.
The device according to any one of claims 21 to 35, wherein the photographing device is arranged on a pan/tilt head, and a focus wheel and/or a zoom wheel are provided on the hand-held part of the pan/tilt head;

At least one of the processors implements the following steps when executing the computer-executed instructions:

A third user operation on the focus wheel and/or the zoom wheel is received, so as to control the camera to follow focus or zoom based on the third user operation.
A pan-tilt assembly comprising a pan-tilt and a motor detachable from the pan-tilt, characterized in that the pan-tilt assembly further comprises a shooting control device, and the shooting control device is arranged on the pan-tilt On one of the stage and the motor, the shooting control device includes:

at least one processor and memory;

the memory stores computer-executable instructions;

At least one of the processors executes the computer-executed instructions stored in the memory, so that the following steps are implemented when executing the computer-executed instructions:

The amount of change in the position of the photographing device in a specific direction is determined through the measurement information of the accelerometer in the inertial measurement unit, the inertial measurement unit is arranged on the photographing device, or the inertial measurement unit is arranged on the cloud carrying the photographing device a stage, the inertial measurement unit is used to measure the attitude information of the photographing device;

The motor is controlled according to the position change amount, so that the photographing device follows focus or zooms.
The pan-tilt assembly according to claim 41, wherein the controlling the motor according to the position change amount, so that the photographing device follows focus or zooms, specifically comprises:

The motor engaged with the lens of the photographing device is controlled to drive the lens to follow focus or zoom according to the position change amount.
The pan/tilt assembly according to claim 42, wherein the controlling the motor engaged with the lens of the photographing device to drive the lens to follow focus or zoom according to the position change amount comprises:

The motor engaged with the lens of the photographing device is controlled to drive the lens to follow focus or zoom according to the position change amount and a mapping relationship, wherein the mapping relationship represents the relationship between the position change amount and the rotor position of the motor.
The pan-tilt assembly according to claim 43, wherein at least one of the processors implements the following steps when executing the computer-executed instructions:

When the re-calibration condition of the mapping relationship is satisfied, a calibration prompt message is output or the use of the mapping relationship is forcibly terminated.
The pan/tilt assembly according to claim 44, wherein the recalibration conditions include at least one of the following:

the inertial measurement unit has undergone a power down operation;

The gimbal has undergone a power-off operation;

The displacement of the gimbal in the power-off state exceeds the re-calibration displacement threshold;

The acceleration of the inertial measurement unit is greater than the set acceleration threshold.
The pan/tilt assembly according to claim 43, wherein the specific direction is the direction of the line connecting the photographing device and the photographing object when the mapping relationship is calibrated.
The pan/tilt assembly according to claim 43, wherein the mapping relationship is obtained by calibration and/or calculation.
The pan/tilt assembly according to claim 47, wherein there is a linear relationship between the position change amount and the focal length of the lens;

Determining the mapping relationship includes at least one of the following:

determining a first mapping relationship between the rotor position of the motor and the position change amount;

or

determining a second mapping relationship between the rotor position of the motor and the amount of change in focus;

The rotor position of the motor corresponding to each position change amount is calculated based on the second mapping relationship, the image size of the subject, and the size of the subject.
The pan-tilt assembly according to claim 48, wherein the motor comprises a zoom motor, and the determining the first mapping relationship between the rotor position of the motor and the position change amount comprises:

obtaining a first preset proportion, where the first preset proportion is the proportion of the image of the photographing object in the photographing screen of the photographing device;

When the photographing device moves a first distance along the specific direction, a first zoom adjustment instruction is acquired, where the first zoom adjustment instruction is used to control the zoom motor to drive the lens to change the focal length, so that the object is photographed The proportion of the image in the shooting picture of the shooting device is in the first preset proportion, and the first rotor position is obtained;

When the photographing device moves a second distance along the specific direction, obtain a second zoom adjustment instruction from the user, where the second zoom adjustment instruction is used to control the zoom motor to drive the lens to change the focal length, so that the The proportion of the image of the photographing object in the photographing picture of the photographing device is at the first preset proportion, and the second rotor position is obtained;

The electric machine determines the first mapping relationship based on the first distance, the first rotor position, the second distance, and the second rotor position.
The pan-tilt assembly according to claim 48, wherein the motor comprises a follow focus motor, and the determining the first mapping relationship between the rotor position of the motor and the position change comprises:

When the photographing device moves a third distance along the specific direction, a first focus adjustment instruction from the user is acquired, and the first focus adjustment instruction is used to control the follow focus motor to drive the lens to follow focus, The third rotor position is obtained so that the image of the photographed object is in the in-focus state on the photographing screen of the photographing device;

When the photographing device moves a fourth distance along the specific direction, a second focus adjustment instruction from the user is acquired, and the second focus adjustment instruction is used to control the follow focus motor to drive the lens to follow focus, obtaining the fourth rotor position so that the image of the photographed object is in focus on the photographing screen of the photographing device;

The first mapping relationship is determined based on the third distance, the third rotor position, the fourth distance, and the fourth rotor position.
The pan/tilt assembly according to claim 47, wherein there is a linear relationship between the position change amount and the focal length of the lens;

When the mapping relationship is determined by calculation, the calculation parameters involved in the calculation include: the image size of the photographed object, the size of the photographed object, the object distance at the starting position, and the flange distance, wherein the flange distance is related to the cam barrel The structural parameters are related to the gear reduction ratio of the motor.
The pan/tilt assembly according to claim 42, wherein the controlling the motor engaged with the lens of the photographing device to drive the lens to follow focus or zoom according to the position change amount comprises at least one of the following:

Drive the zoom of the lens according to the position change amount, so that the proportion of the image of the photographed object in the photographed picture of the photographing device remains stable;

The lens is driven to follow focus according to the position change amount, so that the photographing object is in a focused state in the photographing picture of the photographing device.
The pan/tilt assembly according to claim 52, wherein the proportion of the image of the photographed object in the photographed screen of the photographing device is determined based on a first user operation.
The pan-tilt assembly according to claim 53, wherein the first user operation comprises an operation of the user inputting a proportion in the user interaction interface, wherein the proportion input by the user in the user interaction interface is a preset point as a base point; or

The first user operation includes an operation of the user inputting the percentage and the base point in the user interaction interface; or

The first user operation includes an operation of the user inputting at least two base points in the user interaction interface.
The pan/tilt assembly according to claim 52, wherein the motor comprises a follow focus motor and/or a zoom motor, the rotating shaft of the zoom motor is engaged with the zoom ring of the lens, and the rotating shaft of the follow focus motor Engage with the focus ring of the lens.
The pan-tilt assembly according to claim 54, wherein the photographing object of the photographing device is determined based on a second user operation of the user on the user interface; or

The photographing object of the photographing device is determined through image recognition.
The pan/tilt assembly according to any one of claims 42 to 56, wherein the photographing device is detachably carried on a bearing seat of the pan/tilt; and/or

The motor is detachably carried on the bearing seat of the pan/tilt head.
The pan/tilt assembly according to any one of claims 41 to 56, wherein the photographing device is detachably carried on a bearing seat of the pan/tilt, the pan/tilt is connected in communication with the photographing device, and the camera is A display screen is provided on the hand-held part of the PTZ, and the display screen can display the shooting picture of the shooting device.
The pan-tilt assembly according to any one of claims 42 to 56, wherein, when at least one of the processors executes the computer-executed instructions, the following steps are implemented:

If the angle between the connection direction between the photographing device and the photographing object and the specific direction satisfies the simplified condition, the position change amount determined by the motor based on the measurement information of the accelerometer in the inertial measurement unit is used as The amount of change in the position of the photographing device in the specific direction.
The pan/tilt assembly according to any one of claims 41 to 56, wherein the photographing device is arranged on the pan/tilt, and a focus wheel and/or a zoom wheel are provided on the hand-held part of the pan/tilt;

When at least one of the processors executes the computer-executed instructions, the following steps are implemented:

A third user operation on the focus wheel and/or the zoom wheel is received, so as to control the motor based on the third user operation and cause the photographing device to follow focus or zoom.
A readable storage medium, characterized in that a computer program is stored on the readable storage medium; when the computer program is executed, the shooting control method according to any one of claims 1-20 is implemented.
A computer program product, characterized by comprising a computer program, which implements the shooting control method according to any one of claims 1 to 20 when the computer program is executed.