WO2022109860A1 - 跟踪目标对象的方法和云台 - Google Patents

跟踪目标对象的方法和云台 Download PDF

Info

Publication number
WO2022109860A1
WO2022109860A1 PCT/CN2020/131446 CN2020131446W WO2022109860A1 WO 2022109860 A1 WO2022109860 A1 WO 2022109860A1 CN 2020131446 W CN2020131446 W CN 2020131446W WO 2022109860 A1 WO2022109860 A1 WO 2022109860A1
Authority
WO
WIPO (PCT)
Prior art keywords
target object
load
tracking mode
tracking
pan
Prior art date
Application number
PCT/CN2020/131446
Other languages
English (en)
French (fr)
Inventor
楼致远
杨天豪
常贤茂
苏铁
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN202080074276.8A priority Critical patent/CN114641642A/zh
Priority to PCT/CN2020/131446 priority patent/WO2022109860A1/zh
Publication of WO2022109860A1 publication Critical patent/WO2022109860A1/zh

Links

Images

Classifications

    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F16—ENGINEERING ELEMENTS AND UNITS; GENERAL MEASURES FOR PRODUCING AND MAINTAINING EFFECTIVE FUNCTIONING OF MACHINES OR INSTALLATIONS; THERMAL INSULATION IN GENERAL
    • F16M—FRAMES, CASINGS OR BEDS OF ENGINES, MACHINES OR APPARATUS, NOT SPECIFIC TO ENGINES, MACHINES OR APPARATUS PROVIDED FOR ELSEWHERE; STANDS; SUPPORTS
    • F16M11/00—Stands or trestles as supports for apparatus or articles placed thereon ; Stands for scientific apparatus such as gravitational force meters
    • F16M11/02—Heads
    • F16M11/04—Means for attachment of apparatus; Means allowing adjustment of the apparatus relatively to the stand
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing

Definitions

  • the present application relates to the technical field of control, and in particular, to a method and a pan/tilt for tracking a target object.
  • the pan/tilt head can be used to carry a load and adjust the attitude of the load, eg, change the height, inclination and/or orientation of the load.
  • the motion of the target object to be tracked by the load is usually diversified, and the related technology can automatically track the target object by changing the attitude of the load through the gimbal, which effectively improves the convenience of using the load to track the target object.
  • the user may need more control flexibility for the load, for example, it is convenient for the user to use the shooting device mounted on the gimbal to follow the target object with various shooting methods.
  • the embodiments of the present application provide a method and a pan/tilt for tracking a target object, so as to meet the user's requirement for flexibility in control of the load, and can realize the tracking of the target object with diverse movements.
  • an embodiment of the present application provides a pan/tilt head, including: a bracket assembly including at least two relatively movable bracket parts, the bracket assembly is used to support a load; at least two motors are respectively used to drive corresponding brackets The component moves to adjust the attitude of the load; wherein the PTZ has at least two tracking modes, and each of the at least two tracking modes can drive the motor of the bracket component to move with the change of the position of the target object within the sensing range of the load. The number is different to achieve the tracking of the target object in the direction of different dimensions.
  • an embodiment of the present application provides a method for tracking a target object, which is used in a pan/tilt head, where the pan/tilt head includes a bracket assembly and at least two motors, the bracket assembly includes at least two relatively movable bracket components, and is used for To support the load, at least two motors are respectively used to drive the corresponding bracket parts to move, so as to adjust the attitude of the load.
  • the method includes: acquiring a mode selection instruction; in response to the mode selection instruction, determining a current tracking mode from at least two tracking modes, wherein each of the at least two tracking modes is capable of driving the support member to follow the target object within the sensing range of the load
  • the number of motors that move due to the change of the position of the motor is different, and the load is set on the bracket part; the current tracking mode is used to control the motor corresponding to the current tracking mode, so as to realize the tracking of the target object by the load in the direction of the specified dimension.
  • embodiments of the present application provide a computer-readable storage medium, which stores executable instructions, and when executed by one or more processors, the executable instructions can cause one or more processors to execute the above-mentioned method.
  • an embodiment of the present application provides a computer program, including executable instructions, which, when executed, implement the above method.
  • the pan/tilt head has at least two tracking modes, and in the at least two tracking modes, the number of motors that can drive the support member to move according to the position change of the target object within the sensing range of the load is different, so that The user can conveniently adjust the user's control flexibility for the load by adopting different tracking modes, and can track the target object in different dimensions.
  • 1 is an application scenario of a method for tracking a target object and a PTZ provided by an embodiment of the present application
  • FIG. 2 provides a method for tracking a target object and an application scenario of a PTZ provided by another embodiment of the present application
  • FIG. 3 is a schematic structural diagram of a PTZ provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a load mounted on a gimbal according to an embodiment of the present application
  • FIG. 5 is a schematic diagram when the target object is far away from the photographing device in the embodiment of the present application.
  • FIG. 6 is a schematic diagram when the target object is relatively close to the photographing device in the embodiment of the present application.
  • FIG. 7 is a schematic diagram when the target object provided by the embodiment of the present application is relatively close to the photographing device
  • FIG. 8 is a schematic diagram of the perspective of the photographing device in the process of the target object passing from the side of the photographing device in the embodiment of the present application;
  • Fig. 9 is the schematic diagram of the angle of view of the photographing device in the process of the target object passing through the side of the photographing device provided by another embodiment of the present application;
  • FIG. 10 is a schematic diagram of non-overlapping between an optical axis and a roll axis of a photographing device provided by an embodiment of the present application;
  • FIG. 11 is a schematic diagram of the movement trajectory of the center of the target object of FIG. 10 in a plurality of captured images
  • FIG. 12 is a schematic diagram of an interactive interface provided by an embodiment of the present application.
  • FIG. 13 is a data flow diagram of target object tracking provided by an embodiment of the present application.
  • FIG. 14 is an intention of diversifying movements of a target object provided by an embodiment of the present application.
  • FIG. 15 is a schematic diagram of a captured image of the target object of FIG. 14;
  • 16 is a flowchart of a method for tracking a target object provided by an embodiment of the present application.
  • 17 is a block diagram of an apparatus for tracking a target object provided by an embodiment of the present application.
  • FIG. 18 is a schematic diagram of a movable platform with a power system provided by an embodiment of the present application.
  • FIG. 19 is a schematic diagram of a PTZ and a terminal device provided by an embodiment of the present application.
  • the embodiment of the present application can be applied to a user's follow-up of a target object with a specified degree of freedom, where the specified degree of freedom may be a single degree of freedom, two degrees of freedom, or three degrees of freedom, or the like.
  • the target object such as the subject to be photographed
  • the position of the subject's face image in the captured image including the left and right position and the up and down position) position
  • the gimbal can control at least part of its motor to act, so that the camera can track the target object.
  • the image of the subject's face can always be located at the center of the image, so as to realize the follow-up function.
  • this also makes it inconvenient for the user to intervene, such as the composition of the background image. Because it is inconvenient for the user to implement a variety of lens movement methods, the shooting effect expected by the user may not be achieved.
  • scenes are only exemplary scenes, and may also be shooting scenes, scanning scenes, etc. for animal shooting, image capture, video shooting, movie shooting, TV drama shooting, etc., which should not be construed as a limitation of this application.
  • the appearance of the gimbal not only provides a stable shooting environment for taking pictures and video recordings, but also provides a wealth of possibilities for photographers to move the mirror.
  • the hand-held gimbal can make the camera set on it rotate around at least one of the yaw axis, the pitch axis, and the roll axis, so as to achieve a specific lens movement shooting effect, such as "follow the hand" lens movement.
  • the combination of the gimbal and the intelligent follow algorithm can make the object to be photographed well placed in the frame, which improves the shooting quality and greatly improves the flexibility of shooting.
  • the combined application of the gimbal and the intelligent following algorithm in the related art enables the gimbal to automatically adjust the yaw axis and pitch axis to adjust the posture of the photographing device when the user moves or rotates the gimbal, so that the target object is always placed on the screen. fixed location.
  • this method also has some shortcomings. Since most of the control of the shooting picture is handed over to the gimbal algorithm, the user can only control the position of the gimbal, and cannot do more lens movements. For example, in the scene of shooting people and buildings, if the user wants to follow intelligently only in the panning direction, the current intelligent following algorithm cannot be realized.
  • the application of intelligent follow-up of a handheld gimbal in the related art is to adjust the angle of the gimbal through the movement of the pan axis and the pitch axis, and the following modes of each axis cannot be set independently.
  • the picture obtained by the intelligent follow-up control method of "pan axis + tilt axis" is sometimes not what the photographer wants. Moving in the vertical direction causes the content of the framing to change.
  • some handheld gimbal supports third-party photography equipment (such as mobile phones and cameras). After leveling, the rotation center of its roll axis does not coincide with the optical axis center of the camera equipment. In the FPV rotating mirror, the user cannot be photographed. expected picture.
  • third-party photography equipment such as mobile phones and cameras.
  • the gimbal and the method for tracking a target object provided by the embodiments of the present application propose the combined application of the intelligent following algorithm and various gimbal tracking modes, which enables the gimbal to perform specific operations, such as moving the mirror, while the load is still in the desired position. It needs to track the object to be photographed with a degree of freedom, and on the premise of ensuring the content of the picture, it enriches the user's shooting methods, and reduces the difficulty for users to shoot materials of higher specifications.
  • FIG. 1 is an application scenario of a method for tracking a target object and a PTZ provided by an embodiment of the present application.
  • the gimbal can control at least part of its own motor to drive the shooting device to follow the rider.
  • the gimbal may be a gimbal held by the photographer, or a gimbal set on a movable platform, such as a gimbal set on a drone, etc., which is not limited here.
  • the photographer can also control the movable platform during the shooting process, such as flight trajectory control.
  • the rider's face image can always be located at the center of the image, so as to realize the follow-up function.
  • the user since most of the control of the shooting picture is handed over to the gimbal algorithm, the user can only control the position of the gimbal, which makes it inconvenient to compose pictures such as background images, and may not achieve the shooting effect expected by the photographer.
  • the method and pan/tilt for tracking a target object provided by the embodiments of the present application can be selected by the tracking mode, so that the user can independently select a motor that can drive the load to track the target object, and realize the free selection of various degrees of freedom, which is convenient for the user.
  • necessary mirror movement techniques, etc. are implemented to capture a desired image.
  • intelligent follow-up control of a single yaw axis can be implemented to provide users with a more stable picture in the vertical direction.
  • the method for tracking a target object and the pan/tilt provided by the embodiments of the present application can also realize an intelligent following control method at any angle, and can provide a user with a more free composition method.
  • FIG. 2 provides a method for tracking a target object and an application scenario of a PTZ according to another embodiment of the present application.
  • a gimbal capable of carrying a third-party load is used as an example for description. It should be noted that the third-party load may also be integrated with the gimbal, which is not limited here.
  • the pan/tilt head in FIG. 2 may include a carrier 200 and a handle 201 and the like.
  • the carrier 200 may include a matching motor and a shaft arm, and the motor is used to drive the shaft arm to rotate, so as to drive the movement of the third-party load.
  • the carrier body 200 may be a variety of support structures, including but not limited to a single-axis or multi-axis attitude adjustable structure, for placing a load on the handle 201 .
  • the payload may be a camera that allows the camera to be displaced relative to the handle 201, or rotated along one or more axes, such as the carrier 200 that allows the camera to move along the pitch, pan, and roll axes A combined translational movement of one or more axes.
  • the carrier 200 may allow the camera to rotate about one or more of a pitch axis, a pan axis, and a roll axis.
  • There may be a linkage conversion relationship between the carrier 200 and the handle 201 for example, a first movement (eg, movement or rotation) of the handle 201 can be converted into a second movement of the carrier 200 . vice versa.
  • a sensor system may also be included on the gimbal.
  • the sensing system may include one or more sensors to sense spatial orientation, velocity, and/or acceleration (eg, rotation and translation with respect to up to three degrees of freedom).
  • the one or more sensors include, but are not limited to, GPS sensors, motion sensors, inertial sensors, or image sensors.
  • the sensing data provided by the sensing system can be used to control the pose, velocity and/or acceleration of the load.
  • a sensing system may be used to detect data about the environment of the gimbal, such as climatic conditions, the location of man-made structures, and the like.
  • the PTZ may also include a communication system.
  • the communication system can realize the communication between the PTZ and the control terminal with the communication system through wired or wireless signals sent and received.
  • a communication system may include any number of transmitters, receivers, and/or transceivers for wireless communication.
  • Communication can be one-way communication, so that data can be sent from one direction.
  • one-way communication may involve only the pan/tilt transmitting data to the control terminal, or vice versa.
  • One or more transmitters of the communication system may transmit data to one or more receivers of the communication system, and vice versa.
  • the communication can be two-way communication, so that data can be transmitted in both directions between the PTZ and the control terminal. Two-way communication includes that one or more transmitters of the communication system can send data to one or more receivers of the communication system, and vice versa.
  • control terminal may be connected to the pan/tilt or the load, the control terminal may provide control instructions to one or more of the pan/tilt and the payload, and receive information from one or more of the pan/tilt and the payload (For example, position and/or motion information of the carrier 200 or the load, data of load sensing, such as image data captured by the camera).
  • control data of the control terminal may include instructions regarding position, movement, braking, or control of the pan/tilt and/or load.
  • the control data may cause a change in the position and/or orientation of the carrier 200 .
  • Control data from the control terminal can result in load control, such as controlling the operation of a camera or other image capture device (capturing still or moving images, zooming, turning on or off, switching imaging modes, changing image resolution, changing focus, changing depth of field, change the exposure time, change the viewing angle or field of view).
  • communications to the pan/tilt and/or payload may include information from one or more sensors.
  • Communication may include sensory information transmitted from one or more different types of sensors, such as GPS sensors, motion sensors, inertial sensors, proximity sensors, or image sensors.
  • Sensing information is about position (eg, orientation, position), motion, or acceleration to the gimbal and/or load.
  • the sensed information transmitted from the load includes the data captured by the load or the status of the load.
  • the control data transmitted and provided by the control terminal may be used to control the state of one or more of the pan/tilt, the carrier 200 or the load.
  • one or more of the carrier 200 and the load may include a communication module for communicating with the control terminal, so that the control terminal can communicate individually or control the pan-tilt and the load.
  • the control terminal may be a remote controller of the PTZ, or may be an intelligent electronic device such as a mobile phone, an iPad, a wearable electronic device, etc., which can be used to control the PTZ.
  • control terminal can be far away from the gimbal to realize remote control of the gimbal, and can be fixed or detachable on the gimbal, which can be set as required.
  • the pan-tilt can communicate with other remote devices other than the control terminal, or with remote devices other than the control terminal.
  • the control terminal can also communicate with another remote device and PTZ.
  • the pan-tilt and/or control terminal may communicate with another movable platform or a carrier or payload of another movable platform.
  • the additional remote device may be a second terminal or other computing device (eg, a computer, desktop, tablet, smartphone, or other mobile device).
  • the remote device may transmit data to the pan/tilt, receive data from the pan/tilt, transmit data to the control terminal, and/or receive data from the control terminal.
  • the remote device may be connected to the Internet or other telecommunication network to allow data received from the pan-tilt and/or control terminal to be uploaded to a website or server.
  • FIG. 3 is a schematic structural diagram of a pan/tilt according to an embodiment of the present application.
  • the pan/tilt head 300 may include: a bracket assembly 31 and at least two motors 32 .
  • the bracket assembly 31 may include at least two relative movable bracket parts 311 and 312 , and the bracket assembly 31 is used to support the load 40 .
  • At least two motors 32 are respectively used to drive the corresponding bracket parts to move, so as to adjust the posture of the load 40 .
  • the pan/tilt 300 has at least two tracking modes, and each of the at least two tracking modes has a different number of motors 32 that can drive the bracket member to move with the change of the position of the target object within the sensing range of the load 40 , so that in different The tracking of the target object is realized in the direction of the dimension.
  • a pitch axis motor and a pitch axis arm cooperate to drive the load 40 in rotation about the pitch axis.
  • the roll axis motor and the roll axis arm cooperate to drive the load 40 in rotation about the roll axis.
  • a yaw axis motor and a yaw axis arm cooperate to drive the load 40 in rotation about the yaw axis.
  • the pitch axis motor can drive the movement of the pitch axis arm
  • the roll axis motor can drive the movement of the roll axis arm
  • the yaw axis motor can drive the movement of the yaw axis arm.
  • the yaw axis arm may be connected to one end of the roll axis arm, and the other end of the roll axis arm may be connected to the pitch axis arm, but the embodiment of the present application is not limited to this, the yaw axis arm, the roll axis arm and the pitch axis arm
  • the axle arms can also be connected in other sequences.
  • pan/tilt head 300 can also allow the load to rotate around only one, two or four axes, etc., which is not limited herein.
  • the tracking modes include at least two of the first to third tracking modes shown below.
  • the number of motors capable of driving the support member to move according to the position change of the target object within the sensing range of the load is one.
  • the motor may be any of a pitch axis motor, a yaw axis motor, or a roll axis motor.
  • the load has only one degree of freedom in the first tracking mode, such as tracking the target object driven by the yaw axis motor.
  • the number of motors capable of driving the support member to move according to the position change of the target object within the sensing range of the load is two.
  • the motor may be any two of a pitch axis motor, a yaw axis motor, or a roll axis motor. This allows the payload to have two degrees of freedom in the second tracking mode, such as tracking the target object driven by the yaw axis motor and/or the pitch axis motor.
  • the number of motors capable of driving the support member to move according to the position change of the target object within the sensing range of the load is three.
  • the motor may be any one or more of a pitch axis motor, a yaw axis motor, or a roll axis motor.
  • the load has three degrees of freedom in the third tracking mode, such as tracking the target object driven by at least one of the yaw axis motor, the pitch axis motor and the roll axis motor.
  • the roll axis motor drives the lens group to rotate around the roll axis
  • the yaw axis motor and the pitch axis motor can be used to adjust the position of the load to improve the distance between the optical axis of the lens group and the roll axis.
  • Non-overlapping causes the problem that the image center of the target object does not overlap with the image center of the captured image, resulting in poor viewing angle of the picture.
  • the motor capable of driving the support member to move with the movement of the target object in the first tracking mode includes a yaw axis motor.
  • the motors capable of driving the support member to move with the movement of the target object include a yaw axis motor and a pitch axis motor.
  • the motors that can drive the support member to move with the movement of the target object include a roll axis motor and a pitch axis motor.
  • the motors capable of driving the support member to move with the movement of the target object include a yaw axis motor and a roll axis motor.
  • the motors capable of driving the support member to move with the movement of the target object include a yaw axis motor, a pitch axis motor and a roll axis motor.
  • the embodiments of the present application do not exclude that the power components (such as air cylinders, hydraulic cylinders or linear motors) that drive translation are at least corresponding to any of the above tracking modes.
  • the power components such as air cylinders, hydraulic cylinders or linear motors
  • FIG. 4 is a schematic diagram of a load mounted on a gimbal according to an embodiment of the present application.
  • the pan/tilt head may further include: a load fixing mechanism 50 .
  • the load fixing mechanism 50 is used for fixing the load 40 , and the attitude of the load 40 is adjustable and fixed on the load fixing mechanism 50 .
  • a photographing device such as a camera
  • the first fixing surface of the load and the second fixing surface of the load fixing mechanism are parallel or perpendicular to each other.
  • the load fixing mechanism 50 can rotate relative to one or more axle arms.
  • the load fixing mechanism includes a rotating arm that can be rotated relative to the tilt axis and a fixing part that can be matched with the photographing device.
  • the fixed portion can move linearly relative to the pivoting arm to facilitate securing loads of different sizes or configurations.
  • the load fixing mechanism 50 may be a separate component or a part of a certain axis arm, for example, the load fixing mechanism 50 may be a component of a pitch axis arm or a yaw axis arm, which is not limited herein.
  • the camera when the load needs to be loaded on the head, the camera can be fixed on the rotating arm first, and the position of the fixed part can be adjusted so that the fixed part can be matched with the positioning part of the camera, and then the camera can be fixed on the rotating arm. The designated position, so that the camera is set on the load fixing mechanism.
  • the gimbal may further include: an inertial measurement unit (Inertial measurement unit, IMU for short).
  • the inertial measurement unit can be set anywhere on the pan/tilt, and is used to determine the attitude information of the components supported by the set inertial measurement unit.
  • the inertial measurement unit may be arranged on the load fixing mechanism 50 for measuring the attitude information of the fixing mechanism 50 .
  • the inertial measurement unit may be arranged on the shaft arm.
  • the inertial measurement unit may be at least one of an accelerometer or a gyroscope, and may be used to measure the attitude and acceleration of the photographing device.
  • an inertial measurement unit may be provided on the payload 40 for measuring the pose information of the payload.
  • each inertial measurement unit may be disposed on a different component, so as to measure the pose information of the component.
  • the gimbal may include a handheld gimbal, an airborne gimbal, and the like.
  • a stand assembly is used to secure to a movable platform with a powered system.
  • the mobile platform is an unmanned aerial vehicle as an example to illustrate.
  • the movable platform may include a power mechanism, a sensing system.
  • the movable platform may also include a communication system.
  • the power mechanism may include one or more rotating bodies, propellers, blades, engines, motors, wheels, bearings, magnets, and nozzles.
  • the rotating body of the powertrain may be a self-tightening rotating body, a rotating body assembly, or other rotating body power unit.
  • the movable platform may have one or more power mechanisms. All powertrains can be of the same type or of different types.
  • the power mechanism enables the movable platform to take off vertically from a surface, or to land vertically on a surface, without any horizontal movement of the movable platform (eg, without taxiing on a runway).
  • the movable platform may have a plurality of horizontal rotating bodies to control the lifting and/or pushing of the movable platform.
  • the sensing system may include one or more sensors to sense surrounding obstacles, spatial orientation, velocity and/or acceleration (eg, rotation and translation with respect to up to three degrees of freedom) of the movable platform.
  • sensors to sense surrounding obstacles, spatial orientation, velocity and/or acceleration (eg, rotation and translation with respect to up to three degrees of freedom) of the movable platform.
  • acceleration eg, rotation and translation with respect to up to three degrees of freedom
  • the communication system please refer to the relevant part of the communication system of the PTZ, which will not be repeated here.
  • the pan/tilt head further includes: a holding component 60 , and the holding component 60 is used to support the bracket component 31 .
  • the holding assembly 60 can also function such as accommodating batteries, processors, setting input/output components, etc., which are not limited herein.
  • the pan/tilt head may include a pitch axis motor 322, a roll axis motor 323, a yaw axis motor 321, a holding assembly 60, a yaw axis arm 311, a load fixing mechanism 50 (which may include an inertial measurement element inside), a pitch A shaft arm 312, a roll shaft arm 313, a camera 40, and the like.
  • FIG. 5 is a schematic diagram when the target object is far away from the photographing device in the embodiment of the present application.
  • FIG. 6 is a schematic diagram when the target object is relatively close to the photographing device in the embodiment of the present application.
  • the captured image as shown in Figure 6 can be obtained, and the rider's image will always remain in the center of the captured image.
  • the resulting picture is not what the photographer wants. For example, when photographing a rider who is gradually approaching, as shown in Figure 6, as the rider gets closer and closer to the camera, the image of the rider In the picture captured by the camera, the image will become larger and larger, causing the facial image of the rider to gradually move toward the upper position of the image captured by the camera.
  • the intelligent tracking of the pitch axis The control will move the screen in the vertical direction Z, resulting in changes in the content of the viewfinder. If the user wishes to shoot a video with a relatively stable background and track the rider in the X and Y directions, the related art cannot easily achieve this shooting effect.
  • FIG. 7 is a schematic diagram when the target object is relatively close to the photographing device according to an embodiment of the present application.
  • the embodiment of the present application can limit the degree of freedom in the Z-axis direction, so that the gimbal can only control the motor corresponding to the Y-axis and the motor corresponding to the X-axis to track the target object.
  • the framing is kept stable to meet the user's specific shooting effect needs.
  • FIG. 8 is a schematic diagram of the viewing angle of the photographing device when the target object passes through the side of the photographing device according to the embodiment of the present application.
  • FIG. 9 is a schematic diagram of a viewing angle of a photographing device when a target object passes through the side of the photographing device according to another embodiment of the present application.
  • the rider's image is always kept in the center of the captured image, achieving the effect of following the rider.
  • the follow-up shooting effect cannot meet the user's shooting effect requirements. For example, as the rider gets closer and closer to the user, it occupies a larger and larger proportion of the image captured by the user, and at the same time, the user's face image is closer and closer to the top of the captured image, even closer to or more than Take the top of the image.
  • the gimbal will adjust the camera's pose in the Z-axis direction, so that the rider's face, etc., is always kept in the center of the captured image.
  • the background of the captured image changes from the background image of the circle on the left in Figure 8 to the background image of the circle on the right in Figure 8, causing the viewing range of the background to move up significantly. Gradually increase, the above situation will occur.
  • the gimbal since the degree of freedom of the gimbal in the Z-axis direction is limited, the gimbal can not track the rider in the Z-axis direction, but only track the rider in the X-axis and Y-axis directions.
  • the difficulty of moving the mirror is easy to obtain the shooting effect expected by the user.
  • FIG. 10 is a schematic diagram showing that the optical axis and the roll axis of the photographing device provided by the embodiment of the present application do not overlap.
  • FIG. 11 is a schematic diagram of the movement trajectory of the center of the target object in FIG. 10 in a plurality of captured images
  • the load can be provided by a third party, such as a camera purchased by the user, the size of the cameras produced by different manufacturers is different, and it is not easy to adjust the optical axis Op1 or the optical axis Op2 of the camera to be different from the optical axis Op2.
  • the roll axis of the gimbal overlaps.
  • the optical axes Op1 and Op2 of the camera and the roll axis of the gimbal are overlapped by adjustment, due to the difference in the center of gravity of different models of cameras, the optical axis is likely to be caused when the camera is driven to rotate.
  • Op1 and Op2 are separated from the Roll axis. When Op1, Op2 are separated from the Roll axis, the situation shown in Figure 11 may result.
  • the center of the target object is located at the center c of the captured image, as shown in the dotted box in Figure 11, the center image of the target object should be located in the dotted box.
  • the center image of the target object will have a circular motion trajectory as shown in Figure 11, making it impossible to meet the desired shooting effect.
  • the payload includes a camera.
  • the position of the target object in the photographed picture of the photographing device is the central position of the picture.
  • the yaw axis motor and the pitch axis motor can jointly track the target object to correct the shooting effect caused by the non-overlapping between the optical axis and the Roll axis. abnormal.
  • the PTZ is further configured to determine a tracking mode, and the tracking mode is determined by at least one of the following methods.
  • the tracking mode is determined in response to a mode selection instruction received from the user interface.
  • the tracking mode is determined in response to a mode selection operation for a preset function key.
  • the user interaction interface is displayed on the display located on the handle assembly, and the handle component is used to support the bracket assembly; or, the user interaction interface is displayed on the display located on the load; or, the user interaction interface is displayed on the terminal device connected to the PTZ or, the preset function keys are located on the handle assembly, and the handle assembly is used to support the stand assembly.
  • the PTZ may be provided with an input part and/or an output part.
  • the input part may be used for inputting the user's operation instruction on the handheld pan/tilt head, and the input part may include a tracking mode input part and a control joystick.
  • the control rocker can control the movement of the pivot arm. For example, by turning the control rocker, the pivot arm of the handheld gimbal can be rotated in a corresponding direction.
  • the tracking mode input component may select the tracking mode mentioned in the embodiment of the present application.
  • the tracking mode input component may include a display screen for displaying an interactive interface, and a user may input a mode selection instruction in the interactive interface, so that the processor determines the tracking mode based on the mode selection instruction.
  • FIG. 12 is a schematic diagram of an interactive interface provided by an embodiment of the present application.
  • the interactive interface may include a tracking mode selection component. After the user clicks on a component in the interactive interface, the corresponding tracking mode will be triggered.
  • an input component may be further displayed in the current interactive interface or other interactive interfaces to facilitate the user to input attitude, or to display the current load attitude.
  • the tracking mode input part may also include control keys, and the tracking mode output part may include a plurality of status indicators.
  • the control button is used to select the tracking mode, one indicator light can correspond to one tracking mode, and the indicator light is on when the corresponding tracking mode is selected.
  • the user briefly presses the control button once, and the light on the far right turns on, indicating that the tracking mode corresponding to the far right is selected.
  • the indicator light in the middle is on. If you operate it again, the indicator light in the middle is off, and the leftmost indicator light is on.
  • the selection manner of the tracking mode in this embodiment of the present application is not limited to the foregoing manner.
  • the number of the above-mentioned indicator lights is also not limited to three, and may be more than three or less than three.
  • the input unit may also have other components or parts, for example, may have a switch of a handheld pan/tilt and the like.
  • a processor may be provided in the input unit for processing input control commands, or sending and receiving signals.
  • the processor can also be arranged in the handle assembly.
  • the processor may be a central processing unit (Central Processing Unit, referred to as CPU), and the processor may also be other general-purpose processors, digital signal processors (Digital Signal Processor, referred to as DSP), application-specific integrated circuits (application specific integrated circuit, referred to as ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, referred to as FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the following is an exemplary description of the tracking algorithm of the PTZ.
  • the payload is a camera.
  • the gimbal is used to convert the image captured by the photographing device in the current posture to the specified posture based on the posture information of the load, and identify the position of the target object from the image converted to the specified posture.
  • the computing power of the gimbal is limited, especially for the handheld gimbal, it is not convenient to determine the position of the image of the target object in the captured image from the image by means of cloud image recognition.
  • the gimbal When the image of the target object rotates in the captured image, Then, it is inconvenient for the gimbal to identify the image of the target object from the captured image based on less computation, which makes it inconvenient in the related art to control the gimbal to track the rotating target object.
  • the attitude information of the photographing device is determined by means of the IMU and/or the user input attitude information, and the photographed image is processed based on the attitude information, for example, converted to a coordinate system of horizontal shooting, so that it is convenient to
  • the images captured by the photographing device in various postures are used for target object recognition, so as to facilitate the tracking of the target object in various postures.
  • the first resource consumed by recognizing the position of the target object from the image in the current posture is greater than or equal to the second resource consumed by recognizing the location of the target object from the image in the specified posture.
  • the target object recognition process can adopt a variety of related image recognition technologies, for example, through feature extraction, feature comparison, etc., to identify the image of the target object from the captured image, and then determine the position of the target object image in the captured image.
  • the target object in the captured image can be recognized through a deep learning model, a lightweight recognition model, and the like.
  • the intelligent learning module can be set in the PTZ or the load to train the recognition model used for target object recognition to obtain a trained recognition model, and then use the recognition model to recognize the captured images and determine the target object's identity. the location of the image.
  • the position of the target object is identified based on image features of the head and shoulders. This helps to reduce the requirements and consumption of computing resources for the identification process.
  • the pan/tilt is also used to: determine the offset of the target object with respect to each coordinate axis in a specified coordinate system (eg, a pan/tilt coordinate system) based on the position change of the target object in the image captured by the photographing device.
  • a specified coordinate system eg, a pan/tilt coordinate system
  • the offset of the specified coordinate axis in the specified coordinate system is zero-trapped, and the corresponding motor rotation is controlled according to the result of the zero-trapping.
  • the specified coordinate axis is prohibited from driving the bracket components in the selected tracking mode.
  • the coordinate axis corresponding to the motor that moves with the change of the position of the target object within the sensing range of the load.
  • the position change of the target object in the image captured by the capturing device can be determined in the following manner.
  • the target image of the target object is identified from one frame of image, so as to determine the position information of the target image in one frame of image. Then, the amount of positional change between positional information in different frame images is determined.
  • determining the position change amount between the position information in different frame images may include the following operations: based on the first position information of the first target image in the first frame image and the second position information of the second target image in the second frame image The amount of position change between the position information, wherein the first target image is the image of the target object in the first frame image, and the second target image is the image of the target object in the second frame image.
  • identifying a target image of a target object from a frame of images to determine the position information of the target image in a frame of images may include the following operations: first, converting a frame of images in a first coordinate system to a second coordinate system , get a frame of normalized image. Then, the target image of the target object is identified from one frame of normalized image to determine the position information of the target image in one frame of normalized image.
  • the pan/tilt is specifically used for: first, determining the plane deviation of the photographing device based on the first coordinate of the composition target and the second coordinate normalized by the current frame. Then, convert the plane deviation into the offset of each coordinate axis of the target object in the specified coordinate system.
  • the tracking algorithm is exemplified below by taking single yaw axis tracking as an example.
  • FIG. 13 is a data flow diagram of target object tracking provided by an embodiment of the present application.
  • the IMU measures the attitude information of the third-party camera.
  • the measured Euler angles are pitch, roll, and yaw, and the abscissa and ordinate coordinates of a pixel in the recorded image are (x n , y n ), then the converted coordinates It can be shown as formula (1):
  • the user can input the image in the interactive interface as shown in Figure 12.
  • the user inputs the attitude, so that the actual attitude of the load can be determined by combining the attitude measured by the IMU and the user input attitude.
  • the image can be converted into a pose that is convenient for target object recognition based on the above formula (1), and the position of the image of the target object in the captured image can be determined.
  • composition target is (tgt x , tgt y ), the normalized coordinate value of the current frame is (x, y), and the camera plane deviation can be shown as formula (2).
  • NED north, east and earth coordinate system
  • the user can set the degrees of freedom to be followed (pitch Ex, roll E y and yaw E z ) as required.
  • the deviation in the NED (North, East, Earth coordinate system) coordinate system is zero.
  • FIG. 14 is a schematic diagram of diversified movements of a target object provided by an embodiment of the present application.
  • FIG. 15 is a schematic diagram of a captured image of the target object of FIG. 14 .
  • the rider performs stunts in a specific shape of the field.
  • the rider not only has displacements in the yaw and pitch degrees of freedom, but also has a posture change in the roll degree of freedom.
  • Figure 15 it can be a display (or partial display) of a preview image or a captured image.
  • FIG. 15 it is a schematic diagram of the effect of following the rider on the yaw and pitch degrees of freedom. The rider is free to roll The rider is not tracked in degrees.
  • the rider is followed in three degrees of freedom, yaw, pitch and roll, to meet the user's needs for special materials.
  • FIG. 16 is a flowchart of a method for tracking a target object provided by an embodiment of the present application.
  • the method for tracking a target object is applied to a pan/tilt head, the pan/tilt head includes a bracket assembly and at least two motors, the bracket assembly includes at least two relatively movable bracket parts and is used to support a load, and the at least two motors are respectively used to drive corresponding The support member moves to adjust the attitude of the load.
  • the method for tracking a target object may include operations S1602 to S1606.
  • the mode selection instruction may be determined based on a user operation input by the user on the PTZ.
  • the PTZ is provided with buttons, levers and other components, and the user can input mode selection commands by operating these components.
  • a display screen may be included on the PTZ, and a user may input a mode selection command through interactive components (eg, virtual keys, joysticks, etc.) displayed on the display screen.
  • the object to which the user operates may be a pan/tilt communicatively connected to the movable platform.
  • the user inputs at least one of the following information on the PTZ: selection information, attitude information, a specified operation (eg, taking a photo), an object, and parameters of the specified operation (eg, focal length, aperture, exposure duration), and the like.
  • the pan/tilt can be integrated, for example, the remote controller is provided with a processor, a memory, a display screen, and the like.
  • the PTZ can be split.
  • the PTZ can form a control terminal together with other terminal devices.
  • the PTZ and a smartphone are interconnected to form a control terminal.
  • an application APP
  • the smart phone may input operation instructions, set operation parameters, and the like.
  • the specified state instruction can also be determined and input based on gesture recognition, gesture recognition, somatosensory or voice recognition.
  • a user can tilt the head to control the position, attitude, orientation, or other aspects of the movable platform.
  • the tilt of the gimbal can be detected by one or more inertial sensors, and corresponding motion commands can be generated.
  • the user can use the touch screen to adjust the operating parameters of the payload (such as zoom), the attitude of the payload (via the carrier), or other aspects of any object on the movable platform.
  • a current tracking mode is determined from at least two tracking modes, wherein each of the at least two tracking modes is capable of driving the support member according to the position change of the target object within the sensing range of the load.
  • the number of moving motors varies, and the load is placed on the bracket part.
  • the motor corresponding to the current tracking mode is controlled to realize the tracking of the target object by the load in the direction of the specified dimension.
  • the tracking modes include at least two of the following.
  • the number of motors capable of driving the support member to move according to the position change of the target object within the sensing range of the load is one.
  • the number of motors capable of driving the support member to move according to the position change of the target object within the sensing range of the load is two.
  • the number of motors capable of driving the support member to move according to the position change of the target object within the sensing range of the load is three.
  • the motor capable of driving the support member to move with the movement of the target object in the first tracking mode includes a yaw axis motor.
  • the motors capable of driving the support member to move with the movement of the target object include a yaw axis motor and a pitch axis motor.
  • the motors capable of driving the support member to move with the movement of the target object include a yaw axis motor, a pitch axis motor and a roll axis motor.
  • a relatively stable picture in the vertical direction can be provided for the user.
  • the payload includes a camera.
  • the position of the target object on the photographed picture of the photographing device is the central position of the picture.
  • the long sides of the photographing picture are parallel or perpendicular to each other with respect to the ground, or the long sides of the photographing picture are parallel or perpendicular to each other with respect to the horizontal plane.
  • the above method further includes: acquiring attitude information of the load, so as to determine the target object within the sensing range of the load based on at least the attitude information of the load.
  • acquiring the attitude information of the load includes: detecting the attitude information of the load based on the inertial measurement unit.
  • acquiring the attitude information of the load includes: determining the attitude information of the load based on the attitude information input by the user and the attitude information of the load obtained by the inertial measurement unit.
  • the inertial measurement unit is provided on the bracket assembly or the load.
  • the payload is a camera.
  • determining the target object within the sensing range of the load based on at least the attitude information of the load includes: based on the attitude information of the load, converting the image captured by the photographing device under the current attitude to the specified attitude, and from converting to the image under the specified attitude. to identify the location of the target object.
  • the first resource consumed by recognizing the position of the target object from the image in the current posture is greater than or equal to the second resource consumed by recognizing the location of the target object from the image in the specified posture.
  • the embodiments of the present application can realize intelligent tracking of the load at any angle, and can provide users with a more free way of composing pictures.
  • the position of the target object is identified based on image features of the head and shoulders.
  • using the current tracking mode to control the motor corresponding to the current tracking mode to realize the tracking of the target object by the load in the direction of the specified dimension may include the following operations.
  • the offset of the target object with respect to each coordinate axis in the specified coordinate system is determined.
  • the offset of the specified coordinate axis in the specified coordinate system is zero-trapped, and the corresponding motor rotation is controlled according to the result of the zero-trapping.
  • the specified coordinate axis is prohibited from driving in the selected tracking mode.
  • determining the offset of the target object with respect to each coordinate axis in the specified coordinate system may include the following operations.
  • the plane deviation of the photographing device is determined based on the first coordinate of the composition target and the second coordinate normalized by the current frame.
  • obtaining the mode selection instruction may include the following manner.
  • the tracking mode is determined in response to a mode selection instruction received from the user interface.
  • the tracking mode is determined in response to a mode selection operation for a preset function key.
  • various tracking modes can be combined with the intelligent tracking algorithm.
  • the user can be provided with various tracking modes for the user to choose by himself, so that the user can select a suitable tracking mode according to his own needs and the operation level of the UAV.
  • Mode in order to meet the convenience of shooting, you can use a variety of mirror movement methods to improve the shooting experience. It can provide users with rich application of mirror movement techniques.
  • the execution subject of each of the above operations may be a handheld pan/tilt, and the corresponding functions may be implemented by an input unit of the handheld pan/tilt, a holding part, a processor and a motor provided in the holding part, and the like.
  • the operation of acquiring the mode selection instruction may be determined based on the user operation input by the user on the control terminal of the movable platform or on the movable platform.
  • the operation of determining the current tracking mode from the at least two tracking modes may be determined by the control terminal of the movable platform, the PTZ or the movable platform (eg, a processor).
  • the operation of controlling the motor corresponding to the current tracking mode may be performed by the PTZ.
  • the operation of obtaining the attitude information of the load can be determined by the control terminal of the movable platform (such as the user input attitude), the PTZ (such as the attitude detected by the IMU) or the movable platform (such as the fusion of multiple attitudes by the processor) .
  • Determining the operation of the target object within the sensing range of the payload based on at least the attitude information of the payload may be determined by the control terminal of the movable platform, the PTZ, the payload or the movable platform.
  • the operation of image processing and target object recognition can be determined by the control terminal of the movable platform, the pan/tilt, the load or the movable platform.
  • the execution subjects of the above operations are only exemplary descriptions, and should not be construed as limitations on this application, and may be independently completed by one of the movable platform, the control terminal, the photographing device, and the PTZ, or several of them.
  • the cooperation is complete.
  • a human-computer interaction module such as a display for displaying a human-computer interaction interface, etc.
  • the independent completion includes actively or passively, directly or indirectly acquiring corresponding data from other devices to perform corresponding operations.
  • the tracking method of the target object provided by the embodiment of the present application, through the combined application of various PTZ following modes and intelligent tracking of the handheld PTZ, and through algorithm processing, the control methods in various following modes are obtained, so that the user can A method for realizing intelligent follow-up control.
  • the embodiments of the present application can perform automatic tracking at any angle, including FPV mode and vertical shooting mode.
  • the above method can also be applied to loads other than the photographing device, etc., to track the target object.
  • the load includes, but is not limited to, an acoustic wave detection device, an infrared detection device, and the like.
  • FIG. 17 is a block diagram of an apparatus for tracking a target object according to an embodiment of the present application.
  • the apparatus 1700 for tracking a target object may include one or more processors 1710, and the one or more processors 1710 may be integrated in one processing unit, or may be separately provided in multiple processing units.
  • the apparatus 1700 for tracking a target object may be set in one execution body or respectively set in multiple execution bodies.
  • the apparatus 1700 for tracking a target object may be provided in a pan/tilt head.
  • part of it is set in the PTZ, and part is set in a control terminal that can be connected to the PTZ, such as a display screen for displaying an interactive interface.
  • the processing unit may comprise a Field-Programmable Gate Array (FPGA) or one or more ARM processors.
  • the processing unit may be connected to non-volatile computer readable storage medium 1720.
  • the non-volatile computer-readable storage medium 1720 may store logic, code, and/or computer instructions executed by the processing unit for performing one or more steps.
  • the non-volatile computer readable storage medium 1720 may include one or more storage units (removable media or external memory such as SD card or RAM).
  • the data sensed by the sensors may be transferred and stored directly into a storage unit of the non-volatile computer-readable storage medium 1720 .
  • the storage units of the non-volatile computer-readable storage medium 1720 may store logic, code, and/or computer instructions executed by the processing unit to perform various embodiments of the various methods described herein.
  • a processing unit may be configured to execute instructions to cause one or more processors of the processing unit to perform the tracing functions described above.
  • the storage unit may store sensing module sensing data, the data sensing being processed by the processing unit.
  • the storage unit of the non-volatile computer-readable storage medium 1720 may store processing results generated by the processing unit.
  • the processing unit may be connected to the control module for controlling the state of the movable platform.
  • the control module may be used to control the power mechanism of the movable platform to adjust the spatial orientation, velocity and/or acceleration of the movable platform relative to six degrees of freedom.
  • the control module may control one or more of the carrier, load or sensing module.
  • the processing unit may also be connected to the communication module for transmitting and/or receiving data with one or more peripheral devices (eg, terminals, display devices, or other remote control devices).
  • peripheral devices eg, terminals, display devices, or other remote control devices.
  • Any suitable communication method may be utilized here, such as wired communication or wireless communication.
  • the communication module may utilize one or more local area networks, wide area networks, infrared, radio, Wi-Fi, peer-to-peer (P2P) networks, telecommunication networks, cloud networks, and the like.
  • P2P peer-to-peer
  • a relay station such as a signal tower, a satellite, or a mobile base station, can be used.
  • the above-mentioned various components may be compatible with each other.
  • one or more components are located on a movable platform, carrier, payload, terminal, sensing system, or additional external device in communication with each of the foregoing.
  • one or more of the processing unit and/or non-transitory computer-readable medium may be located in different locations, such as on a removable platform, carrier, payload, terminal, sensing system, or Additional external devices that communicate with the foregoing devices and various combinations of the foregoing.
  • control terminal adapted to the movable platform may include an input module, a processing unit, a memory, a display module, and a communication module, all of which are connected by a bus or similar network.
  • the input module includes one or more input mechanisms to obtain input generated by the user by manipulating the input module.
  • Input mechanisms include one or more joysticks, switches, knobs, slide switches, buttons, dials, touchscreens, keypads, keyboards, mice, voice controls, gesture controls, inertial modules, and the like.
  • the input module may be used to obtain user input for controlling the movable platform, carrier, load, or any aspect of the components therein. Any aspect includes attitude, position, orientation, flight, tracking, etc.
  • the input mechanism may be that the user manually sets one or more positions, each position corresponding to a preset input, to control the movable platform.
  • the input mechanism may be operated by a user to input control commands to control the movement of the movable platform.
  • a user can use a knob, switch, or similar input mechanism to input a motion mode of the movable platform, such as auto-flying, auto-pilot, or moving according to a preset motion path.
  • the user can control the position, attitude, orientation, or other aspects of the movable platform by tilting the control terminal in a certain way.
  • the tilt of the control terminal can be detected by one or more inertial sensors, and corresponding motion commands can be generated.
  • the user may utilize the input mechanisms described above to adjust operational parameters of the payload (eg, zoom), the attitude of the payload (via the carrier), or other aspects of any object on the movable platform.
  • the input mechanism may be operated by the user to input the aforementioned descriptive object information.
  • the user may select an appropriate tracking mode, such as a manual tracking mode or an automatic tracking mode, using a knob, switch, or similar input mechanism.
  • the user may also utilize this input mechanism to select a specific target to be tracked, target type information to execute, or other similar information.
  • the input module may be executed by more than one device.
  • the input module can be implemented by a standard remote controller with a joystick.
  • a standard remote controller with a joystick connects to a mobile device (eg, a smartphone) running a suitable application ("app") to generate control commands for the movable platform.
  • the app can be used to get input from the user.
  • the processing unit may be connected to the memory.
  • Memory includes volatile or non-volatile storage media for storing data, and/or logic, code, and/or program instructions executable by a processing unit for performing one or more rules or functions.
  • the memory may include one or more storage units (removable media or external memory such as SD card or RAM).
  • the data input to the module may be directly transferred and stored in a storage unit of the memory.
  • the storage units of the memory may store logic, code and/or computer instructions executed by the processing unit to perform various embodiments of the various methods described herein.
  • the processing unit may be configured to execute instructions to cause one or more processors of the processing unit to process and display sensory data (eg, images) obtained from the movable platform, control commands generated based on user input, including motion commands and objects information, and cause the communication module to transmit and/or receive data, etc.
  • the storage unit may store sensed data or other data received from an external device such as a removable platform.
  • the storage unit of the memory may store the processing result generated by the processing unit.
  • the display module may be used to display the information on the position, translation velocity, translation acceleration, orientation, angular velocity, angular acceleration, or a combination thereof, etc., for the gimbal and/or the load as described above.
  • the display module can be used to obtain information sent by the movable platform and/or payload, such as sensory data (images recorded by cameras or other image capture devices), described tracking data, control feedback data, and the like.
  • the display module may be executed by the same device as the input module. In other embodiments, the display module and the input module may be executed by different devices.
  • the communication module may be used to transmit and/or receive data from one or more remote devices (eg, removable platforms, carriers, base stations, etc.).
  • the communication module can transmit control signals (such as motion signals, target information, and tracking control commands) to peripheral systems or devices, such as the above-mentioned PTZ and/or load.
  • the communication module may include a transmitter and a receiver for receiving data from and transmitting data to the remote device, respectively.
  • the communication module may include a transceiver that combines the functions of a transmitter and a receiver.
  • the transmitter and receiver and the processing unit may communicate with each other. Communication may utilize any suitable means of communication, such as wired or wireless communication.
  • Images captured by the movable platform during motion can be transmitted from the movable platform or imaging device back to a control terminal or other suitable device for display, playback, storage, editing, or other purposes. Such transmission may occur in real-time or near real-time as the imaging device captures the imagery. Optionally, there may be a delay between the capture and transmission of the imagery.
  • the imagery may be stored in the removable platform's memory without being transferred anywhere else. The user can view these images in real time and, if necessary, adjust target information or other aspects of the movable platform or its components. Adjusted object information may be provided to the movable platform, and the iterative process may continue until the desired image is obtained.
  • the imagery may be transmitted to a remote server from the removable platform, the imagery device, and/or the control terminal. For example, images can be shared on some social networking platforms, such as WeChat Moments or Weibo.
  • the apparatus 1700 for tracking a target object may be used to perform one or more operations as described above. I will not list them one by one here.
  • FIG. 18 is a schematic structural diagram of a movable platform provided by an embodiment of the present application.
  • the movable platform may be an unmanned aerial vehicle 180, and the unmanned aerial vehicle 180 may include a plurality of power systems 181 and a tripod.
  • the gimbal can be set on the drone 180.
  • the power systems 181 of the UAV 180 correspond to the arms one-to-one.
  • Each power system 181 may include a motor assembly and a blade coupled to the motor assembly.
  • Each power system 181 may be disposed on its corresponding arm, and the power system 181 is supported by the corresponding arm.
  • the drone 180 may also include a tripod.
  • the tripod can be located under the gimbal and connected with the gimbal. When the drone 180 lands, it can be used for the drone 180 to land.
  • FIG. 19 schematically shows a schematic diagram of a movable platform according to another embodiment of the present application.
  • the movable platform is a hand-held pan-tilt 190
  • the hand-held pan-tilt 190 may include the structure of the pan-tilt as described above.
  • the hand-held pan/tilt 190 may include: a pan/tilt and a handle supporting the pan/tilt, the handle is a part that the user holds, and may include control buttons to facilitate the operation of the pan/tilt.
  • the handheld pan/tilt 190 is connected in communication with a functional component (such as a camera) in the stand, so as to obtain image information captured by the camera.
  • the handheld PTZ 190 can also be connected with a terminal device 191 (such as a mobile phone), etc., so as to send information such as images to the mobile phone.
  • a terminal device 191 such as a mobile phone

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)

Abstract

一种跟踪目标对象的方法和云台，该云台包括：支架组件，包括至少两个能够相对活动的支架部件，所述支架组件用于支撑负载；至少两个电机，分别用于驱动对应的所述支架部件运动，以调整所述负载的姿态；其中，所述云台具有至少两个跟踪模式，至少两个所述跟踪模式下各自能够驱动所述支架部件随目标对象在所述负载的感测范围内的位置变化而发生运动的电机的数量不同，以在不同维度的方向上实现对所述目标对象的跟踪。

Description

跟踪目标对象的方法和云台 技术领域
本申请涉及控制技术领域，尤其涉及一种跟踪目标对象的方法和云台。
背景技术
云台可以用于搭载负载，并且调节负载的姿态，例如，改变负载的高度、倾角和/或方向。负载需要跟踪的目标对象的运动通常是多样化的，相关技术可以通过云台改变负载的姿态以自动跟踪目标对象，这样有效提升了使用负载追踪目标对象的便捷度。但是，在一些场景下，用户可能会需要更多针对负载的控制灵活度，例如，便于用户利用搭载在云台上的拍摄装置以多种拍摄手法对目标对象进行跟拍。
如何控制手持云台的运动，以满足用户针对负载的控制灵活度的需求，并且可以实现对多样化运动的目标对象的跟踪，是一项亟待解决的问题。
发明内容
有鉴于此，本申请实施例提供一种跟踪目标对象的方法和云台，以满足用户针对负载的控制灵活度的需求，并且可以实现对多样化运动的目标对象的跟踪。
第一方面，本申请实施例提供了一种云台，包括：支架组件，包括至少两个能够相对活动的支架部件，支架组件用于支撑负载；至少两个电机，分别用于驱动对应的支架部件运动，以调整负载的姿态；其中，云台具有至少两个跟踪模式，至少两个跟踪模式下各自能够驱动支架部件随目标对象在负载的感测范围内的位置变化而发生运动的电机的数量不同，以在不同维度的方向上实现对目标对象的跟踪。
第二方面，本申请实施例提供了一种跟踪目标对象的方法，用于云台，云台包括支架组件和至少两个电机，支架组件包括至少两个能够相对活动的支架部件，且用于支撑负载，至少两个电机分别用于驱动对应的支架部件运动，以调整负载的姿态。该方法包括：获取模式选取指令；响应于模式选取指令，从至少两个跟踪模式中确定当前跟踪模式，其中，至少两个跟踪模式下各自能够驱动支架部件随目标对象在负载的感测范围内的位置变化而发生 运动的电机的数量不同，负载设置在支架部件上；利用当前跟踪模式，控制当前跟踪模式对应的电机，以在指定维度的方向上实现负载对目标对象的跟踪。
第三方面，本申请实施例提供了一种计算机可读存储介质，其存储有可执行指令，可执行指令在由一个或多个处理器执行时，可以使一个或多个处理器执行如上的方法。
第四方面，本申请实施例提供了一种计算机程序，包括可执行指令，该可执行指令在被执行时，实现如上的方法。
在本申请实施例中，云台具有至少两个跟踪模式，至少两个跟踪模式下各自能够驱动支架部件随目标对象在负载的感测范围内的位置变化而发生运动的电机的数量不同，使得用户可以通过采用不同的跟踪模式的方式，来便捷地调整用户针对负载的控制灵活度，并且可以在不同维度的方向上实现对目标对象的跟踪。
本申请的附加方面的优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本申请的实践了解到。
附图说明
通过参照附图的以下详细描述，本申请实施例的上述和其他目的、特征和优点将变得更容易理解。在附图中，将以示例以及非限制性的方式对本申请的多个实施例进行说明，其中：
图1为本申请实施例提供的跟踪目标对象的方法和云台的应用场景；
图2为本申请另一实施例提供的跟踪目标对象的方法和云台的应用场景；
图3为本申请实施例提供的云台的结构示意图；
图4为本申请实施例提供的云台上搭载负载的示意图；
图5为本申请实施例中目标对象距离拍摄装置较远时示意图；
图6为本申请实施例中目标对象距离拍摄装置较近时示意图；
图7为本申请实施例提供的目标对象距离拍摄装置较近时示意图；
图8为本申请实施例中目标对象从拍摄装置侧面经过过程中拍摄装置的视角的示意图；
图9为本申请另一实施例提供的目标对象从拍摄装置侧面经过过程中拍 摄装置的视角的示意图；
图10为本申请实施例提供的拍摄装置的光轴和横滚轴之间不重合的示意图；
图11是针对图10的目标对象的中心在多个拍摄图像中运动轨迹的示意图；
图12为本申请实施例提供的交互界面的示意图；
图13为本申请实施例提供的目标对象跟踪的数据流图；
图14为本申请实施例提供的目标对象的多样化运动的意图；
图15为针对图14的目标对象的拍摄图像的示意图；
图16为本申请实施例提供的跟踪目标对象的方法的流程图；
图17为本申请实施例提供的跟踪目标对象的装置的方框图；
图18为本申请实施例提供的具有动力系统的可移动平台的示意图；以及
图19为本申请实施例提供的云台和终端设备的示意图。
具体实施方式
下面详细描述本申请的实施例，实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，旨在用于解释本申请，而不能理解为对本申请的限制。
以影像拍摄场景为例，本申请实施例可以适用于用户对目标对象进行指定自由度的跟拍，其中，指定自由度可以是单个自由度、两个自由度或三个自由度等。例如，随着目标对象(如被拍摄对象)由远至近的移动向用户的过程中，在用户的摄像装置拍摄的图像中，拍摄对象的面部图像在拍摄的图像中位置(包括左右位置和上下位置)会发生变化，云台可以控制自身的至少部分电机作动，以使得拍摄装置对目标对象进行跟踪。例如，随着拍摄对象距离用户越来越近，不但其面部图像在拍摄图像中的左右位置会发生变化，其面部图像在拍摄图像中的高度位置同样会发生变化。相关技术中对拍摄对象的面部进行跟踪时，可以使得拍摄对象的面部图像始终位于图像的中心位置，以实现跟拍功能。但是，这样同时也造成不便于用户介入诸如针对背景图像的构图等。因不便于用户实施多种运镜手法，可能无法达到用户期望的 拍摄效果。
需要说明的是，以上场景仅为示例性的场景，还可以是针对动物拍摄、图像抓拍、视频拍摄、电影拍摄、电视剧拍摄等的拍摄场景、扫描场景等，不能理解为对本申请的限制。
为了便于更好地理解本申请的实施例，首先对相关技术中云台和基于云台的跟拍技术进行示例性说明。
云台的出现，不仅为拍照、录像过程提供了稳定的拍摄环境，为摄影爱好者提供了丰富的运镜可能性。例如，手持云台可以使得设置在其上的相机围绕偏航轴、俯仰轴和横滚轴中至少一个轴进行转动，以达到特定运镜拍摄效果，如“跟手”运镜。
然而，用户使用云台的运镜手法有一定的学习成本，初级用户常常会得不到想要的画面，不能把被拍对象稳定地置于画面内。但是，得益于智能跟随算法的发展，目前云台与智能跟随算法结合后，可以使被拍对象很好地置于画面内，提高拍摄质量的同时大大提高了拍摄的灵活性。
例如，相关技术中云台与智能跟随算法的结合应用，使得在用户移动或转动云台时，云台自动调整偏航轴和俯仰轴以调整拍摄装置的姿态，使目标对象始终置于画面的固定位置处。
然而，这种方式也存在一些缺点，由于将拍摄画面的大部分控制权交给了云台算法，用户只能控制云台的位置，并不能作更多的运镜手法。例如在拍摄人物和建筑物的场景，用户想只在平移方向进行智能跟随，目前的智能跟随算法无法实现。
例如，相关技术中手持云台的智能跟随的应用，是通过平移轴和俯仰轴的运动以调整云台角度，各轴的跟随方式不能单独设置。
又例如，“平移轴+俯仰轴”的智能跟随控制方式得出的画面有时并不是摄影师想要的效果，比如拍摄一个逐渐走近的人像时，俯仰轴的智能跟随控制会使画面在竖直方向移动，导致取景内容产生变化。
又例如，当相机不处于横拍模式(如竖拍，或第一人称主视角(First Person View，简称FPV)模式下旋转画面的拍摄)，由于智能跟随中的机器学习无法得知此信息，导致其还是采用横拍模式进行识别，因此会导致识别目标不准确的问题。
又例如，部分手持云台支持第三方摄影设备(如手机、相机)，经过调平后，其横滚轴旋转中心和摄像设备光轴中心不重合，在FPV旋转运镜中，无法拍出用户预想的画面。
本申请实施例提供的云台和跟踪目标对象的方法，提出了智能跟随算法与多种云台跟踪模式的结合应用，能够使云台在用户进行特定操作，如运镜时，负载仍然在所需自由度下跟踪被拍对象，保证画面内容的前提下丰富了用户的拍摄手段，降低用户拍摄出更高规格的素材的难度。
为了便于理解本申请的技术方案，以下结合图1～图19进行详细说明。
图1为本申请实施例提供的跟踪目标对象的方法和云台的应用场景。
如图1所示，随着骑手在由远处D1至近处D2的骑行向拍摄者的场景中，骑手的面部图像在拍摄者拍摄的图像中的位置(包括左右位置和上下位置)会发生变化，云台可以控制自身的至少部分电机驱动拍摄装置，以对骑手进行跟拍。其中，云台可以是由拍摄者手持的云台，还可以是设置在可移动平台上的云台，如设置在无人机等上的云台，在此不做限定。拍摄者在进行拍摄的过程中同样可以对可移动平台进行控制，如进行飞行轨迹控制等。
相关技术中对骑手的面部进行跟踪时，可以使得骑手的面部图像始终位于图像的中心位置，以实现跟拍功能。但是，由于将拍摄画面的大部分控制权交给了云台算法，用户只能控制云台的位置，不便于对诸如背景图像等进行构图，可能无法达到拍摄者期望的拍摄效果。
本申请实施例提供的跟踪目标对象的方法和云台，可以通过跟踪模式选取的方式，实现由用户自主选取可以驱动负载对目标对象进行跟踪的电机，实现了多种自由度的自由选取，便于在实现追踪的同时，实施所需的运镜手法等，以拍摄所期望的图像。例如，可以实现单偏航轴的智能跟随控制，为用户提供垂直方向上较稳定的画面。此外，本申请实施例提供的跟踪目标对象的方法和云台，还可以实现任意角度的智能跟随控制方法能够为用户提供更自由的构图方式。
图2为本申请另一实施例提供的跟踪目标对象的方法和云台的应用场景。如图2所示，以可以承载第三方负载的云台为例进行说明。需要说明的是，第三方负载也可以是与云台一体的，在此不做限定。
图2中云台可以包括承载体200和手柄201等。其中，承载体200可以 包括配套的电机和轴臂，电机用于驱动轴臂旋转，以驱动第三方负载的运动。
承载体200可以是多种支撑结构，包括但不限于针对单轴或多轴的姿态可调结构，用于将负载设置在手柄201上。例如，负载可以是拍摄装置，该云台允许拍摄装置相对于手柄201发生位移，或者，沿着一个或多个轴转动，如承载体200允许拍摄装置沿着俯仰轴、航向轴和横滚轴中一个轴或多个轴的结合平移运动。又例如，承载体200可以允许拍摄装置围绕俯仰轴、航向轴和横滚轴中的一个或多个轴转动。其中承载体200和手柄201之间可以具有联动换算关系，如手柄201发生的第一运动(如移动或转动)可以换算成承载体200发生的第二运动。反之亦然。
此外，云台上还可以包括传感系统。传感系统可以包括一个或者多个传感器，以感测空间方位、速度及/或加速度(如相对于多达三个自由度的旋转及平移)。一个或者多个传感器包括但不限于GPS传感器、运动传感器、惯性传感器或者影像传感器。传感系统提供的感测数据可以用于控制负载的位姿、速度及/或加速度等。可选地，传感系统可以用于检测云台的环境的数据，如气候条件、人造结构的位置等。
此外，云台上还可以包括通讯系统。通讯系统能够实现云台与具有通讯系统的控制终端通过有线或无线收发的信号进行通讯。通讯系统可以包括任何数量的用于无线通讯的发送器、接收器、及/或收发器。通讯可以是单向通讯，这样数据可以从一个方向发送。例如，单向通讯可以包括，只有云台传送数据给控制终端，或者反之亦然。通讯系统的一个或者多个发送器可以发送数据给通讯系统的一个或者多个接收器，反之亦然。可选地，通讯可以是双向通讯，这样，数据可以在云台与控制终端之间在两个方向传输。双向通讯包括通讯系统的一个或者多个发送器可以发送数据给通讯系统的一个或者多个接收器，及反之亦然。
在某些实施例中，控制终端可以与云台或负载相连，控制终端可以向云台及负载中的一个或者多个提供控制指令，并且从云台及负载中的一个或者多个中接收信息(如承载体200或者负载的位置及/或运动信息，负载感测的数据，如拍摄装置捕获的影像数据)。在某些实施例中，控制终端的控制数据可以包括关于位置、运动、制动的指令，或者对云台及/或负载的控制。例如，控制数据可以导致承载体200位置及/或方向的改变。控制终端的控制数据可 以导致负载控制，如控制拍摄装置或者其它影像捕获设备的操作(捕获静止或者运动的影像、变焦、开启或关闭、切换成像模式、改变影像分辨率、改变焦距、改变景深、改变曝光时间、改变可视角度或者视场)。在某些实施例中，对云台及/或负载的通讯可以包括一个或者多个传感器发出的信息。通讯可以包括从一个或者多个不同类型的传感器(如GPS传感器、运动传感器、惯性传感器、近程传感器或者影像传感器)传送的感应信息。感应信息是关于对云台及/或负载的位置(如方向、位置)、运动、或者加速度。从负载传送的感应信息包括负载捕获的数据或者负载的状态。控制终端传送提供的控制数据可以用于控制云台、承载体200或者负载中一个或者多个的状态。可选地，承载体200及负载中一个或多个可以包括通讯模块，用于与控制终端通讯，以便控制终端可以单独地通讯或者控制云台及负载。其中，控制终端可以为云台的遥控器，也可以为诸如手机、iPad、可穿戴电子设备等能够用于控制云台的智能电子设备。
需要说明的是，控制终端可以远离云台，以实现对云台的远程控制，可以固定或可拆卸地设于云台上，具体可以根据需要设置。
在某些实施例中，云台可以与除了控制终端之外的其它远程设备，或者非控制终端的远程设备通讯。控制终端也可以与另外一个远程设备及云台进行通讯。例如，云台及/或控制终端可以与另一个可移动平台或者另一个可移动平台的承载体或负载通讯。当有需要的时候，另外的远程设备可以是第二终端或者其它计算设备(如计算机、桌上型电脑、平板电脑、智能手机、或者其它移动设备)。该远程设备可以向云台传送数据，从云台接收数据，传送数据给控制终端，及/或从控制终端接收数据。可选地，该远程设备可以连接到因特网或者其它电信网络，以使从云台及/或控制终端接收的数据上传到网站或者服务器上。
图3为本申请实施例提供的云台的结构示意图。
如图3所示，该云台300可以包括：支架组件31和至少两个电机32。其中，支架组件31可以包括至少两个能够相对活动的支架部件311、312，支架组件31用于支撑负载40。至少两个电机32，分别用于驱动对应的支架部件运动，以调整负载40的姿态。其中，云台300具有至少两个跟踪模式，至少两个跟踪模式下各自能够驱动支架部件随目标对象在负载40的感测范 围内的位置变化而发生运动的电机32的数量不同，以在不同维度的方向上实现对目标对象的跟踪。
例如，俯仰(pitch)轴电机和俯仰轴臂相配合以驱动负载40绕俯仰轴转动。横滚(roll)轴电机和横滚轴臂相配合以驱动负载40绕横滚轴转动。偏航(yaw)轴电机和偏航轴臂相配合以驱动负载40绕偏航轴转动。
其中，俯仰轴电机可以带动俯仰轴臂的运动，横滚轴电机可以带动横滚轴臂的运动，偏航轴电机可以带动偏航轴臂的运动。
例如，偏航轴臂可以连接于横滚轴臂的一端，横滚轴臂的另一端连接于俯仰轴臂，但是本申请实施例并不限于此，偏航轴臂、横滚轴臂和俯仰轴臂也可以以其它顺序进行连接。
应当理解的是，云台300也可以使得负载仅能围绕一个、两个或四个轴等进行进行转动，在此不做限定。
在一个实施例中，跟踪模式包括以下所示的第一跟踪模式至第三跟踪模式中至少两种。
其中，第一跟踪模式，第一跟踪模式下能够驱动支架部件随目标对象在负载的感测范围内的位置变化而发生运动的电机的数量是一个。例如，该电机可以是俯仰轴电机、偏航轴电机或横滚轴电机中任意一个。这样就使得负载在第一跟踪模式只有一个自由度，如在偏航轴电机的驱动下跟踪目标对象。
第二跟踪模式，第二跟踪模式下能够驱动支架部件随目标对象在负载的感测范围内的位置变化而发生运动的电机的数量是两个。例如，该电机可以是俯仰轴电机、偏航轴电机或横滚轴电机中任意两个。这样就使得负载在第二跟踪模式有两个自由度，如在偏航轴电机和/或俯仰轴电机的驱动下跟踪目标对象。
第三跟踪模式，第三跟踪模式下能够驱动支架部件随目标对象在负载的感测范围内的位置变化而发生运动的电机的数量是三个。例如，该电机可以是俯仰轴电机、偏航轴电机或横滚轴电机中任意一个或多个。这样就使得负载在第三跟踪模式有三个自由度，如在偏航轴电机、俯仰轴电机和横滚轴电机中至少一个的驱动下跟踪目标对象。需要说明的是，当横滚轴电机驱动镜头组围绕横滚轴转动时，偏航轴电机和俯仰轴电机可以用于调节负载的位置，以改善由于镜头组的光轴与横滚轴之间不重叠导致目标对象的图像中心与拍 摄图像的图像中心不重叠造成的画面视角效果不佳的问题。
在一个实施例中，第一跟踪模式下能够驱动支架部件随目标对象的运动而发生运动的电机包括偏航轴电机。
第二跟踪模式下能够驱动支架部件随目标对象的运动而发生运动的电机包括偏航轴电机和俯仰轴电机。又例如，第二跟踪模式下能够驱动支架部件随目标对象的运动而发生运动的电机包括横滚轴电机和俯仰轴电机。又例如，第二跟踪模式下能够驱动支架部件随目标对象的运动而发生运动的电机包括偏航轴电机和横滚轴电机。
第三跟踪模式下能够驱动支架部件随目标对象的运动而发生运动的电机包括偏航轴电机、俯仰轴电机和横滚轴电机。
需要说明的是，当云台还能驱动负载发生平移运动时，本申请的实施例不排除将驱动平移的动力部件(如气缸、液缸或线性马达等)与上述任一跟踪模式对应的至少一个电机相结合的实施方案。
图4为本申请实施例提供的云台上搭载负载的示意图。
如图4所示，云台还可以包括：负载固定机构50。其中，该负载固定机构50用于固定负载40，负载40姿态可调地固定在负载固定机构50上。
为了便于用户采取横拍模式或竖拍模式进行拍摄，拍摄装置(如相机)可以横向或纵向设置在云台上。具体地，负载的第一固定面和负载固定机构的第二固定面之间相互平行或相互垂直。
其中，负载固定机构50可相对一个或多个轴臂转动。例如，负载固定机构包括一可相对俯仰轴转动的转动臂以及可与拍摄装置配合的固定部。例如，固定部可以相对转动臂发生线性移动，以便于对不同尺寸或不同构造的负载进行固定。需要说明的是负载固定机构50可以是单独的一个部件，也可以是某个轴臂的一部分，如负载固定机构50可以是俯仰轴臂或偏航轴臂的组成部分，在此不做限定。
例如，在需要将负载加载在云台上时，可以先将拍摄装置固定在转动臂上，调节固定部的位置，以使固定部能与拍摄装置的定位部相配合后，将拍摄装置固定在指定的位置，以使拍摄装置设置于负载固定机构上。
在一个实施例中，该云台还可以包括：惯性测量单元(Inertial measurement unit，简称IMU)。该惯性测量单元可以设置在云台上任意一处，以用于确定 设置惯性测量单元支持的部件的姿态信息。例如，为了便于确定负载的姿态信息，该惯性测量单元可以设置在负载固定机构50上，用于测量固定机构50的姿态信息。又例如，为了便于确定轴臂的姿态信息，可以将惯性测量单元设置在轴臂上。惯性测量单元可以是加速度计或陀螺仪中的至少一种，可以用于测量拍摄装置的姿态和加速度等。
在一个实施例中，如果负载40和云台是一体的，则为了便于测量负载40的位姿，可以将惯性测量单元可以设置在负载40上，用于测量负载的位姿信息。
需要说明的是，惯性测量单元可以是一个或多个，各惯性测量单元可以分别设置在不同的部件上，以便于测量该部件的位姿信息。
在一个实施例中，该云台可以包括手持云台、机载云台等。
例如，支架组件用于固定在具有动力系统的可移动平台上。以可移动平台是无人机为例进行说明。该可移动平台可以包括包括动力机构，传感系统。此外，该可移动平台还可以包括通讯系统。
其中，动力机构可以包括一个或者多个旋转体、螺旋桨、桨叶、引擎、电机、轮子、轴承、磁铁、喷嘴。例如，动力机构的旋转体可以是自紧固(self-tightening)旋转体、旋转体组件、或者其它的旋转体动力单元。可移动平台可以有一个或者多个动力机构。所有的动力机构可以是相同的类型或不同的类型。动力机构能够使可移动平台垂直地从表面起飞，或者垂直地降落在表面上，而不需要可移动平台任何水平运动(如不需要在跑道上滑行)。例如，可移动平台可以有多个水平方向的旋转体，以控制可移动平台的提升及/或推动。传感系统可以包括一个或者多个传感器，以感测可移动平台的周边障碍物、空间方位、速度及/或加速度(如相对于多达三个自由度的旋转及平移)。通讯系统可以参考云台的通讯系统相关部分内容，在此不再赘述。
又例如，云台还包括：把持组件60，把持组件60用于支撑支架组件31。把持组件60除了可以起到支撑支架组件31的作用之外，还可以起到诸如容置电池、处理器、设置输入/输出部件等，在此不做限定。
参考图4所示，该云台可以包括pitch轴电机322、roll轴电机323、yaw轴电机321、把持组件60、yaw轴轴臂311、负载固定机构50(内部可以包含惯性测量元件)、pitch轴轴臂312、roll轴轴臂313、拍摄装置40等。
以下结合图5至图11对基于云台对目标对象进行跟踪的过程进行示例性说明。
图5为本申请实施例中目标对象距离拍摄装置较远时示意图。图6为本申请实施例中目标对象距离拍摄装置较近时示意图。
例如，采用“偏航轴+俯仰轴”的智能跟随控制方式可以得到如图6所示的拍摄图像，骑手的图像始终会保持在拍摄图像的中央位置。然而，在某些场景下，该得到的画面并不是摄影师想要的效果，比如拍摄一个逐渐骑近的骑手时，参考图6所示，随着骑手距离相机越来越近，骑手的图像在相机拍摄的画面中图像会变得越来越大，导致骑手的面部图像逐渐朝相机拍摄图像的靠上位置移动，为了使得相机跟踪骑手(面部或身体中心等)，则俯仰轴的智能跟随控制会使画面在竖直方向Z移动，导致取景内容产生变化。如果用户希望拍摄的视频中是背景比较稳定，并且，在X方向和Y方向上追踪骑手，则相关技术无法简便实现该拍摄效果。
图7为本申请实施例提供的目标对象距离拍摄装置较近时示意图。
如图7所示，本申请实施例可以对Z轴方向的自由度进行限制，使得云台仅能控制Y轴对应的电机和X轴对应的电机以跟踪目标对象，则可实现如图7所示的效果：随着骑手距离用户越来越近，期间相机一直在Y轴和X轴方向跟踪骑手，但是，Z轴方向上保持不变，则可以实现在Y轴和X轴方向跟踪骑手的同时，保持取景稳定，满足用户的特定拍摄效果需求。
图8为本申请实施例中目标对象从拍摄装置侧面经过过程中拍摄装置的视角的示意图。图9为本申请另一实施例提供的目标对象从拍摄装置侧面经过过程中拍摄装置的视角的示意图。
为了更清楚地理解上述跟拍过程，结合图8和图9进行说明。参考图8所示，随着骑手的移动，骑手的图像始终保持在拍摄图像的中央位置，实现了针对骑手的跟拍效果。然而，在一些场景下，该跟拍效果还无法满足用户的拍摄效果需求。例如，随着骑手距离用户越来越近，其在用户拍摄的图像中占据的图像比例越来越大，同时，导致用户的面部图像距离拍摄图像的顶部越来越近，甚至会接近或超过拍摄图像的顶部。图8中云台会在Z轴方向调整相机的位姿，使得骑手的面部等始终保持在拍摄图像的中心位置。导致拍摄图像的背景从图8的左边圆圈的背景图像变为图8的右边圆圈的背景图 像，造成背景的取景范围明显上移，如果用户的运镜手法不够娴熟，没能及时将相机的高度逐渐提升，则会出现上述情形。
参考图9所示，本申请实施例中，由于限制了云台在Z轴方向的自由度，使得云台可以不在Z轴方向跟踪骑手，而仅在X轴和Y轴方向跟踪骑手，能降低运镜操作的难度，便于获取用户所期望的拍摄效果。
以下对本申请实施例有助于实现在三轴自由度下跟踪目标对象时，使得目标对象的图像保持在拍摄图像中央处进行示例性说明。
图10为本申请实施例提供的拍摄装置的光轴和横滚轴之间不重合的示意图。图11是针对图10的目标对象的中心在多个拍摄图像中运动轨迹的示意图
如图10所示，由于负载可以是由第三方提供的，如用户自行购买的相机等，不同厂商生产的相机的尺寸不同，不易通过调校的方式使得相机的光轴Op1或光轴Op2与云台的横滚轴Roll轴重叠。此外，即使通过调校使得相机的光轴Op1、Op2与云台的横滚轴Roll轴重叠，但是由于不同型号的相机的重心等不同，在相机被驱动旋转的过程中，很可能导致光轴Op1、Op2与Roll轴分离。当Op1、Op2与Roll轴分离时，可能导致如图11所示的情形。
如图11所示，以目标对象发生原位旋转为例进行说明，目标对象的中心位于拍摄图像的中心c位置，如图11中虚线方框所示，目标对象的中心图像应当位于虚线方框的中心，然而，由于相机的光轴和Roll轴之间不重叠，则会导致目标对象的中心图像发生如图11中中央圆形的运动轨迹，使得无法满足所需拍摄效果。
在一个实施例中，负载包括拍摄装置。相应地，在第三跟踪模式下，目标对象在拍摄装置的拍摄画面的位置为画面中心位置。
在本实施例中，当跟踪的目标对象发生旋转运动时，则可以由偏航轴电机和俯仰轴电机共同对目标对象进行跟踪，以修正因光轴和Roll轴之间不重叠导致的拍摄效果异常。
在一个实施例中，云台还用于确定跟踪模式，跟踪模式为通过如下至少一种方式确定。
例如，响应于从用户交互界面中接收的模式选取指令，确定跟踪模式。例如，响应于针对预设功能键的模式选取操作，确定跟踪模式。
具体地，用户交互界面显示在位于把持组件的显示器上，把持组件用于支撑支架组件；或者，用户交互界面显示在位于负载的显示器上；或者，用户交互界面显示在与云台相连的终端设备的显示器上；或者，预设功能键位于把持组件上，把持组件用于支撑支架组件。
为了便于用户与云台之间进行交互，该云台上可以设置有输入部和/或输出部。
例如，输入部可以用于输入用户对手持云台的操作指令，该输入部可以包括跟踪模式输入部件和控制摇杆。
其中，控制摇杆可以控制转轴臂的运动，例如，通过拨动控制摇杆，实现手持云台的转轴臂在对应方向的转动。
例如，跟踪模式输入部件可以选择本申请实施例提到的跟踪模式。
具体地，该跟踪模式输入部件可以包括显示屏，用于显示交互界面，用户可以在交互界面中输入模式选取指令，以便处理器基于模式选取指令确定跟踪模式。
图12为本申请实施例提供的交互界面的示意图。
如图12所示，该交互界面中可以包括跟踪模式选择组件，用户在交互界面中点击某个组件后，则会触发对应的跟踪模式。此外，为了便于用户输入姿态信息，以及便于用户查看负载姿态信息，在当前交互界面或其它交互界面中还可以进一步显示输入组件以便于用户输入姿态，或显示当前负载姿态等。
此外，该跟踪模式输入部件也可以包括控制按键，该跟踪模式输出部件可以包括多个状态指示灯。其中，控制按键用于选择跟踪模式，一个指示灯可以对应一个跟踪模式，在对应的跟踪模式被选择时，指示灯亮起。
例如，在手持云台开启之后，用户短按一次控制按键，则最右侧的灯亮起，表明选择了最右侧对应的跟踪模式，再次短按一次控制按键，最右侧的指示灯灭，中间的指示灯亮起，如果再次操作，则中间的指示灯灭掉，最左侧的指示灯亮起。
应理解，本申请实施例的跟踪模式的选择方式并不限于上述方式。上述的指示灯的数量也不限于三个，可以多于三个，也可以少于三个。
还应理解，输入部除了可以包括跟踪模式输入部件以及控制摇杆之外， 还可以具有其他部件或者部分，例如，可以具有手持云台的开关等。
输入部中可以设置处理器，用于对输入的控制指令进行处理，或者收发信号等。当然，处理器也可以设置于把持组件中。
可选地，该处理器可以是中央处理单元(Central Processing Unit，简称CPU)，该处理器还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，简称DSP)、专用集成电路(application specific integrated circuit，简称ASIC)、现成可编程门阵列(Field-Programmable Gate Array，简称FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
以下对云台的跟踪算法进行示例性说明。
在一个实施例中，负载是拍摄装置。相应地，云台用于基于负载的姿态信息将拍摄装置在当前姿态下拍摄的图像转换至指定姿态下，从转换至指定姿态下的图像中识别出目标对象的位置。
云台的运算能力有限，尤其是对于手持云台，不便于通过云端图像识别等方式从图像中确定目标对象的图像在拍摄图像中的位置，当目标对象的图像在拍摄图像中发生旋转时，则云台不便于基于较少的运算从拍摄的图像中识别出目标对象的图像，这样造成了相关技术中不便于通过控制云台来追踪发生旋转的目标对象。在本申请实施例中，借助于IMU和/或用户输入姿态信息来确定拍摄装置的姿态信息，基于该姿态信息对拍摄的图像进行处理，如转换到水平拍摄的坐标系下，使得便于对处于各种姿态下拍摄装置拍摄的图像进行目标对象识别，以便于在各种姿态下对目标对象进行跟踪。
其中，从当前姿态下的图像中识别目标对象的位置所消耗的第一资源，大于或者等于从指定姿态下图像中识别目标对象的位置所消耗的第二资源。
目标对象识别过程可以采用多种相关图像识别技术，例如，通过特征提取、特征比对等方式从拍摄的图像中识别出目标对象的图像，进而确定目标对象的图像在拍摄的图像中的位置。例如，可以通过深度学习模型、轻量的识别模型等对拍摄的图像中目标对象进行识别。
例如，智能学习模块可以设置在云台或负载中，以对用于进行目标对象识别的识别模型进行训练，得到训练好的识别模型，进而利用识别模型对拍摄的图像进行识别，确定目标对象的图像的位置。
在一个实施例中，目标对象的位置为基于头部和肩部图像特征进行识别得到。这样有助于降低识别过程对计算资源的要求和消耗。
例如，云台还用于：基于目标对象在拍摄装置拍摄的图像中的位置变化，确定目标对象在指定坐标系(如云台坐标系)下针对各坐标轴的偏移量。基于已选定的跟踪模式对指定坐标系下指定坐标轴的偏移量进行陷零，并根据陷零的结果控制相应的电机转动，指定坐标轴是已选定的跟踪模式下禁止驱动支架部件随目标对象在负载的感测范围内的位置变化而发生运动的电机对应的坐标轴。其中，由于对图像进行了旋转处理，使得该确定偏移量的过程可以适用于竖拍模式和横拍模式。
可以通过如下方式确定目标对象在拍摄装置拍摄的图像中的位置变化。
例如，在相机拍摄多帧图像的过程中，对于每一帧图像，首先，从一帧图像中识别目标对象的目标图像，以确定目标图像在一帧图像中的位置信息。然后，确定不同帧图像中位置信息之间的位置改变量。
其中，确定不同帧图像中位置信息之间的位置改变量可以包括如下操作：基于第一目标图像在第一帧图像中的第一位置信息和第二目标图像在第二帧图像中的第二位置信息之间的位置改变量，其中，第一目标图像是目标对象在第一帧图像中的图像，第二目标图像是目标对象在第二帧图像中的图像。
例如，从一帧图像中识别目标对象的目标图像，以确定目标图像在一帧图像中的位置信息可以包括如下操作，首先，将第一坐标系下的一帧图像转换至第二坐标系下，得到一帧归一化图像。然后，从一帧归一化图像中识别目标对象的目标图像，以确定目标图像在一帧归一化图像中的位置信息。
在一个实施例中，云台具体用于：首先，基于构图目标的第一坐标和当前帧归一化的第二坐标确定拍摄装置的平面偏差。然后，将平面偏差转换为目标对象在指定坐标系下针对各坐标轴的偏移量。
以下以单偏航轴跟踪为例对跟踪算法进行示例性说明。
图13为本申请实施例提供的目标对象跟踪的数据流图。
关于图像旋转变换，IMU测量得到第三方相机的姿态信息，其测量欧拉角为pitch、roll、yaw，记录图像某一像素点横纵坐标为(x n，y n)，则转换后坐标
Figure PCTCN2020131446-appb-000001
可以如式(1)所示：
Figure PCTCN2020131446-appb-000002
需要说明的是，由于用户可以根据自身需求将第三方相机横放或竖放在云台上，为了便于处理器确定正确的图像旋转角度，可以由用户在如图12所示的交互界面中输入用户输入姿态，以便于结合IMU测量的姿态和用户输入姿态确定负载的实际姿态。这样就可以基于上述式(1)将图像转换至便于进行目标对象识别的姿态下，确定目标对象的图像在拍摄图像中的位置。需要说明的是，式(1)中仅针对roll做了变换，是考虑到在进行目标对象识别时，目标对象针对偏航轴的改变不会影响诸如头部特征和肩部特征的识别，因此可以无需针对yaw进行转换。同样地，用户在原地转身等同样不会影响诸如头部特征和肩部特征的识别。但是，在本申请的实施例不排除可以针对yaw和pitch对拍摄的图像进行转换的实施方式。
构图目标为(tgt x，tgt y)，当前帧归一化坐标值为(x，y)，相机平面偏差可以如式(2)所示。
e x＝tgt x-x
e y＝tgt y-y    式(2)
需要将相机平面偏差转为NED(北、东、地坐标系)坐标偏差，记实际fov为(FOV x，FOV y)，相机坐标系下偏差记做式(3)所示。
Figure PCTCN2020131446-appb-000003
其中，E x＝0；
Figure PCTCN2020131446-appb-000004
NED(北、东、地坐标系)坐标偏差如式(4)所示。
Figure PCTCN2020131446-appb-000005
用户根据需要可以设置需要跟随的自由度(俯仰E x、横滚E y和偏航E z)，对于不需要跟随的自由度，NED(北、东、地坐标系)坐标系下的偏差为零。
例如：只需跟随偏航E z，则实际输入云台的
Figure PCTCN2020131446-appb-000006
如式(5)所示。
Figure PCTCN2020131446-appb-000007
以下以目标对象的多样化运动的场景为例，对本申请的实施例的跟踪效果进行示意性说明。
图14为本申请实施例提供的目标对象的多样化运动的意图。图15为针对图14的目标对象的拍摄图像的示意图。
如图14所示，骑手在特定形状的场地中进行特技表演，骑手不但在yaw和pitch自由度上发生位移，还在roll自由度上发生了姿态改变。
如图15所示，可以为预览图像或拍摄图像的展示(或局部展示)，图15的上边图像所示，是在yaw和pitch自由度上对骑手进行跟拍的效果示意图，骑手在roll自由度上并没有跟踪骑手。如图15的下边图像所示，是在yaw、pitch和roll三个自由度上跟拍骑手，以满足用户对特殊素材的需求。
图16为本申请实施例提供的跟踪目标对象的方法的流程图。跟踪目标对象的方法用于云台，云台包括支架组件和至少两个电机，支架组件包括至少两个能够相对活动的支架部件，且用于支撑负载，至少两个电机分别用于驱动对应的所述支架部件运动，以调整所述负载的姿态。
如图16所示，该跟踪目标对象的方法可以包括操作S1602～操作S1606。
在操作S1602，获取模式选取指令。
在本实施例中，模式选取指令可以是基于用户在云台上输入的用户操作确定的。例如，云台上设置有按键、拨杆等部件，用户可以通过操作这些部件输入模式选取指令。又例如，云台上可以包括显示屏，用户可以通过显示屏上显示的交互组件(如虚拟的按键、摇杆等)来输入模式选取指令。
例如，用户操作所针对的对象可以是与可移动平台通信连接的云台。例如，用户在云台上输入以下至少一种信息：选取信息、姿态信息、指定操作(如拍照)、对象以及该指定操作的参数(如焦距、光圈、曝光时长)等。其中，云台可以是一体式的，如遥控器上设置有处理器、存储器、显示屏等。云台可以是分体式的，如云台可以和其它终端设备共同构成控制终端，如云台和智能手机互连后共同构成控制终端。其中，智能手机上可以安装有应用(APP)，该APP上可以输入操作指令、设置操作参数等。
进一步地，指定状态指令还可以基于手势识别、姿势识别、体感或语音识别等方式确定并输入。例如，用户可以通过倾斜云台，以控制可移动平台的位置、姿态、方向、或者其它方面。云台的倾斜可以由一个或者多个惯性传感器所侦测，并产生对应的运动指令。再如，用户可以利用触控屏调整负载的操作参数(如变焦)、负载的姿态(通过承载体)，或者可移动平台上的 任何物体的其它方面。
在操作S1604，响应于模式选取指令，从至少两个跟踪模式中确定当前跟踪模式，其中，至少两个跟踪模式下各自能够驱动支架部件随目标对象在负载的感测范围内的位置变化而发生运动的电机的数量不同，负载设置在支架部件上。
在操作S1606，利用当前跟踪模式，控制当前跟踪模式对应的电机，以在指定维度的方向上实现负载对目标对象的跟踪。
其中，跟踪模式、电机、负载、位置变化等可以参考如上相关内容，在此不再赘述。
在一个实施例中，跟踪模式包括以下至少两个。
第一跟踪模式，第一跟踪模式下能够驱动支架部件随目标对象在负载的感测范围内的位置变化而发生运动的电机的数量是一个。
第二跟踪模式，第二跟踪模式下能够驱动支架部件随目标对象在负载的感测范围内的位置变化而发生运动的电机的数量是两个。
第三跟踪模式，第三跟踪模式下能够驱动支架部件随目标对象在负载的感测范围内的位置变化而发生运动的电机的数量是三个。
具体内容参考前面的实施例的相同部分，此处不再做赘述。
在一个实施例中，第一跟踪模式下能够驱动支架部件随目标对象的运动而发生运动的电机包括偏航轴电机。
第二跟踪模式下能够驱动支架部件随目标对象的运动而发生运动的电机包括偏航轴电机和俯仰轴电机。
第三跟踪模式下能够驱动支架部件随目标对象的运动而发生运动的电机包括偏航轴电机、俯仰轴电机和横滚轴电机。
例如，本申请实施例在单偏航轴自由度下进行智能跟踪时，能够为用户提供垂直方向上较稳定的画面。
在一个实施例中，负载包括拍摄装置。在第三跟踪模式下，目标对象在拍摄装置的拍摄画面的位置为画面中心位置。
具体内容参考前面的实施例的相同部分，此处不再做赘述。
在一个实施例中，拍摄画面的长边相对于地面是相互平行或相互垂直的，或者拍摄画面的长边相对于水平面是相互平行或相互垂直的。
在一个实施例中，上述方法还包括：获取负载的姿态信息，以便至少基于负载的姿态信息在负载的感测范围内确定目标对象。
在一个实施例中，获取负载的姿态信息包括：基于惯性测量单元检测负载的姿态信息。
在一个实施例中，获取负载的姿态信息包括：基于用户输入姿态信息和惯性测量单元获取的负载的姿态信息，确定负载的姿态信息。惯性测量单元设于支架组件或所述负载上。
在一个实施例中，负载是拍摄装置。相应地，至少基于负载的姿态信息在负载的感测范围内确定目标对象包括：基于负载的姿态信息将拍摄装置在当前姿态下拍摄的图像转换至指定姿态下，从转换至指定姿态下的图像中识别出目标对象的位置。
在一个实施例中，从当前姿态下的图像中识别目标对象的位置所消耗的第一资源，大于或者等于从指定姿态下图像中识别目标对象的位置所消耗的第二资源。
具体内容参考前面的实施例的相同部分，此处不再做赘述。
本申请实施例可以实现负载在任意角度下的智能跟踪，能够为用户提供更自由的构图方式。
在一个实施例中，目标对象的位置为基于头部和肩部图像特征进行识别得到。
在一个实施例中，利用当前跟踪模式，控制当前跟踪模式对应的电机，以在指定维度的方向上实现负载对目标对象的跟踪可以包括如下操作。
首先，基于目标对象在拍摄装置拍摄的图像中的位置变化，确定目标对象在指定坐标系下针对各坐标轴的偏移量。
然后，基于已选定的跟踪模式对指定坐标系下指定坐标轴的偏移量进行陷零，并根据陷零的结果控制相应的电机转动，指定坐标轴是已选定的跟踪模式下禁止驱动支架部件随目标对象在负载的感测范围内的位置变化而发生运动的电机对应的坐标轴。
在一个实施例中，基于目标对象在拍摄装置拍摄的图像中的位置变化，确定目标对象在指定坐标系下针对各坐标轴的偏移量可以包括如下操作。
首先，基于构图目标的第一坐标和当前帧归一化的第二坐标确定拍摄装 置的平面偏差。
然后，将平面偏差转换为目标对象在指定坐标系下针对各坐标轴的偏移量。
在一个实施例中，获取模式选取指令可以包括如下方式。
例如，响应于从用户交互界面中接收的模式选取指令，确定跟踪模式。
又例如，响应于针对预设功能键的模式选取操作，确定跟踪模式。
本申请实施例中可以将多种跟踪模式与智能跟踪算法进行结合，例如，可以给用户提供多种跟踪模式供用户自行选择，使得用户可以根据自身需求和无人机操作水平选取适合自身的跟踪模式，以在满足跟拍便捷度的同时，可以采用多种运镜手法，提升拍摄体验。可以为用户提供丰富的运镜手法应用。
以下以手持云台为例，对上述各操作的执行主体进行示例性说明。例如，上述各操作的执行主体都可以是手持云台，具体可以由手持云台的输入部、把持部件、设置在把持部件中的处理器、电机等实现对应的功能。
以下以可移动平台机载的云台为例，对上述各操作的执行主体进行示例性说明。
获取模式选取指令的操作，可以是基于由用户在可移动平台的控制终端或可移动平台上输入的用户操作确定的。
从至少两个跟踪模式中确定当前跟踪模式的操作，可以是由可移动平台的控制终端、云台或可移动平台(如处理器)确定的。
利用当前跟踪模式，控制当前跟踪模式对应的电机的操作，可以是由云台执行的。
获取负载的姿态信息的操作，可以是由可移动平台的控制终端(如用户输入姿态)、云台(如IMU检测的姿态)或可移动平台(如处理器对多个姿态进行融合)确定的。
至少基于负载的姿态信息在负载的感测范围内确定目标对象的操作，可以是由可移动平台的控制终端、云台、负载或可移动平台确定的。
图像处理和目标对象识别的操作，可以是由可移动平台的控制终端、云台、负载或可移动平台确定的。
需要说明的是，上述各操作的执行主体仅为示例性说明，不能理解为对本申请的限定，可以由可移动平台、控制终端、拍摄装置、云台其中的一个独立完成，或其中的几个配合完成。例如，对于可移动平台是陆地机器人的情形下，可以在陆地机器人上设置人机交互模块(如包括用于显示人机交互界面的显示器等)，用户可以直接在可移动平台展示的交互界面上获取用户操作，以生成用户指令，确定目标对象的图像等。其中，独立完成包括主动或被动地、直接或间接地从其它设备获取相应数据以执行相应操作。
本申请实施例提供的目标对象的追踪方法，通过手持云台多种云台跟随模式与智能跟随的结合应用，通过算法处理得出多种跟随模式下的控制方式，使得用户可以只通过单自由度实现智能跟随控制的方法。此外，本申请实施例可以任意角度进行自动跟踪，包括FPV模式和竖拍模式。
需要说明的是，在实际应用中，上述方法还可以应用于除拍摄装置等以外的负载，以追踪目标对象。其中，负载包括但不限于声波检测装置、红外检测装置等。
图17为本申请实施例提供的跟踪目标对象的装置的方框图。
如图17所示，该跟踪目标对象的装置1700可以包括一个或多个处理器1710，该一个或多个处理器1710可以集成在一个处理单元中，也可以分别设置在多个处理单元中。计算机可读存储介质1720，用于存储一个或多个计算机程序1721，计算机程序在被处理器执行时，实现如上的跟踪方法，例如，获取模式选取指令；响应于模式选取指令，从至少两个跟踪模式中确定当前跟踪模式，其中，至少两个跟踪模式下各自能够驱动支架部件随目标对象在负载的感测范围内的位置变化而发生运动的电机的数量不同，负载设置在支架部件上；利用当前跟踪模式，控制当前跟踪模式对应的电机，以在指定维度的方向上实现负载对目标对象的跟踪。
其中，该跟踪目标对象的装置1700可以被设置在一个执行主体中或分别设置在多个执行主体中。例如，跟踪目标对象的装置1700可以设置在云台中。例如，其部分设置在云台中，部分设置在可以与云台连接的控制终端中，如用于显示交互界面的显示屏。
例如，处理单元可以包括现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者一个或者多个ARM处理器。处理单元可以与非易失性计 算机可读存储介质1720连接。与非易失性计算机可读存储介质1720可以存储由处理单元所执行的逻辑、代码及/或者计算机指令，用于执行一个或者多个步骤。非易失性计算机可读存储介质1720可以包括一个或者多个存储单元(可去除的介质或者外部存储器，如SD卡或者RAM)。在某些实施例中，传感器感测的数据可以直接传送并存储到非易失性计算机可读存储介质1720的存储单元中。非易失性计算机可读存储介质1720的存储单元可以存储由处理单元所执行的逻辑、代码及/或者计算机指令，以执行本案描述的各种方法的各个实施例。例如，处理单元可以用于执行指令，以导致处理单元的一个或者多个处理器执行上述描述的追踪功能。存储单元可以存储感测模块感测数据，该数据感测由处理单元所处理。在某些实施例中，非易失性计算机可读存储介质1720的存储单元可以存储处理单元产生的处理结果。
在某些实施例中，处理单元可以与控制模块连接，用以控制可移动平台的状态。例如，控制模块可以用于控制可移动平台的动力机构，以调整可移动平台相对于六个自由度的空间方位、速度及/或加速度。可选地或者相结合的，控制模块可以控制承载体，负载或者感测模块中的一个或者多个。
处理单元还可以与通讯模块连接，用以与一个或者多个外围设备(如终端、显示设备、或者其它远程控制设备)传送及/或者接收数据。这里可以利用任何合适的通讯方法，如有线通讯或者无线通讯。例如，通讯模块可以利用到一个或者多个局域网、广域网、红外线、无线电、Wi-Fi、点对点(P2P)网络、电信网络、云网络等。可选地，可以用到中继站，如信号塔、卫星、或者移动基站等。
上述各个部件之间可以是相互适配的。例如，一个或者多个部件位于可移动平台、承载体、负载、终端、感测系统、或者与前述各设备通讯的额外的外部设备上。在某些实施例中，处理单元及/或非易失性计算机可读介质中的一个或者多个可以位于不同的位置，如在可移动平台、承载体、负载、终端、感测系统、或者与前述各设备通讯的额外的外部设备以及前述的各种结合上。
此外，与可移动平台相适配的控制终端可以包括输入模块、处理单元、存储器、显示模块、以及通讯模块，所有这样的部件都是通过总线或者相似的网络相连接。
输入模块包括一个或者多个输入机制，以获取用户通过操作该输入模块产生的输入。输入机制包括一个或者多个操纵杆、开关、旋钮、滑动开关、按钮、拨号盘、触摸屏、小键盘、键盘、鼠标、声音控制、手势控制、惯性模块等。输入模块可以用于获取用户的输入，该输入用于控制可移动平台、承载体、负载、或者其中部件的任何方面。任何方面包括姿态、位置、方向、飞行、追踪等。例如，输入机制可以是用户手动设置一个或者多个位置，每个位置对应一个预设输入，以控制可移动平台。
在某些实施例中，输入机制可以由用户操作，以输入控制指令，控制可移动平台的运动。例如，用户可以利用旋钮、开关或者相似的输入机制，输入可移动平台的运动模式，如自动飞行、自动驾驶或者根据预设运动路径运动。又如，用户可以通过用某种方法倾斜控制终端，以控制可移动平台的位置、姿态、方向、或者其它方面。控制终端的倾斜可以由一个或者多个惯性传感器所侦测，并产生对应的运动指令。再如，用户可以利用上述输入机制调整负载的操作参数(如变焦)、负载的姿态(通过承载体)，或者可移动平台上的任何物体的其它方面。
在某些实施例中，输入机制可以由用户操作，以输入前述描述目标物信息。例如，用户可以利用旋钮、开关或者相似的输入机制，选择合适的追踪模式，如人工追踪模式或者自动追踪模式。用户也可以利用该输入机制选择所要追踪的特定目标物、执行的目标物类型信息、或者其它相似的信息。在各种实施例中，输入模块可以由不止一个设备所执行。例如，输入模块可以由带有操纵杆的标准远程控制器所执行。带有操纵杆的标准远程控制器连接到运行适合应用程序(“app”)的移动设备(如智能手机)中，以产生可移动平台的控制指令。app可以用于获取用户的输入。
处理单元可以与存储器连接。存储器包括易失性或者非易失性存储介质，用于存储数据，及/或处理单元可执行的逻辑、代码、及/或程序指令，用于执行一个或者多个规则或者功能。存储器可以包括一个或者多个存储单元(可去除的介质或者外部存储器，如SD卡或者RAM)。在某些实施例中，输入模块的数据可以直接传送并存储在存储器的存储单元中。存储器的存储单元可以存储由处理单元所执行的逻辑、代码及/或者计算机指令，以执行本案描述的各种方法的各个实施例。例如，处理单元可以用于执行指令，以导致处 理单元的一个或者多个处理器处理及显示从可移动平台获取的感应数据(如影像)，基于用户输入产生的控制指令，包括运动指令及目标物信息，并导致通讯模块传送及/或者接收数据等。存储单元可以存储感测数据或者从外部设备(如可移动平台)接收的其它数据。在某些实施例中，存储器的存储单元可以存储处理单元生成的处理结果。
在某些实施例中，显示模块可以用于显示如上对云台及/或负载关于位置、平移速度、平移加速度、方向、角速度、角加速度、或者其结合等的信息。显示模块可以用于获取可移动平台及/或者负载发送的信息，如感测数据(相机或者其它影像捕获设备记录的影像)、所描述的追踪数据、控制反馈数据等。在某些实施例中，显示模块可以与输入模块由相同的设备所执行。在其它实施例中，显示模块与输入模块可以由不相同的设备所执行。
通讯模块可以用于从一个或者多个远程设备(如可移动平台、承载体、基站等)传送及/或者接收数据。例如，通讯模块可以传送控制信号(如运动信号、目标物信息、追踪控制指令)给外围系统或者设备，如上述中对云台及/或负载。通讯模块可以包括传送器及接收器，分别用于从远程设备接收数据以及传送数据给远程设备。在某些实施例中，通讯模块可以包括收发器，其结合了传送器与接收器的功能。在某些实施例中，传送器与接收器之间以及与处理单元之间可以彼此通讯。通讯可以利用任何合适的通讯手段，如有线通讯或者无线通讯。
可移动平台在运动过程中捕获的影像可以从可移动平台或者影像设备传回给控制终端或者其它适合的设备，以显示、播放、存储、编辑或者其它目的。这样的传送可以是当影像设备捕获影像时，实时的或者将近实时的发生。可选地，影像的捕获及传送之间可以有延迟。在某些实施例中，影像可以存储在可移动平台的存储器中，而不用传送到任何其它地方。用户可以实时看到这些影像，如果需要，调整目标物信息或者调整可移动平台或者其部件的其它方面。调整的目标物信息可以提供给可移动平台，重复的过程可能继续直到获得可想要的影像。在某些实施例中，影像可以从可移动平台、影像设备及/或控制终端传送给远程服务器。例如，影像可以在一些社交网络平台，如微信朋友圈或者微博上以进行分享。
该跟踪目标对象的装置1700可以用于执行如上所述的一个或多个操作。 在此不再一一列举。
以下对可移动平台进行示例性说明。
图18为本申请实施例提供的可移动平台的结构示意图。
如图18所示，该可移动平台可以是无人机180，该无人机180可以包括多个动力系统181以及脚架。云台可以设置在无人机180上。
在一实施例中，无人机180的多个动力系统181与多个机臂一一对应。每个动力系统181可以包括电机组件和与电机组件连接的桨叶。每个动力系统181可以设置于与其对应的机臂上，由该对应的机臂来支撑该动力系统181。
此外，无人机180还可以包括脚架。该脚架可以位于云台下方，且与云台连接。在无人机180降落时，可以用于无人机180着陆。
图19示意性示出了本申请另一实施例的可移动平台的示意图。
如图19所示，该可移动平台是手持云台190，该手持云台190可以包括如上述的云台的结构。手持云台190可以包括：云台及支撑云台的手柄，该手柄为用户进行握持的部分，可以包括控制按钮，以便于对云台进行操作。手持云台190与支架中的功能部件(如相机)通信连接，以获取相机所拍摄的图像信息。
此外，该手持云台190还可以与终端设备191(如手机)等相连，以向手机发送图像等信息。
以上为本申请的最优实施例，需要说明的，该最优的实施例仅用于理解本申请，并不用于限制本申请的保护范围。并且，最优实施例中的特征，在无特别注明的情况下，均同时适用于方法实施例和装置实施例，在相同或不同实施例中出现的技术特征在不相互冲突的情况下可以组合使用。
最后应说明的是：以上实施方式仅用以说明本申请的技术方案，而非对其进行限制；尽管参照前述实施方式对本申请已经进行了详细的说明，但本领域的普通技术人员应当理解：其依然可以对前述实施方式所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请实施方式技术方案的范围。

Claims (31)

  1. 一种云台，其特征在于，包括：
    支架组件，包括至少两个能够相对活动的支架部件，所述支架组件用于支撑负载；
    至少两个电机，分别用于驱动对应的所述支架部件运动，以调整所述负载的姿态；
    其中，所述云台具有至少两个跟踪模式，至少两个所述跟踪模式下各自能够驱动所述支架部件随目标对象在所述负载的感测范围内的位置变化而发生运动的电机的数量不同，以在不同维度的方向上实现对所述目标对象的跟踪。
  2. 根据权利要求1所述的云台，其特征在于，所述跟踪模式包括以下至少两个：
    第一跟踪模式，所述第一跟踪模式下能够驱动所述支架部件随目标对象在所述负载的感测范围内的位置变化而发生运动的电机的数量是一个；
    第二跟踪模式，所述第二跟踪模式下能够驱动所述支架部件随目标对象在所述负载的感测范围内的位置变化而发生运动的电机的数量是两个；
    第三跟踪模式，所述第三跟踪模式下能够驱动所述支架部件随目标对象在所述负载的感测范围内的位置变化而发生运动的电机的数量是三个。
  3. 根据权利要求2所述的云台，其特征在于，所述第一跟踪模式下能够驱动所述支架部件随目标对象的运动而发生运动的电机包括偏航轴电机；
    所述第二跟踪模式下能够驱动所述支架部件随目标对象的运动而发生运动的电机包括偏航轴电机和俯仰轴电机；
    所述第三跟踪模式下能够驱动所述支架部件随目标对象的运动而发生运动的电机包括偏航轴电机、俯仰轴电机和横滚轴电机。
  4. 根据权利要求3所述的云台，其特征在于，所述负载包括拍摄装置；
    在所述第三跟踪模式下，所述目标对象在所述拍摄装置的拍摄画面的位置为画面中心位置。
  5. 根据权利要求1所述的云台，其特征在于，还包括：
    负载固定机构，用于固定所述负载，所述负载姿态可调地固定在所述负载固定机构上。
  6. 根据权利要求5所述的云台，其特征在于，所述负载的第一固定面和所述负载固定机构的第二固定面之间相互平行或相互垂直。
  7. 根据权利要求5所述的云台，其特征在于，还包括：
    惯性测量单元，设置在所述负载固定机构上，用于测量所述负载的姿态信息。
  8. 根据权利要求1所述的云台，其特征在于，所述负载是拍摄装置；
    所述云台用于基于所述负载的姿态信息将所述拍摄装置在当前姿态下拍摄的图像转换至指定姿态下，从转换至所述指定姿态下的所述图像中识别出所述目标对象的位置。
  9. 根据权利要求8所述的云台，其特征在于，从当前姿态下的图像中识别所述目标对象的位置所消耗的第一资源，大于或者等于从指定姿态下图像中识别所述目标对象的位置所消耗的第二资源。
  10. 根据权利要求8所述的云台，其特征在于，所述目标对象的位置为基于头部和肩部图像特征进行识别得到。
  11. 根据权利要求8所述的云台，其特征在于，所述云台还用于：
    基于目标对象在所述拍摄装置拍摄的图像中的位置变化，确定目标对象在指定坐标系下针对各坐标轴的偏移量；
    基于已选定的跟踪模式对所述指定坐标系下指定坐标轴的偏移量进行陷零，并根据陷零的结果控制相应的电机转动，所述指定坐标轴是已选定的跟踪模式下禁止驱动所述支架部件随目标对象在所述负载的感测范围内的位置变化而发生运动的电机对应的坐标轴。
  12. 根据权利要求11所述的云台，其特征在于，所述云台具体用于：
    基于构图目标的第一坐标和当前帧归一化的第二坐标确定所述拍摄装置的平面偏差；
    将所述平面偏差转换为所述目标对象在所述指定坐标系下针对各坐标轴的偏移量。
  13. 根据权利要求8所述的云台，其特征在于，所述云台还用于确定跟踪模式，所述跟踪模式为通过如下至少一种方式确定：
    响应于从用户交互界面中接收的模式选取指令，确定所述跟踪模式；
    响应于针对预设功能键的模式选取操作，确定所述跟踪模式。
  14. 根据权利要求13所述的云台，其特征在于，所述用户交互界面显示在位于把持组件的显示器上，所述把持组件用于支撑所述支架组件；或者
    所述用户交互界面显示在位于负载的显示器上；或者
    所述用户交互界面显示在与所述云台相连的终端设备的显示器上；或者
    所述预设功能键位于把持组件上，所述把持组件用于支撑所述支架组件。
  15. 根据权利要求1所述的云台，其特征在于，还包括：
    惯性测量单元，设置在所述负载上，用于测量所述负载的姿态信息。
  16. 根据权利要求1所述的云台，其特征在于，所述支架组件用于固定在具有动力系统的可移动平台上；或者
    所述云台还包括：把持组件，所述把持组件用于支撑所述支架组件。
  17. 一种跟踪目标对象的方法，其特征在于，用于云台，所述云台包括支架组件和至少两个电机，所述支架组件包括至少两个能够相对活动的支架部件，且用于支撑负载，所述至少两个电机分别用于驱动对应的所述支架部件运动，以调整所述负载的姿态；所述方法包括：
    获取模式选取指令；
    响应于所述模式选取指令，从至少两个跟踪模式中确定当前跟踪模式，其中，至少两个所述跟踪模式下各自能够驱动所述支架部件随目标对象在负载的感测范围内的位置变化而发生运动的电机的数量不同；
    利用所述当前跟踪模式，控制所述当前跟踪模式对应的电机，以在指定维度的方向上实现所述负载对所述目标对象的跟踪。
  18. 根据权利要求17所述的方法，其特征在于，所述跟踪模式包括以下至少两个：
    第一跟踪模式，所述第一跟踪模式下能够驱动所述支架部件随目标对象在所述负载的感测范围内的位置变化而发生运动的电机的数量是一个；
    第二跟踪模式，所述第二跟踪模式下能够驱动所述支架部件随目标对象在所述负载的感测范围内的位置变化而发生运动的电机的数量是两个；
    第三跟踪模式，所述第三跟踪模式下能够驱动所述支架部件随目标对象在所述负载的感测范围内的位置变化而发生运动的电机的数量是三个。
  19. 根据权利要求18所述的方法，其特征在于：
    所述第一跟踪模式下能够驱动所述支架部件随目标对象的运动而发生运动的电机包括偏航轴电机；
    所述第二跟踪模式下能够驱动所述支架部件随目标对象的运动而发生运动的电机包括偏航轴电机和俯仰轴电机；
    所述第三跟踪模式下能够驱动所述支架部件随目标对象的运动而发生运动的电机包括偏航轴电机、俯仰轴电机和横滚轴电机。
  20. 根据权利要求19所述的方法，其特征在于，所述负载包括拍摄装置；
    在所述第三跟踪模式下，所述目标对象在所述拍摄装置的拍摄画面的位置为画面中心位置。
  21. 根据权利要求20所述的方法，其特征在于，所述拍摄画面的长边相对于地面是相互平行或相互垂直的，或者所述拍摄画面的长边相对于水平面是相互平行或相互垂直的。
  22. 根据权利要求17所述的方法，其特征在于，还包括：
    获取所述负载的姿态信息，以便至少基于所述负载的姿态信息在所述负载的感测范围内确定所述目标对象。
  23. 根据权利要求22所述的方法，其特征在于，所述获取所述负载的姿态信息包括：
    基于惯性测量单元检测所述负载的姿态信息。
  24. 根据权利要求22所述的方法，其特征在于，所述获取所述负载的姿态信息包括：
    基于用户输入姿态信息和惯性测量单元获取的负载的姿态信息，确定所述负载的姿态信息，所述惯性测量单元设于所述支架组件或所述负载上。
  25. 根据权利要求22所述的方法，其特征在于，所述负载是拍摄装置；
    所述至少基于所述负载的姿态信息在所述负载的感测范围内确定所述目标对象包括：
    基于所述负载的姿态信息将所述拍摄装置在当前姿态下拍摄的图像转换至指定姿态下，从转换至所述指定姿态下的所述图像中识别出所述目标对象的位置。
  26. 根据权利要求25所述的方法，其特征在于，从当前姿态下的图像中识别所述目标对象的位置所消耗的第一资源，大于或者等于从指定姿态下图像中识别所述目标对象的位置所消耗的第二资源。
  27. 根据权利要求25所述的方法，其特征在于，所述目标对象的位置为基于头部和肩部图像特征进行识别得到。
  28. 根据权利要求25所述的方法，其特征在于，所述利用所述当前跟踪模式，控制所述当前跟踪模式对应的电机，以在指定维度的方向上实现所述负载对所述目标对象的跟踪包括：
    基于目标对象在所述拍摄装置拍摄的图像中的位置变化，确定目标对象在指定坐标系下针对各坐标轴的偏移量；
    基于已选定的跟踪模式对所述指定坐标系下指定坐标轴的偏移量进行陷零，并根据陷零的结果控制相应的电机转动，所述指定坐标轴是已选定的跟踪模式下禁止驱动所述支架部件随目标对象在所述负载的感测范围内的位置变化而发生运动的电机对应的坐标轴。
  29. 根据权利要求28所述的方法，其特征在于，所述基于目标对象在所述拍摄装置拍摄的图像中的位置变化，确定目标对象在指定坐标系下针对各坐标轴的偏移量包括：
    基于构图目标的第一坐标和当前帧归一化的第二坐标确定所述拍摄装置的平面偏差；
    将所述平面偏差转换为所述目标对象在所述指定坐标系下针对各坐标轴的偏移量。
  30. 根据权利要求17所述的方法，其特征在于，所述获取模式选取指令包括：
    响应于从用户交互界面中接收的模式选取指令，确定所述跟踪模式；或
    响应于针对预设功能键的模式选取操作，确定所述跟踪模式。
  31. 一种计算机可读存储介质，其特征在于，其存储有可执行指令，所述可执行指令在由一个或多个处理器执行时，可以使所述一个或多个处理器执行如权利要求17至30中任一项权利要求所述的方法。
PCT/CN2020/131446 2020-11-25 2020-11-25 跟踪目标对象的方法和云台 WO2022109860A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202080074276.8A CN114641642A (zh) 2020-11-25 2020-11-25 跟踪目标对象的方法和云台
PCT/CN2020/131446 WO2022109860A1 (zh) 2020-11-25 2020-11-25 跟踪目标对象的方法和云台

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/131446 WO2022109860A1 (zh) 2020-11-25 2020-11-25 跟踪目标对象的方法和云台

Publications (1)

Publication Number Publication Date
WO2022109860A1 true WO2022109860A1 (zh) 2022-06-02

Family

ID=81755005

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/131446 WO2022109860A1 (zh) 2020-11-25 2020-11-25 跟踪目标对象的方法和云台

Country Status (2)

Country Link
CN (1) CN114641642A (zh)
WO (1) WO2022109860A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115686053A (zh) * 2023-01-04 2023-02-03 北京航科京工科技有限责任公司 负载姿态调节装置及方法

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115599128A (zh) * 2022-11-02 2023-01-13 泉州装备制造研究所（Cn） 基于跟随机器人跟随模式动态调整方法、装置及可读介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN203703540U (zh) * 2014-02-27 2014-07-09 黑龙江科大科技开发有限公司 一种高精度快速响应三轴云台
CN105487552A (zh) * 2016-01-07 2016-04-13 深圳一电航空技术有限公司 无人机跟踪拍摄的方法及装置
US20170039671A1 (en) * 2015-08-07 2017-02-09 Seoul National University R&Db Foundation Robotic self-filming system
CN106814754A (zh) * 2017-03-28 2017-06-09 东莞松山湖国际机器人研究院有限公司 一种三轴云台、拍摄机器人及控制方法
CN206694770U (zh) * 2017-03-28 2017-12-01 东莞松山湖国际机器人研究院有限公司 一种三轴云台和拍摄机器人
WO2019183845A1 (zh) * 2018-03-28 2019-10-03 深圳市大疆创新科技有限公司 云台的控制方法、装置、系统、计算机存储介质及无人机

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN203703540U (zh) * 2014-02-27 2014-07-09 黑龙江科大科技开发有限公司 一种高精度快速响应三轴云台
US20170039671A1 (en) * 2015-08-07 2017-02-09 Seoul National University R&Db Foundation Robotic self-filming system
CN105487552A (zh) * 2016-01-07 2016-04-13 深圳一电航空技术有限公司 无人机跟踪拍摄的方法及装置
CN106814754A (zh) * 2017-03-28 2017-06-09 东莞松山湖国际机器人研究院有限公司 一种三轴云台、拍摄机器人及控制方法
CN206694770U (zh) * 2017-03-28 2017-12-01 东莞松山湖国际机器人研究院有限公司 一种三轴云台和拍摄机器人
WO2019183845A1 (zh) * 2018-03-28 2019-10-03 深圳市大疆创新科技有限公司 云台的控制方法、装置、系统、计算机存储介质及无人机

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115686053A (zh) * 2023-01-04 2023-02-03 北京航科京工科技有限责任公司 负载姿态调节装置及方法

Also Published As

Publication number Publication date
CN114641642A (zh) 2022-06-17

Similar Documents

Publication Publication Date Title
US10587790B2 (en) Control method for photographing using unmanned aerial vehicle, photographing method using unmanned aerial vehicle, mobile terminal, and unmanned aerial vehicle
US11560920B2 (en) Gimbal for image capturing
CN109952755B (zh) 飞行路径生成方法、飞行路径生成系统、飞行体以及记录介质
US20190291864A1 (en) Transformable apparatus
WO2019227441A1 (zh) 可移动平台的拍摄控制方法和设备
US11076082B2 (en) Systems and methods for digital video stabilization
US20210112194A1 (en) Method and device for taking group photo
CN103426282A (zh) 遥控方法及终端
WO2022151473A1 (zh) 拍摄控制方法、拍摄控制装置及云台组件
WO2021098453A1 (zh) 目标跟踪方法及无人飞行器
CN113286089B (zh) 一种控制移动设备拍摄的方法、控制装置及控制系统
WO2022109860A1 (zh) 跟踪目标对象的方法和云台
WO2021217371A1 (zh) 可移动平台的控制方法和装置
WO2020233682A1 (zh) 一种自主环绕拍摄方法、装置以及无人机
CN110291777B (zh) 图像采集方法、设备及机器可读存储介质
WO2021135824A1 (zh) 图像曝光方法及装置、无人机
WO2019183789A1 (zh) 无人机的控制方法、装置和无人机
WO2022061934A1 (zh) 图像处理方法、装置、系统、平台及计算机可读存储介质
WO2020062024A1 (zh) 基于无人机的测距方法、装置及无人机
WO2022056683A1 (zh) 视场确定方法、视场确定装置、视场确定系统和介质
WO2022188151A1 (zh) 影像拍摄方法、控制装置、可移动平台和计算机存储介质
JP7081198B2 (ja) 撮影システム及び撮影制御装置
CN114071003B (zh) 一种基于光通信装置的拍摄方法和系统
WO2022205091A1 (zh) 云台控制方法、云台和移动平台
WO2021059684A1 (ja) 情報処理システム、情報処理方法、情報処理プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20962751

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20962751

Country of ref document: EP

Kind code of ref document: A1