CN114641642A

CN114641642A - Method and cradle head for tracking target object

Info

Publication number: CN114641642A
Application number: CN202080074276.8A
Authority: CN
Inventors: 楼致远; 杨天豪; 常贤茂; 苏铁
Original assignee: SZ DJI Technology Co Ltd
Current assignee: SZ DJI Technology Co Ltd
Priority date: 2020-11-25
Filing date: 2020-11-25
Publication date: 2022-06-17
Also published as: WO2022109860A1

Abstract

A method of tracking a target object and a head, the head comprising: a bracket assembly including at least two relatively movable bracket parts, the bracket assembly for supporting a load; the at least two motors are respectively used for driving the corresponding bracket parts to move so as to adjust the posture of the load; the cradle head is provided with at least two tracking modes, and the number of motors which can drive the bracket component to move along with the position change of the target object in the sensing range of the load in the at least two tracking modes is different, so that the target object can be tracked in different dimensions.

Description

Method and cradle head for tracking target object

Technical Field

The present application relates to the field of control technologies, and in particular, to a method and a pan-tilt for tracking a target object.

Background

The pan and tilt head may be used to carry a load and adjust the attitude of the load, for example, to change the height, inclination and/or orientation of the load. The motion of the target object to be tracked by the load is diversified, and the target object can be automatically tracked by changing the posture of the load through the cradle head in the related technology, so that the convenience of tracking the target object by using the load is effectively improved. However, in some scenarios, the user may need more control flexibility for the load, for example, to facilitate the user to follow the target object with multiple shooting methods by using the shooting device mounted on the pan/tilt head.

How to control the motion of the handheld cloud deck to satisfy the requirement of a user on the control flexibility of a load and realize the tracking of target objects in diversified motion is a problem to be solved urgently.

Disclosure of Invention

In view of this, embodiments of the present application provide a method and a pan-tilt for tracking a target object, so as to meet a requirement of a user for control flexibility of a load, and achieve tracking of the target object with diversified motions.

In a first aspect, an embodiment of the present application provides a pan/tilt head, including: a bracket assembly including at least two relatively movable bracket parts, the bracket assembly for supporting a load; the at least two motors are respectively used for driving the corresponding bracket parts to move so as to adjust the posture of the load; the cradle head is provided with at least two tracking modes, and the number of motors which can respectively drive the bracket component to move along with the position change of the target object in the sensing range of the load in the at least two tracking modes is different, so that the target object can be tracked in different dimensions.

In a second aspect, an embodiment of the present application provides a method for tracking a target object, where the method is used for a pan/tilt head, the pan/tilt head includes a support assembly and at least two motors, the support assembly includes at least two support members capable of moving relatively and is used for supporting a load, and the at least two motors are respectively used for driving the corresponding support members to move so as to adjust a posture of the load. The method comprises the following steps: acquiring a mode selection instruction; responding to a mode selection instruction, and determining a current tracking mode from at least two tracking modes, wherein the number of motors which can respectively drive the bracket component to move along with the position change of the target object in a sensing range of a load in the at least two tracking modes is different, and the load is arranged on the bracket component; and controlling a motor corresponding to the current tracking mode by using the current tracking mode so as to realize the tracking of the load on the target object in the direction of the specified dimension.

In a third aspect, embodiments of the present application provide a computer-readable storage medium storing executable instructions that, when executed by one or more processors, may cause the one or more processors to perform the method as above.

In a fourth aspect, embodiments of the present application provide a computer program comprising executable instructions that, when executed, implement the method as above.

In the embodiment of the application, the pan-tilt has at least two tracking modes, and the number of motors which can respectively drive the bracket component to move along with the position change of the target object in the sensing range of the load in the at least two tracking modes is different, so that a user can conveniently adjust the control flexibility of the user for the load by adopting different tracking modes, and the target object can be tracked in different dimensions.

Advantages of additional aspects of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.

Drawings

The foregoing and other objects, features and advantages of embodiments of the present application will be more readily understood from the following detailed description taken in conjunction with the accompanying drawings. Embodiments of the present application will be described by way of example and not limitation in the accompanying drawings, in which:

fig. 1 is an application scenario of a method for tracking a target object and a pan-tilt provided in an embodiment of the present application;

fig. 2 is an application scenario of a method for tracking a target object and a pan-tilt head according to another embodiment of the present application;

fig. 3 is a schematic structural diagram of a pan-tilt provided in the embodiment of the present application;

fig. 4 is a schematic view of a load carried on a pan/tilt head according to an embodiment of the present application;

FIG. 5 is a schematic diagram illustrating an embodiment of the present disclosure when the target object is farther from the camera;

FIG. 6 is a schematic diagram illustrating a target object being closer to a camera in the embodiment of the present application;

fig. 7 is a schematic diagram illustrating a target object provided in the embodiment of the present application when the target object is closer to a shooting device;

FIG. 8 is a schematic view of the perspective of the camera during the passage of the target object from the side of the camera in an embodiment of the present application;

FIG. 9 is a schematic view of a perspective of a camera during a side-by-side passage of a target object through the camera according to another embodiment of the present disclosure;

fig. 10 is a schematic diagram illustrating misalignment between an optical axis and a roll axis of a photographing device according to an embodiment of the present application;

fig. 11 is a schematic diagram of a motion trajectory in a plurality of captured images for the center of the target object of fig. 10;

FIG. 12 is a schematic diagram of an interactive interface provided by an embodiment of the present application;

FIG. 13 is a dataflow diagram of target object tracking as provided by an embodiment of the present application;

FIG. 14 is an illustration of a diversified movement of a target object provided by an embodiment of the present application;

FIG. 15 is a schematic illustration of a captured image for the target object of FIG. 14;

FIG. 16 is a flowchart of a method for tracking a target object according to an embodiment of the present disclosure;

fig. 17 is a block diagram of an apparatus for tracking a target object according to an embodiment of the present disclosure;

FIG. 18 is a schematic illustration of a movable platform with a power system provided in accordance with an embodiment of the present application; and

fig. 19 is a schematic diagram of a holder and a terminal device provided in the embodiment of the present application.

Detailed Description

Reference will now be made in detail to the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.

Taking an image shooting scene as an example, the embodiment of the application can be suitable for a user to follow the shooting of the target object with the specified degree of freedom, wherein the specified degree of freedom can be a single degree of freedom, two degrees of freedom or three degrees of freedom. For example, as a target object (e.g., a photographed object) moves from far to near toward a user, in an image photographed by a camera of the user, a position (including a left-right position and an up-down position) of a face image of the photographed object in the photographed image may change, the pan/tilt head may control at least a portion of its motor to operate, so that the photographing device tracks the target object. For example, as the subject is closer to the user, not only the left-right position of the face image thereof in the captured image but also the height position of the face image thereof in the captured image changes. When the face of the photographic subject is tracked in the related art, the face image of the photographic subject can be always positioned in the center of the image, so that the follow-up function is realized. However, this also causes inconvenience for user intervention such as composition for the background image. Due to inconvenience of implementing various mirror-moving methods by a user, a shooting effect desired by the user may not be achieved.

The above scenes are merely exemplary scenes, and may be shooting scenes, scanning scenes, and the like for animal shooting, image capturing, video shooting, movie shooting, television play shooting, and the like, and are not to be construed as limiting the present application.

In order to better understand the embodiments of the present application, a cradle head and a cradle-head-based tracking technology in the related art are first exemplified.

The appearance of cloud platform not only provides stable shooting environment for the process of shooing, video recording, provides abundant fortune mirror possibility for the photographic fan. For example, the handheld pan/tilt head can rotate the camera arranged thereon around at least one of the yaw axis, the pitch axis and the roll axis to achieve a specific mirror-moving shooting effect, such as 'following hand' mirror-moving.

However, the mirror-moving method using the pan-tilt has a certain learning cost, and the primary user often cannot obtain a desired picture, and cannot stably place the photographed object in the picture. However, thanks to the development of the intelligent following algorithm, after the pan-tilt and the intelligent following algorithm are combined at present, a shot object can be well placed in a picture, so that the shooting quality is improved, and meanwhile, the shooting flexibility is greatly improved.

For example, in the related art, the cloud deck and the intelligent following algorithm are applied in combination, so that when a user moves or rotates the cloud deck, the cloud deck automatically adjusts a yaw axis and a pitch axis to adjust the posture of the shooting device, and a target object is always placed at a fixed position of a picture.

However, this method also has some disadvantages, because most of the control right of the shot picture is handed to the pan-tilt algorithm, the user can only control the position of the pan-tilt, and can not do more mirror moving methods. For example, in a scene of shooting a person and a building, a user wants to perform intelligent following only in a translation direction, and the current intelligent following algorithm cannot be realized.

For example, in the related art, the application of intelligent following of the handheld pan/tilt head is to adjust the angle of the pan/tilt head through the motions of the translation axis and the pitch axis, and the following modes of the axes cannot be set independently.

For example, a picture obtained by the intelligent tracking control method of "pan axis + tilt axis" may not be the effect desired by the photographer, for example, when a person gradually approaches is photographed, the intelligent tracking control of the tilt axis may move the picture in the vertical direction, and the framing content may change.

For another example, when the camera is not in a horizontal shooting mode (for example, vertical shooting, or shooting of a rotating picture in a First Person main View (FPV) mode), since machine learning in intelligent following cannot know this information, it still adopts the horizontal shooting mode for recognition, and therefore the problem of inaccurate recognition target is caused.

For another example, a part of the handheld pan/tilt/zoom (FPV) head supports a third-party photographing device (such as a mobile phone or a camera), and after leveling, the rotation center of the roll axis does not coincide with the optical axis center of the photographing device, so that the user cannot photograph the desired image in the FPV rotating mirror.

The cloud deck and the method for tracking the target object provided by the embodiment of the application provide the combined application of the intelligent following algorithm and various cloud deck tracking modes, so that the cloud deck can track the shot object under the required degree of freedom when a user performs specific operation such as moving a mirror, the shooting means of the user is enriched on the premise of ensuring the picture content, and the difficulty of shooting materials with higher specifications by the user is reduced.

In order to facilitate understanding of the technical aspects of the present invention, the following detailed description is made with reference to fig. 1 to 19.

Fig. 1 is an application scenario of a method for tracking a target object and a pan-tilt head provided in an embodiment of the present application.

As shown in fig. 1, as the rider changes the position (including the left-right position and the up-down position) of the rider's face image in the captured image of the photographer in the scene of the photographer riding from the far D1 to the near D2, the pan-tilt can control at least part of the motor-driven camera of the pan-tilt to follow the rider. Wherein, the cloud platform can be by the handheld cloud platform of shooter, can also be the cloud platform of setting on movable platform, if the cloud platform of putting on unmanned aerial vehicle etc. does not limit here. The photographer can also control the movable platform during shooting, such as flight trajectory control and the like.

When the face of a rider is tracked in the related art, the face image of the rider can be always positioned in the center of the image, so that the follow-up shooting function is realized. However, since most of the control right of the shot picture is given to the pan-tilt algorithm, the user can only control the position of the pan-tilt, which is inconvenient to compose the picture such as the background image, and the shooting effect expected by the photographer may not be achieved.

The method and the cloud platform for tracking the target object can realize that a user autonomously selects a motor capable of driving a load to track the target object through a mode of selecting in a tracking mode, realize free selection of multiple degrees of freedom, and facilitate implementation of a required mirror moving method and the like while tracking is realized so as to shoot an expected image. For example, the intelligent following control of the single yaw axis can be realized, and a relatively stable picture in the vertical direction is provided for a user. In addition, the method for tracking the target object and the holder provided by the embodiment of the application can also realize that the intelligent following control method at any angle can provide a more free composition mode for a user.

Fig. 2 is an application scenario of a method for tracking a target object and a pan/tilt head according to another embodiment of the present application. As shown in fig. 2, a cradle head capable of carrying a third party load is taken as an example for explanation. It should be noted that the third party load may also be integrated with the pan/tilt head, which is not limited herein.

The cloud platform in fig. 2 may comprise a carrier 200, a handle 201 and the like. The supporting body 200 may include a motor and a shaft arm, wherein the motor is configured to drive the shaft arm to rotate so as to drive the third party load to move.

Carrier 200 may be a variety of support structures including, but not limited to, a single or multi-axis adjustable-posture structure for placing a load on handle 201. For example, the load may be a camera, the pan/tilt head allowing the camera to be displaced relative to the handle 201, or rotated along one or more axes, such as the carrier 200 allowing combined translational movement of the camera along one or more of the pitch, yaw, and roll axes. For another example, the carrier 200 may allow the camera to rotate about one or more of a pitch axis, a yaw axis, and a roll axis. There may be a linkage conversion relationship between the carrier 200 and the handle 201, such that a first movement (e.g., movement or rotation) of the handle 201 may be converted to a second movement of the carrier 200. And vice versa.

In addition, the holder can also comprise a sensing system. The sensing system may include one or more sensors to sense spatial orientation, velocity, and/or acceleration (e.g., rotation and translation with respect to up to three degrees of freedom). The one or more sensors include, but are not limited to, a GPS sensor, a motion sensor, an inertial sensor, or an image sensor. The sensing data provided by the sensing system can be used to control the pose, speed and/or acceleration of the load, etc. Alternatively, the sensing system may be used to detect data of the environment of the head, such as weather conditions, the position of artificial structures, etc.

In addition, the cloud platform can also comprise a communication system. The communication system can realize that the cradle head communicates with the control terminal with the communication system through signals received and transmitted in a wired or wireless mode. The communication system may include any number of transmitters, receivers, and/or transceivers for wireless communication. The communication may be a one-way communication such that data may be transmitted from one direction. For example, one-way communication may include only the head transmitting data to the control terminal, or vice versa. One or more transmitters of the communication system may transmit data to one or more receivers of the communication system and vice versa. Alternatively, the communication may be a two-way communication, so that data may be transmitted in both directions between the head and the control terminal. Bi-directional communication includes the communication system where one or more transmitters may transmit data to one or more receivers of the communication system and vice versa.

In some embodiments, the control terminal may be connected to the cradle head or the load, and the control terminal may provide control instructions to one or more of the cradle head and the load and receive information from one or more of the cradle head and the load (e.g., position and/or motion information of the carrier 200 or the load, and data sensed by the load, such as image data captured by a camera). In some embodiments, the control data of the control terminal may include instructions on position, motion, braking, or control of the head and/or load. For example, the control data may cause a change in the position and/or orientation of the carrier 200. Control data for the control terminal may result in load control, such as controlling the operation of a camera or other image capture device (capturing still or moving images, zooming, turning on or off, switching imaging modes, changing image resolution, changing focal length, changing depth of field, changing exposure time, changing viewing angle or field of view). In some embodiments, the communication to the head and/or load may include information from one or more sensors. The communication may include sensed information transmitted from one or more different types of sensors, such as a GPS sensor, a motion sensor, an inertial sensor, a proximity sensor, or an image sensor. The sensed information may relate to position (e.g., direction, position), motion, or acceleration of the pan/tilt head and/or the load. The sensed information transmitted from the load includes data captured by the load or a state of the load. The control terminal transmits the provided control data for controlling the state of one or more of the pan/tilt head, the carrier 200 or the load. Optionally, one or more of the carrier 200 and the load may include a communication module for communicating with the control terminal, so that the control terminal may communicate separately or control the pan/tilt head and the load. The control terminal can be a remote controller of the cradle head, and can also be intelligent electronic equipment which can be used for controlling the cradle head, such as a mobile phone, an iPad, wearable electronic equipment and the like.

It should be noted that, control terminal can keep away from the cloud platform to realize the remote control to the cloud platform, can fix or detachably locate on the cloud platform, specifically can set up as required.

In some embodiments, the head may communicate with other remote devices than the control terminal, or with remote devices other than the control terminal. The control terminal can also communicate with another remote device and the cradle head. For example, the head and/or control terminal may communicate with another movable platform or a carrier or load of another movable platform. The additional remote device may be a second terminal or other computing device (e.g., a computer, desktop, tablet, smartphone, or other mobile device) when desired. The remote device may transmit data to, receive data from, transmit data to, and/or receive data from the control terminal. Alternatively, the remote device may be connected to the internet or other telecommunications network to enable data received from the pan/tilt and/or control terminal to be uploaded to a website or server.

Fig. 3 is a schematic structural diagram of a pan-tilt provided in the embodiment of the present application.

As shown in fig. 3, the holder 300 may include: a carriage assembly 31 and at least two motors 32. The bracket assembly 31 may include at least two

bracket parts

311, 312 capable of moving relatively, and the bracket assembly 31 is used for supporting the load 40. And at least two motors 32, each for driving a corresponding bracket component to move to adjust the attitude of the load 40. The pan/tilt head 300 has at least two tracking modes, and the number of the motors 32 in the at least two tracking modes, which can respectively drive the support component to move along with the position change of the target object in the sensing range of the load 40, is different, so as to realize the tracking of the target object in the directions with different dimensions.

For example, a pitch axis motor and pitch axis arm cooperate to drive the load 40 in rotation about the pitch axis. A roll shaft motor and roll shaft arm cooperate to drive the load 40 about the roll shaft. The yaw (yaw) axis motor and yaw axis arm cooperate to drive the load 40 in rotation about the yaw axis.

The pitching shaft motor can drive the pitching shaft arm to move, the rolling shaft motor can drive the rolling shaft arm to move, and the yawing shaft motor can drive the yawing shaft arm to move.

For example, the yaw axis arm may be connected to one end of the roll axis arm and the other end of the roll axis arm is connected to the pitch axis arm, but the embodiment of the present application is not limited thereto, and the yaw axis arm, the roll axis arm, and the pitch axis arm may be connected in other order.

It should be understood that the pan/tilt head 300 may also be configured to allow a load to rotate about only one, two, or four axes, etc., and is not limited thereto.

In one embodiment, the tracking modes include at least two of the first to third tracking modes shown below.

The number of the motors capable of driving the bracket component to move along with the position change of the target object in the sensing range of the load in the first tracking mode is one. For example, the motor may be any one of a pitch axis motor, a yaw axis motor, or a roll axis motor. This allows the load to track the target object with only one degree of freedom in the first tracking mode, e.g. driven by the yaw axis motor.

And a second tracking mode in which the number of motors capable of driving the stand member to move in accordance with a change in position of the target object within the sensing range of the load is two. For example, the motors may be any two of a pitch axis motor, a yaw axis motor, or a roll axis motor. This allows the load to have two degrees of freedom in the second tracking mode, such as tracking the target object under drive of the yaw axis motor and/or the pitch axis motor.

And a third tracking mode in which the number of motors capable of driving the stand member to move in accordance with a change in position of the target object within the sensing range of the load is three. For example, the motor may be any one or more of a pitch axis motor, a yaw axis motor, or a roll axis motor. This allows the load to track the target object in the third tracking mode with three degrees of freedom, such as driving at least one of the yaw axis motor, the pitch axis motor, and the roll axis motor. It should be noted that, when the lens group is driven by the roll motor to rotate around the roll shaft, the yaw axis motor and the pitch axis motor can be used to adjust the position of the load, so as to solve the problem of poor image viewing effect caused by the non-overlapping of the optical axis of the lens group and the roll shaft, which results in the non-overlapping of the image center of the target object and the image center of the captured image.

In one embodiment, the motor capable of driving the carriage assembly in the first tracking mode in response to movement of the target object comprises a yaw axis motor.

The motors capable of driving the support member to move in accordance with the movement of the target object in the second tracking mode include a yaw axis motor and a pitch axis motor. For another example, the motors capable of driving the support member to move in accordance with the movement of the target object in the second tracking mode include a roll motor and a pitch motor. For another example, the motors capable of driving the support member to move in accordance with the movement of the target object in the second tracking mode include a yaw axis motor and a roll axis motor.

The motors capable of driving the bracket component to move along with the movement of the target object in the third tracking mode comprise a yaw axis motor, a pitch axis motor and a roll axis motor.

It should be noted that, when the pan/tilt head is also capable of driving the load to perform a translational motion, the embodiments of the present application do not exclude an embodiment in which a power component (such as an air cylinder, a hydraulic cylinder, or a linear motor) for driving the translation is combined with at least one motor corresponding to any of the above tracking modes.

Fig. 4 is a schematic view of a load carried on a pan/tilt head according to an embodiment of the present application.

As shown in fig. 4, the cradle head may further include: a load securing mechanism 50. The load fixing mechanism 50 is used for fixing the load 40, and the load 40 is fixed on the load fixing mechanism 50 in an adjustable mode.

In order to facilitate the user to shoot in a horizontal shooting mode or a vertical shooting mode, the shooting device (such as a camera) can be transversely or longitudinally arranged on the holder. Specifically, the first fixing surface of the load and the second fixing surface of the load fixing mechanism are parallel or perpendicular to each other.

Wherein the load securing mechanism 50 is rotatable relative to one or more of the shaft arms. For example, the load fixing mechanism includes a rotating arm that is rotatable with respect to the pitch axis and a fixing portion that is engageable with the photographing device. For example, the fixed portion may be linearly movable relative to the rotatable arm to facilitate the fixing of loads of different sizes or different configurations. It should be noted that the load securing mechanism 50 may be a single component or may be a part of a certain shaft arm, for example, the load securing mechanism 50 may be a component of a pitch shaft arm or a yaw shaft arm, and is not limited herein.

For example, when the load needs to be loaded on the pan/tilt head, the shooting device may be fixed on the rotating arm, and the position of the fixing portion may be adjusted, so that the fixing portion can be matched with the positioning portion of the shooting device, and then the shooting device is fixed at a specified position, so that the shooting device is disposed on the load fixing mechanism.

In one embodiment, the cradle head may further include: an Inertial measurement unit (IMU for short). The inertial measurement unit can be arranged at any position on the holder and used for determining the attitude information of the part supported by the inertial measurement unit. For example, to facilitate determination of attitude information of the load, the inertial measurement unit may be provided on the load securing mechanism 50 for measuring attitude information of the securing mechanism 50. For another example, to facilitate determination of attitude information of the axle arm, an inertial measurement unit may be provided on the axle arm. The inertial measurement unit may be at least one of an accelerometer or a gyroscope, and may be used to measure the attitude, acceleration, and the like of the photographing device.

In one embodiment, if the load 40 and the pan/tilt head are integral, to facilitate measuring the pose of the load 40, an inertial measurement unit may be provided on the load 40 for measuring pose information of the load.

It should be noted that the number of the inertial measurement units may be one or more, and each of the inertial measurement units may be disposed on a different component, so as to measure pose information of the component.

In one embodiment, the head may include a handheld head, an airborne head, or the like.

For example, the carriage assembly is adapted to be secured to a movable platform having a powered system. The movable platform is an unmanned aerial vehicle for example. The movable platform may include a power mechanism and a sensing system. In addition, the movable platform may also include a communication system.

The power mechanism can comprise one or more of a rotating body, a propeller, a blade, an engine, a motor, a wheel, a bearing, a magnet and a nozzle. For example, the rotator of the power mechanism may be a self-fastening rotator, a rotator assembly, or other rotator power unit. The movable platform may have one or more powered mechanisms. All the power mechanisms may be of the same type or of different types. The powered mechanism enables the movable platform to take off from the surface vertically, or land on the surface vertically, without requiring any horizontal movement of the movable platform (e.g., without requiring taxiing on a runway). For example, the movable platform may have a plurality of horizontally oriented rotating bodies to control the lifting and/or pushing of the movable platform. The sensing system may include one or more sensors to sense peripheral obstructions, spatial orientation, velocity, and/or acceleration (e.g., rotation and translation with respect to up to three degrees of freedom) of the movable platform. The communication system may refer to the content of the relevant part of the communication system of the pan/tilt head, and is not described herein again.

For another example, the head further includes: the holding member 60, the holding member 60 is used for supporting the bracket assembly 31. The holding member 60 may function as a support bracket assembly 31, and may also function as a battery, a processor, an input/output component, and the like, which are not limited herein.

Referring to fig. 4, the pan/tilt head may include a pitch axis motor 322, a roll axis motor 323, a yaw axis motor 321, a grip member 60, a yaw axis arm 311, a load fixing mechanism 50 (which may include an inertia measuring element therein), a pitch axis arm 312, a roll axis arm 313, a camera 40, and the like.

The following describes an exemplary process of tracking a target object based on a pan/tilt unit with reference to fig. 5 to 11.

Fig. 5 is a schematic diagram illustrating a case where a target object is farther from a camera according to an embodiment of the present disclosure. Fig. 6 is a schematic diagram of a target object when the target object is closer to the shooting device in the embodiment of the present application.

For example, the captured image shown in fig. 6 can be obtained by the intelligent follow-up control method of "yaw axis + pitch axis", and the image of the rider always remains at the center of the captured image. However, in some scenes, the obtained picture is not the effect desired by the photographer, such as shooting a rider who rides closer, as shown in fig. 6, as the rider gets closer to the camera, the image of the rider becomes larger in the picture shot by the camera, causing the face image of the rider to gradually move toward the upper position of the picture shot by the camera, and in order to make the camera track the rider (face or body center, etc.), the smart follow-up control of the pitch axis causes the picture to move in the vertical direction Z, causing the change of the contents of the framing. The related art cannot easily realize the photographing effect if the background in the video that the user wants to photograph is stable and the rider is tracked in the X direction and the Y direction.

Fig. 7 is a schematic diagram of a target object when the target object is closer to a shooting device according to an embodiment of the present application.

As shown in fig. 7, in the embodiment of the present application, the degree of freedom in the Z axis direction may be limited, so that the pan/tilt head can only control the motor corresponding to the Y axis and the motor corresponding to the X axis to track the target object, and then the effects shown in fig. 7 may be achieved: along with the rider is closer to the user, the camera always tracks the rider in the Y-axis direction and the X-axis direction in the period, however, the Z-axis direction is kept unchanged, so that the view finding stability can be kept while the rider is tracked in the Y-axis direction and the X-axis direction, and the requirement of the user for a specific shooting effect is met.

Fig. 8 is a schematic view of the angle of view of the photographing device during the passage of the target object from the side of the photographing device in the embodiment of the present application. Fig. 9 is a schematic view of an angle of view of a camera during a process of passing a target object from a side of the camera according to another embodiment of the present application.

In order to more clearly understand the above tracking process, the following description is made with reference to fig. 8 and 9. Referring to fig. 8, as the rider moves, the image of the rider is always kept at the center position of the captured image, achieving a follow-up effect for the rider. However, in some scenarios, the follow-up effect cannot meet the shooting effect requirement of the user. For example, as the rider gets closer to the user, the rider occupies a greater proportion of the image in the image captured by the user, and at the same time, the user's facial image is caused to get closer to the top of the captured image, and may even approach or exceed the top of the captured image. In fig. 8, the camera position of the pan/tilt head is adjusted in the Z-axis direction, so that the rider's face and the like are always kept at the center of the captured image. The background of the shot image is changed from the background image of the left circle of fig. 8 to the background image of the right circle of fig. 8, which causes the view range of the background to obviously move upwards, and if the user's technique of moving the mirror is not skillful enough and fails to gradually raise the height of the camera in time, the situation can occur.

Referring to fig. 9, in the embodiment of the present application, because the degree of freedom of the pan/tilt head in the Z axis direction is limited, the pan/tilt head may not track the rider in the Z axis direction, but only track the rider in the X axis direction and the Y axis direction, which can reduce the difficulty of the mirror-moving operation and facilitate obtaining the shooting effect desired by the user.

The following exemplifies that the embodiments of the present application help to realize that an image of a target object is kept at the center of a captured image when the target object is tracked in three-axis degrees of freedom.

Fig. 10 is a schematic diagram illustrating misalignment between an optical axis and a roll axis of a photographing device according to an embodiment of the present application. FIG. 11 is a schematic diagram of a motion trajectory in a plurality of captured images for the center of the target object of FIG. 10

As shown in fig. 10, since the load may be provided by a third party, such as a camera purchased by a user, and the size of the cameras manufactured by different manufacturers is different, it is not easy to make the optical axis Op1 or the optical axis Op2 of the camera overlap with the Roll axis of the pan/tilt head in a calibration manner. Further, even if the optical axes Op1, Op2 of the cameras overlap with the Roll axis of the pan/tilt head by the adjustment, the optical axes Op l, Op2 are likely to be separated from the Roll axis during the process of the cameras being driven to rotate due to the difference in the center of gravity and the like of the cameras of different models. When Opl, Op2 are separated from the Roll axis, a situation as shown in FIG. 11 may result.

As shown in fig. 11, the in-situ rotation of the target object is taken as an example, the center of the target object is located at the position c of the center of the shot image, as shown by the dashed line box in fig. 11, the center image of the target object should be located at the center of the dashed line box, however, since the optical axis of the camera and the Roll axis do not overlap, the center image of the target object has a motion track as a central circle in fig. 11, and thus the required shooting effect cannot be satisfied.

In one embodiment, the load comprises a camera. Accordingly, in the third tracking mode, the position of the target object on the captured picture of the capturing device is the picture center position.

In this embodiment, when the tracked target object performs a rotational motion, the yaw axis motor and the pitch axis motor can jointly track the target object to correct the abnormal shooting effect caused by the non-overlapping between the optical axis and the Roll axis.

In one embodiment, the pan/tilt head is further configured to determine a tracking mode, the tracking mode being determined by at least one of the following.

For example, the tracking mode is determined in response to a mode selection instruction received from the user interaction interface. For example, the tracking mode is determined in response to a mode selection operation for a preset function key.

Specifically, the user interaction interface is displayed on a display located on a handle assembly, the handle assembly being used to support a rack assembly; alternatively, the user interaction interface is displayed on a display located on the load; or the user interaction interface is displayed on a display of the terminal equipment connected with the holder; or the preset function key is positioned on the holding component, and the holding component is used for supporting the bracket component.

In order to facilitate interaction between a user and the cradle head, the cradle head may be provided with an input and/or an output.

For example, the input portion may be used to input an operation instruction of the handheld pan/tilt head by a user, and the input portion may include a tracking mode input member and a control stick.

Wherein, the motion of pivot arm can be controlled to the control rocker, for example, through stirring the control rocker, realizes the pivot arm of handheld cloud platform and in the rotation of corresponding direction.

For example, the tracking mode input means may select the tracking mode mentioned in the embodiments of the present application.

In particular, the tracking mode input component may include a display screen for displaying an interactive interface in which a user may input a mode selection instruction for the processor to determine the tracking mode based on the mode selection instruction.

Fig. 12 is a schematic diagram of an interactive interface provided in an embodiment of the present application.

As shown in fig. 12, the interactive interface may include a tracking mode selection component, and after a user clicks a certain component in the interactive interface, a corresponding tracking mode is triggered. In addition, in order to facilitate the user to input the posture information and to facilitate the user to view the load posture information, an input component can be further displayed in the current interactive interface or other interactive interfaces so as to facilitate the user to input the posture, or to display the current load posture and the like.

In addition, the tracking mode input means may also include control buttons, and the tracking mode output means may include a plurality of status indicator lamps. The control key is used for selecting a tracking mode, one indicator light can correspond to one tracking mode, and the indicator light is turned on when the corresponding tracking mode is selected.

For example, after the handheld cloud deck is turned on, a user presses the control key for a short time, the rightmost lamp is turned on to indicate that the tracking mode corresponding to the rightmost side is selected, the control key is pressed for a short time again, the rightmost indicator lamp is turned off, the middle indicator lamp is turned on, and if the operation is performed again, the middle indicator lamp is turned off and the leftmost indicator lamp is turned on.

It should be understood that the selection manner of the tracking mode of the embodiment of the present application is not limited to the above manner. The number of the indicator lights is not limited to three, and may be more than three or less than three.

It will also be appreciated that the input may have other components or parts in addition to the tracking mode input means and the control joystick, for example, a switch or the like for holding the pan/tilt head.

The input unit may be provided with a processor for processing an input control command, transmitting and receiving a signal, and the like. Of course, the processor may also be provided in the handle member.

Alternatively, the Processor may be a Central Processing Unit (CPU), and the Processor may also be other general-purpose processors, Digital Signal Processors (DSP), Application Specific Integrated Circuits (ASIC), Field-Programmable Gate arrays (FPGA) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The following is an exemplary description of the pan/tilt head tracking algorithm.

In one embodiment, the load is a camera. Accordingly, the pan/tilt head is configured to convert an image captured by the imaging device in the current posture into a specified posture based on the posture information of the load, and to recognize the position of the target object from the image converted into the specified posture.

The operation capability of the pan/tilt head is limited, and particularly, for a handheld pan/tilt head, it is inconvenient to determine the position of the image of the target object in the captured image from the image by means of cloud image recognition or the like, and when the image of the target object rotates in the captured image, the pan/tilt head is inconvenient to recognize the image of the target object from the captured image based on less operations, which causes inconvenience in tracking the rotating target object by controlling the pan/tilt head in the related art. In embodiments of the present application, pose information of the camera is determined by means of the IMU and/or user input pose information, and the captured images are processed based on the pose information, such as converted to a horizontally captured coordinate system, to facilitate target object recognition of images captured by the camera in various poses to facilitate tracking of the target object in various poses.

Wherein the first resource consumed for identifying the position of the target object from the image in the current posture is larger than or equal to the second resource consumed for identifying the position of the target object from the image in the specified posture.

The target object identification process may employ a variety of related image identification techniques, such as identifying an image of the target object from the captured image by feature extraction, feature ratio, and the like, and determining the position of the image of the target object in the captured image. For example, the target object in the captured image may be recognized by a deep learning model, a lightweight recognition model, or the like.

For example, the intelligent learning module may be disposed in the cradle head or the load to train a recognition model for recognizing the target object, obtain the trained recognition model, and further recognize the captured image by using the recognition model to determine the position of the image of the target object.

In one embodiment, the location of the target object is identified based on head and shoulder image features. This helps to reduce the computational resource requirements and consumption of the recognition process.

For example, a pan-tilt is also used for: based on the position change of the target object in the image shot by the shooting device, the offset of the target object in a specified coordinate system (such as a cloud platform coordinate system) for each coordinate axis is determined. And zeroing the offset of a designated coordinate axis in a designated coordinate system based on the selected tracking mode, and controlling the corresponding motor to rotate according to the zeroing result, wherein the designated coordinate axis is a coordinate axis corresponding to the motor which prohibits the driving support component from moving along with the position change of the target object in the sensing range of the load in the selected tracking mode. The process of determining the offset can be applied to a vertical shooting mode and a horizontal shooting mode due to the rotation processing of the image.

The positional change of the target object in the image captured by the capturing device can be determined as follows.

For example, in a process in which a camera takes a plurality of frames of images, for each frame of image, first, a target image of a target object is identified from one frame of image to determine positional information of the target image in one frame of image. Then, the amount of positional change between the positional information in the different frame images is determined.

Wherein determining the amount of position change between the position information in the different frame images may include the operations of: the position change amount is based on a position change amount between first position information of a first target image in a first frame image and second position information of a second target image in a second frame image, wherein the first target image is an image of a target object in the first frame image, and the second target image is an image of the target object in the second frame image.

For example, identifying a target image of a target object from a frame of image to determine position information of the target image in the frame of image may include first converting the frame of image in a first coordinate system into a second coordinate system to obtain a frame of normalized image. Then, a target image of the target object is identified from the one frame of normalized image to determine position information of the target image in the one frame of normalized image.

In one embodiment, the pan/tilt head is specifically configured to: first, a plane deviation of the photographing device is determined based on a first coordinate of the composition target and a second coordinate normalized by a current frame. Then, the plane deviation is converted into an offset amount of the target object for each coordinate axis in the designated coordinate system.

The following describes an exemplary tracking algorithm by taking single-yaw-axis tracking as an example.

Fig. 13 is a data flow diagram of target object tracking provided in an embodiment of the present application.

Regarding image rotation transformation, IMU measures attitude information of a third-party camera, wherein the measured Euler angles are pitch, roll and yaw, and the horizontal and vertical coordinates of a certain pixel point of the recorded image are (x)_n，y_n) Then the coordinates after conversion

Can be represented by formula (1):

it should be noted that, since the user can place the third-party camera horizontally or vertically on the pan/tilt head according to the needs of the user, in order to determine the correct image rotation angle by the processor, the user may input a user input gesture in the interactive interface shown in fig. 12, so as to determine the actual gesture of the load by combining the gesture measured by the IMU and the user input gesture. This makes it possible to determine the position of the image of the target object in the captured image by converting the image into a posture that facilitates target object recognition based on the above equation (1). It should be noted that, in equation (1), only roll is transformed, so that it is considered that, when target object recognition is performed, the change of the target object to the yaw axis does not affect recognition of, for example, head features and shoulder features, and therefore, conversion to yaw may not be required. Likewise, the user turning around in place, and the like, does not similarly affect the identification of features such as head and shoulder. However, the embodiments of the present application do not exclude an embodiment in which the captured images can be converted for yaw and pitch.

The composition target is (tgt)_x，tgt_y) The current frame normalized coordinate value is (x, y), and the camera plane deviation can be shown as formula (2).

e_x＝tgt_x-x

e_y＝tgt_y-y formula (2)

The camera plane deviation needs to be converted into NED (north, east, earth coordinate system) coordinate deviation, remembering that the actual FOV is (FOV)_x，FOV_y) The deviation in the camera coordinate system is expressed by equation (3).

Wherein E is_x＝0；

The NED (north, east, and ground coordinate system) coordinate deviation is shown in formula (4).

The user can set the degree of freedom (pitch E) to follow as required_xRolling E_yAnd yaw E_z) For degrees of freedom that do not need to follow, the deviation in the NED (north, east, earth) coordinate system is zero.

For example: only following yaw E_zThen actually input into the head

As shown in formula (5).

The following schematically illustrates the tracking effect of the embodiment of the present application, taking a scene of diversified motions of a target object as an example.

Fig. 14 is an intention of diversified motions of a target object according to an embodiment of the present application. Fig. 15 is a schematic diagram of a photographed image for the target object of fig. 14.

As shown in fig. 14, the rider performs a special performance in a field of a specific shape, and the rider not only displaces in the yaw and pitch degrees of freedom but also changes in the posture in the roll degree of freedom.

As shown in fig. 15, the display (or partial display) of the preview image or the shot image may be provided, and the upper image in fig. 15 is a schematic diagram of the effect of heel-shooting the rider in the yaw and pitch degrees of freedom, and the rider does not track the rider in the roll degree of freedom. As shown in the lower image of fig. 15, the rider is slapped with the three degrees of freedom of yaw, pitch, and roll to meet the user's demand for special materials.

Fig. 16 is a flowchart of a method for tracking a target object according to an embodiment of the present application. The method for tracking the target object is used for the cloud platform, the cloud platform comprises a support assembly and at least two motors, the support assembly comprises at least two support parts capable of moving relatively and is used for supporting a load, and the at least two motors are respectively used for driving the corresponding support parts to move so as to adjust the posture of the load.

As shown in fig. 16, the method of tracking a target object may include operations S1602 to S1606.

In operation S1602, a mode selection instruction is acquired.

In this embodiment, the mode selection instruction may be determined based on a user operation input by the user on the pan/tilt head. For example, the cradle head is provided with a key, a stick and other components, and a user can input a mode selection instruction by operating the components. For another example, the cradle head may include a display screen, and the user may input the mode selection command through an interactive component (such as a virtual key, a joystick, etc.) displayed on the display screen.

For example, the object for which the user operates may be a pan/tilt head communicatively connected to the movable platform. For example, a user enters at least one of the following information on the pan/tilt head: selection information, attitude information, a designated operation (such as photographing), a subject, and parameters (such as focal length, aperture, exposure duration) of the designated operation. The holder can be integrated, such as a remote controller provided with a processor, a memory, a display screen and the like. The cloud platform can be split type, can constitute control terminal with other terminal equipment jointly like the cloud platform, constitutes control terminal jointly after cloud platform and smart mobile phone interconnect. Wherein, can install Application (APP) on the smart mobile phone, can input operating instruction, set up operating parameter etc. on this APP.

Further, the specified state instruction can also be determined and input based on gesture recognition, body sensing, voice recognition or the like. For example, a user may control the position, attitude, orientation, or other aspects of the movable platform by tilting the head. The tilt of the pan/tilt head can be detected by one or more inertial sensors and corresponding movement commands can be generated. As another example, a user may adjust an operating parameter of a load (e.g., zoom), a pose of a load (via a carrier), or other aspect of any object on a movable platform using a touch screen.

In operation S1604, a current tracking mode is determined from at least two tracking modes in response to a mode selection instruction, wherein the number of motors in the at least two tracking modes, each of which is capable of driving a carriage member to move according to a change in a position of a target object within a sensing range of a load, the load being disposed on the carriage member.

In operation S1606, a motor corresponding to the current tracking mode is controlled by using the current tracking mode, so as to achieve tracking of the target object by the load in the direction of the specified dimension.

The tracking mode, the motor, the load, the position change, etc. may refer to the above related contents, and are not described herein again.

In one embodiment, the tracking mode includes at least two of the following.

The number of motors in the first tracking mode capable of driving the carriage member to move in accordance with a change in position of the target object within the sensing range of the load is one.

And a second tracking mode in which the number of motors capable of driving the stand member to move in accordance with a change in position of the target object within the sensing range of the load is two.

And a third tracking mode in which the number of motors capable of driving the stand member to move in accordance with a change in position of the target object within the sensing range of the load is three.

For details, reference is made to the same parts of the foregoing embodiments, and further description is omitted here.

The motors capable of driving the support member to move in accordance with the movement of the target object in the second tracking mode include a yaw axis motor and a pitch axis motor.

For example, when the intelligent tracking is performed under the single yaw axis degree of freedom, the embodiment of the application can provide a relatively stable picture in the vertical direction for a user.

In one embodiment, the load comprises a camera. In the third tracking mode, the position of the target object on the shooting picture of the shooting device is the picture center position.

In one embodiment, the long sides of the photographed picture are parallel or perpendicular to each other with respect to the ground, or the long sides of the photographed picture are parallel or perpendicular to each other with respect to the horizontal plane.

In one embodiment, the method further comprises: attitude information of the load is acquired to determine a target object within a sensing range of the load based at least on the attitude information of the load.

In one embodiment, obtaining attitude information of the load comprises: attitude information of the load is detected based on the inertial measurement unit.

In one embodiment, obtaining attitude information of the load comprises: and determining the attitude information of the load based on the attitude information input by the user and the attitude information of the load acquired by the inertial measurement unit. The inertia measurement unit is arranged on the bracket component or the load.

In one embodiment, the load is a camera. Accordingly, determining the target object within the sensing range of the load based at least on the attitude information of the load comprises: and converting the image shot by the shooting device in the current posture into a specified posture based on the posture information of the load, and identifying the position of the target object from the image converted into the specified posture.

In one embodiment, the first resource consumed to identify the position of the target object from the image in the current pose is greater than or equal to the second resource consumed to identify the position of the target object from the image in the specified pose.

For details, reference is made to the same parts of the previous embodiments, and no further description is made herein.

The method and the device for intelligent tracking of the load can achieve intelligent tracking of the load at any angle, and can provide a freer composition mode for a user.

In one embodiment, the location of the target object is identified based on head and shoulder image features.

In one embodiment, with the current tracking mode, controlling a motor corresponding to the current tracking mode to achieve tracking of the target object by the load in the direction of the specified dimension may include the following operations.

First, the amount of shift of the target object for each coordinate axis in a specified coordinate system is determined based on the positional change of the target object in the image captured by the imaging device.

And then, nulling the offset of a designated coordinate axis in the designated coordinate system based on the selected tracking mode, and controlling the corresponding motor to rotate according to a nulling result, wherein the designated coordinate axis is a coordinate axis corresponding to the motor which prohibits the driving support component from moving along with the position change of the target object in the sensing range of the load in the selected tracking mode.

In one embodiment, determining the offset amount of the target object with respect to each coordinate axis in the specified coordinate system based on a position change of the target object in the image captured by the camera may include the following operations.

First, a plane deviation of the photographing device is determined based on a first coordinate of the composition target and a second coordinate normalized by a current frame.

Then, the plane deviation is converted into an offset amount of the target object for each coordinate axis in the designated coordinate system.

In one embodiment, obtaining the mode selection instruction may include the following.

For example, the tracking mode is determined in response to a mode selection instruction received from the user interaction interface.

For another example, the tracking mode is determined in response to a mode selection operation for a preset function key.

In the embodiment of the application, multiple tracking modes and intelligent tracking algorithm can be combined, for example, multiple tracking modes can be provided for a user to select by self, the user can select the tracking mode suitable for the user according to the self requirement and the unmanned aerial vehicle operation level, multiple mirror moving methods can be adopted when meeting the requirement of taking a photo conveniently, and the shooting experience is improved. Abundant mirror movement method application can be provided for users.

The following takes a handheld pan/tilt head as an example, and exemplifies the execution subject of each operation. For example, the executing body of each operation may be a handheld cradle head, and specifically, the input part of the handheld cradle head, the holding part, the processor provided in the holding part, the motor, and the like may implement corresponding functions.

The following takes a cradle head on board a movable platform as an example, and exemplifies the execution subject of each operation.

The operation of acquiring the mode selection instruction may be determined based on a user operation input by a user on a control terminal of the movable platform or the movable platform.

The operation of determining the current tracking mode from the at least two tracking modes may be determined by a control terminal of the movable platform, the pan/tilt head, or the movable platform (e.g., the processor).

The control of the operation of the motor corresponding to the current tracking mode using the current tracking mode may be performed by the pan/tilt head.

The operation of obtaining the attitude information of the load may be determined by a control terminal of the movable platform (e.g., a user input attitude), a pan-tilt (e.g., an attitude detected by the IMU), or the movable platform (e.g., a processor fusing a plurality of attitudes).

The operation of determining the target object within the sensing range of the load based on at least the attitude information of the load may be determined by a control terminal of the movable platform, a pan-tilt, the load, or the movable platform.

The operations of image processing and target object recognition may be determined by a control terminal of the movable platform, a pan-tilt, a load, or the movable platform.

It should be noted that the execution subjects of the above operations are only exemplary, and are not to be construed as limiting the present application, and may be performed by one of the movable platform, the control terminal, the shooting device, and the pan/tilt head, independently, or in combination. For example, in the case that the movable platform is a terrestrial robot, a human-computer interaction module (such as a display for displaying a human-computer interaction interface) may be disposed on the terrestrial robot, and a user may directly obtain a user operation on the interaction interface displayed on the movable platform to generate a user instruction, determine an image of a target object, and the like. Wherein independently completing comprises actively or passively, directly or indirectly obtaining respective data from other devices to perform respective operations.

According to the target object tracking method provided by the embodiment of the application, the control modes under various following modes are obtained through algorithm processing by combining the various cradle head following modes of the handheld cradle head and intelligent following, so that a user can realize the intelligent following control method only through a single degree of freedom. In addition, the embodiment of the application can automatically track at any angle, including an FPV mode and a vertical shooting mode.

It should be noted that, in practical applications, the above method may also be applied to loads other than a camera or the like to track a target object. The load includes, but is not limited to, an acoustic wave detection device, an infrared detection device, and the like.

Fig. 17 is a block diagram of an apparatus for tracking a target object according to an embodiment of the present application.

As shown in fig. 17, the apparatus 1700 for tracking a target object may include one or more processors 1710, and the one or more processors 1710 may be integrated in one processing unit or may be respectively disposed in a plurality of processing units. A computer readable storage medium 1720 for storing one or more computer programs 1721 which, when executed by a processor, implement the tracking method as above, e.g. acquiring mode selection instructions; responding to a mode selection instruction, and determining a current tracking mode from at least two tracking modes, wherein the number of motors which can respectively drive the bracket component to move along with the position change of the target object in a sensing range of a load in the at least two tracking modes is different, and the load is arranged on the bracket component; and controlling a motor corresponding to the current tracking mode by using the current tracking mode so as to realize the tracking of the load on the target object in the direction of the specified dimension.

The apparatus 1700 for tracking a target object may be provided in one execution body or in a plurality of execution bodies, respectively. For example, the apparatus 1700 for tracking a target object may be provided in a pan/tilt head. For example, part of it is arranged in the cradle head, and part of it is arranged in a control terminal that can be connected with the cradle head, such as a display screen for displaying an interactive interface.

For example, the processing unit may include a Field-Programmable Gate Array (FPGA) or one or more ARM processors. The processing unit may be connected with a non-volatile computer-readable storage medium 1720. The non-volatile computer-readable storage medium 1720 may store logic, code, and/or computer instructions that are executed by a processing unit to perform one or more steps. The non-volatile computer-readable storage medium 1720 may include one or more storage units (removable media or external memory such as an SD card or RAM). In certain embodiments, data sensed by the sensors may be transferred directly to and stored in a storage unit of the non-volatile computer-readable storage medium 1720. A storage unit of the non-transitory computer-readable storage medium 1720 may store logic, code, and/or computer instructions that are executed by a processing unit to perform various embodiments of the various methods described herein. For example, the processing unit may be configured to execute instructions to cause one or more processors of the processing unit to perform the tracking functions described above. The storage unit may store sensing module sensing data, which is processed by the processing unit. In certain embodiments, a storage unit of non-volatile computer-readable storage medium 1720 may store processing results produced by the processing unit.

In some embodiments, the processing unit may be coupled to a control module for controlling the state of the movable platform. For example, the control module may be used to control the power mechanism of the movable platform to adjust the spatial orientation, velocity, and/or acceleration of the movable platform with respect to the six degrees of freedom. Alternatively or in combination, the control module may control one or more of the carrier, load or sensing module.

The processing unit may also be coupled to a communication module for transmitting and/or receiving data with one or more peripheral devices, such as a terminal, a display device, or other remote control device. Any suitable communication method may be utilized, such as wired or wireless communication. For example, the communications module may utilize one or more local area networks, wide area networks, infrared, radio, Wi-Fi, peer-to-peer (P2P) networks, telecommunications networks, cloud networks, and the like. Alternatively, a relay station such as a signal tower, a satellite, or a mobile base station may be used.

The above components may be adapted to each other. For example, one or more components may be located on a movable platform, carrier, load, terminal, sensing system, or additional external device in communication with the foregoing. In some embodiments, one or more of the processing units and/or non-volatile computer-readable media can be located in different locations, such as on a movable platform, a carrier, a load, a terminal, a sensing system, or additional external devices in communication with the foregoing, as well as various combinations of the foregoing.

In addition, the control terminal adapted to the movable platform may include an input module, a processing unit, a memory, a display module, and a communication module, all of which are connected via a bus or similar network.

The input module includes one or more input mechanisms to obtain input generated by a user through manipulation of the input module. The input mechanisms include one or more joysticks, switches, knobs, slide switches, buttons, dials, touch screens, keypads, keyboards, mice, voice controls, gesture controls, inertial modules, and the like. The input module may be used to obtain user input for controlling any aspect of the movable platform, carrier, load, or component thereof. Any aspect includes attitude, position, heading, flight, tracking, etc. For example, the input mechanism may be a user manually setting one or more positions, each position corresponding to a preset input, to control the movable platform.

In some embodiments, the input mechanism may be operable by a user to input control commands to control the movement of the movable platform. For example, a user may input a movement pattern of the movable platform, such as auto-flight, auto-pilot, or movement according to a preset movement path, using knobs, switches, or similar input mechanisms. As another example, a user may control the position, attitude, orientation, or other aspect of the movable platform by tilting the control terminal in some manner. The tilt of the control terminal may be detected by one or more inertial sensors and corresponding movement commands generated. As another example, a user may adjust an operating parameter of the load (e.g., zoom), a pose of the load (via the carrier), or other aspect of any object on the movable platform using the input mechanisms described above.

In some embodiments, the input mechanism may be operable by a user to input the aforementioned descriptive object information. For example, the user may select an appropriate tracking mode, such as a manual tracking mode or an automatic tracking mode, using a knob, switch, or similar input mechanism. The user may also use the input mechanism to select a particular object to track, type of object to perform, or other similar information. In various embodiments, the input module may be executed by more than one device. For example, the input module may be implemented by a standard remote control with a joystick. A standard remote controller with a joystick is connected into a mobile device, such as a smartphone, running a suitable application program ("app") to generate control instructions for the movable platform. The app may be used to obtain user input.

The processing unit may be connected to the memory. The memory includes volatile or non-volatile storage media for storing data and/or logic, code, and/or program instructions executable by the processing unit for executing one or more rules or functions. The memory may include one or more storage units (removable media or external memory such as an SD card or RAM). In some embodiments, the data of the input module may be directly transferred and stored in a storage unit of the memory. The memory storage units may store logic, code, and/or computer instructions that are executed by the processing units to perform the various embodiments of the methods described herein. For example, the processing unit may be configured to execute instructions to cause one or more processors of the processing unit to process and display sensed data (e.g., images) captured from the movable platform, generate control commands based on user input, including motion commands and object information, and cause the communication module to transmit and/or receive data, etc. The storage unit may store sensed data or other data received from an external device, such as a movable platform. In some embodiments, a storage unit of the memory may store processing results generated by the processing unit.

In some embodiments, the display module may be used to display information regarding position, translational velocity, translational acceleration, direction, angular velocity, angular acceleration, or combinations thereof, as described above, for the pan-tilt and/or the load. The display module may be used to capture information transmitted by the movable platform and/or load, such as sensed data (images recorded by a camera or other image capture device), tracking data as described, control feedback data, and the like. In some embodiments, the display module may be executed by the same device as the input module. In other embodiments, the display module and the input module may be executed by different devices.

The communication module may be used to transmit and/or receive data from one or more remote devices (e.g., a movable platform, a bearer, a base station, etc.). For example, the communication module may transmit control signals (e.g., motion signals, target information, tracking control commands) to peripheral systems or devices, such as the pan/tilt head and/or the load, as described above. The communication module may include a transmitter and a receiver for receiving data from and transmitting data to the remote device, respectively. In some embodiments, the communication module may include a transceiver that combines the functionality of a transmitter and a receiver. In some embodiments, the transmitter and receiver and the processing unit may communicate with each other. The communication may be by any suitable communication means, such as wire or wireless communication.

Images captured by the movable platform during motion may be transmitted from the movable platform or imaging device back to a control terminal or other suitable device for display, playback, storage, editing, or other purposes. Such transmission may occur in real time or near real time as the imaging device captures the image. Optionally, there may be a delay between the capture and transmission of the image. In some embodiments, the imagery may be stored in the memory of the removable platform without being transferred to any other location. The user may view these images in real time, adjust object information if desired, or adjust other aspects of the movable platform or its components. The adjusted object information may be provided to the movable platform and the iterative process may continue until a desired image is obtained. In some embodiments, the imagery may be transmitted from the movable platform, the imagery device, and/or the control terminal to a remote server. For example, the imagery may be shared on some social networking platforms, such as a WeChat friend circle or a microblog.

The apparatus 1700 for tracking a target object may be configured to perform one or more operations as described above. Are not listed here.

The movable platform is exemplified below.

Fig. 18 is a schematic structural diagram of a movable platform according to an embodiment of the present application.

As shown in fig. 18, the movable platform may be a drone 180, the drone 180 may include a plurality of powered systems 181 and a foot rest. The cloud platform can set up on unmanned aerial vehicle 180.

In one embodiment, the plurality of power systems 181 of the drone 180 correspond one-to-one to the plurality of booms. Each power system 181 may include a motor assembly and a blade coupled to the motor assembly. Each power system 181 may be disposed on its corresponding horn, with the power system 181 being supported by the corresponding horn.

In addition, the drone 180 may also include a foot rest. The foot rest can be positioned below the holder and is connected with the holder. When the unmanned aerial vehicle 180 lands, can be used for the landing of unmanned aerial vehicle 180.

FIG. 19 schematically illustrates a schematic view of a movable platform according to another embodiment of the present application.

As shown in fig. 19, the movable platform is a handheld pan-tilt 190, and the handheld pan-tilt 190 may include the structure of a pan-tilt as described above. Handheld cloud platform 190 can include: cloud platform and the handle that supports the cloud platform, this handle is the part that the user gripped, can include control button to in operating the cloud platform. The handheld cradle head 190 is communicatively connected to a functional component (e.g., a camera) in the cradle to acquire image information captured by the camera.

In addition, the handheld cloud platform 190 can also be connected with a terminal device 191 (such as a mobile phone) and the like to send information such as images and the like to the mobile phone.

The above is a preferred embodiment of the present application, and it should be noted that the preferred embodiment is only for understanding the present application, and is not intended to limit the scope of the present application. Furthermore, the features of the preferred embodiments, unless otherwise specified, are applicable to both the method embodiments and the apparatus embodiments, and technical features that may be present in the same or different embodiments may be used in combination without conflict with each other.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solutions of the present application, and are not limited thereto; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art will understand that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and these modifications or substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A head, characterized in that it comprises:

a bracket assembly including at least two relatively movable bracket parts, the bracket assembly for supporting a load;

the at least two motors are respectively used for driving the corresponding bracket parts to move so as to adjust the posture of the load;

the cradle head is provided with at least two tracking modes, and the number of motors which can drive the bracket component to move along with the position change of the target object in the sensing range of the load in the at least two tracking modes is different, so that the target object can be tracked in different dimensions.

2. A head according to claim 1, wherein said tracking modes comprise at least two of:

a first tracking mode in which the number of motors that can drive the stand member to move with a change in position of a target object within a sensing range of the load is one;

a second tracking mode in which the number of motors that can drive the stand member to move with a change in position of a target object within a sensing range of the load is two;

a third tracking mode in which the number of motors that can drive the stand member to move with a change in position of a target object within a sensing range of the load is three.

3. A head according to claim 2, wherein the motors operable in the first tracking mode to drive the carriage assembly into movement with movement of the target object comprise yaw axis motors;

the motors capable of driving the bracket component to move along with the movement of the target object in the second tracking mode comprise a yaw axis motor and a pitch axis motor;

4. A head according to claim 3, wherein said load comprises a photographic device;

in the third tracking mode, the position of the target object on the shooting picture of the shooting device is the picture center position.

5. A head according to claim 1, further comprising:

and the load fixing mechanism is used for fixing the load, and the load posture is adjustably fixed on the load fixing mechanism.

6. A head according to claim 5, wherein said first fixed surface of the load and said second fixed surface of the load securing mechanism are parallel or perpendicular to each other.

7. A head according to claim 5, characterized in that it further comprises:

and the inertia measurement unit is arranged on the load fixing mechanism and used for measuring the attitude information of the load.

8. A head according to claim 1, wherein said load is a photographic device;

the cradle head is used for converting an image shot by the shooting device in the current posture into a specified posture based on the posture information of the load, and recognizing the position of the target object from the image converted into the specified posture.

9. A head according to claim 8, wherein a first resource consumed for identifying the position of said target object from the image in the current attitude is greater than or equal to a second resource consumed for identifying the position of said target object from the image in the given attitude.

10. A head according to claim 8, wherein said target object's position is identified on the basis of head and shoulder image features.

11. A head according to claim 8, wherein said head is also adapted to:

determining the offset of the target object for each coordinate axis under a specified coordinate system based on the position change of the target object in the image shot by the shooting device;

and zeroing the offset of a designated coordinate axis in the designated coordinate system based on the selected tracking mode, and controlling the corresponding motor to rotate according to the zeroing result, wherein the designated coordinate axis is a coordinate axis corresponding to the motor which prohibits the support component from being driven to move along with the position change of the target object in the sensing range of the load in the selected tracking mode.

12. A head according to claim 11, characterized in that it is specifically adapted to:

determining a plane deviation of the photographing device based on a first coordinate of the composition target and a second coordinate of the current frame normalization;

and converting the plane deviation into the offset of the target object for each coordinate axis in the specified coordinate system.

13. A head according to claim 8, wherein said head is also adapted to determine a tracking mode, said tracking mode being determined by at least one of:

determining the tracking mode in response to a mode selection instruction received from a user interaction interface;

and responding to a mode selection operation aiming at a preset function key, and determining the tracking mode.

14. A head according to claim 13, wherein said user interface is displayed on a display located on a handle member for supporting said carriage assembly; or

The user interaction interface is displayed on a display located on a load; or

The user interaction interface is displayed on a display of the terminal equipment connected with the holder; or

The preset function key is positioned on the holding component, and the holding component is used for supporting the bracket component.

15. A head according to claim 1, further comprising:

and the inertia measurement unit is arranged on the load and used for measuring the attitude information of the load.

16. A head according to claim 1, wherein said support assembly is intended to be fixed to a movable platform provided with a power system; or

The cloud platform still includes: a handle assembly for supporting the rack assembly.

17. A method for tracking a target object is characterized by being used for a cloud platform, wherein the cloud platform comprises a support component and at least two motors, the support component comprises at least two support parts capable of moving relatively and is used for supporting a load, and the at least two motors are respectively used for driving the corresponding support parts to move so as to adjust the posture of the load; the method comprises the following steps:

acquiring a mode selection instruction;

determining a current tracking mode from at least two tracking modes in response to the mode selection instruction, wherein the number of motors in the at least two tracking modes, which can respectively drive the bracket component to move along with the position change of the target object in the sensing range of the load, is different;

and controlling a motor corresponding to the current tracking mode by using the current tracking mode so as to realize the tracking of the target object by the load in the direction of the specified dimension.

18. The method of claim 17, wherein the tracking mode comprises at least two of:

19. The method of claim 18, wherein:

the motor capable of driving the bracket component to move along with the movement of the target object in the first tracking mode comprises a yaw axis motor;

20. The method of claim 19, wherein the load comprises a camera;

21. The method of claim 20, wherein the long sides of the shot are parallel or perpendicular to each other with respect to the ground, or wherein the long sides of the shot are parallel or perpendicular to each other with respect to a horizontal plane.

22. The method of claim 17, further comprising:

attitude information of the load is acquired to determine the target object within a sensing range of the load based at least on the attitude information of the load.

23. The method of claim 22, wherein the obtaining attitude information of the load comprises:

attitude information of the load is detected based on an inertial measurement unit.

24. The method of claim 22, wherein the obtaining attitude information of the load comprises:

determining attitude information of a load based on attitude information input by a user and attitude information of the load acquired by an inertial measurement unit, wherein the inertial measurement unit is arranged on the bracket assembly or the load.

25. The method of claim 22, wherein the load is a camera;

the determining the target object within the sensing range of the load based at least on the attitude information of the load comprises:

converting an image captured by the capturing device in a current posture to a specified posture based on the posture information of the load, and recognizing a position of the target object from the image converted to the specified posture.

26. The method of claim 25, wherein a first resource consumed to identify the location of the target object from the image in the current pose is greater than or equal to a second resource consumed to identify the location of the target object from the image in the specified pose.

27. The method of claim 25, wherein the target object's location is identified based on head and shoulder image features.

28. The method of claim 25, wherein the controlling, with the current tracking mode, a motor corresponding to the current tracking mode to achieve tracking of the target object by the load in a direction of a specified dimension comprises:

29. The method of claim 28, wherein determining the offset of the target object for each coordinate axis in the specified coordinate system based on the change in the position of the target object in the image captured by the camera comprises:

determining a plane deviation of the photographing device based on a first coordinate of a composition target and a second coordinate of a current frame normalization;

30. The method of claim 17, wherein obtaining the mode selection instruction comprises:

determining the tracking mode in response to a mode selection instruction received from a user interaction interface; or

And responding to a mode selection operation aiming at the preset function key, and determining the tracking mode.

31. A computer-readable storage medium having stored thereon executable instructions that, when executed by one or more processors, may cause the one or more processors to perform the method of any one of claims 17 to 30.