WO2022027596A1

WO2022027596A1 - Control method and device for mobile platform, and computer readable storage medium

Info

Publication number: WO2022027596A1
Application number: PCT/CN2020/107825
Authority: WO
Inventors: 许中研; 钱杰
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-08-07
Filing date: 2020-08-07
Publication date: 2022-02-10
Also published as: CN113906360A

Abstract

A control method and device for a mobile platform, and a computer readable storage medium and the mobile platform. The mobile platform comprises a photographing device. The control method comprises: obtaining position information of a photographing target of a photographing device; obtaining position information of an obstacle in an environment where a mobile platform is located; and determining, at least partially according to the position information of the photographing target and the position information of the obstacle, a movement path along which the mobile platform bypasses the obstacle.

Description

Control method, device, and computer-readable storage medium for removable platform

technical field

The present disclosure relates to the field, and in particular, to a control method, an apparatus, a computer-readable storage medium and a movable platform for a movable platform.

Background technique

Mobile platforms such as drones have target tracking capabilities. The target tracking function usually involves the path planning of the movable platform and the attitude adjustment of the camera. During target tracking, the movable platform can plan a moving path in real time, and move along the moving path to follow the target. In the process of following the target, the photographing device carried by the movable platform photographs the target according to preset rules, so that the tracked target is presented in the image of the photographing device.

SUMMARY OF THE INVENTION

The present disclosure provides a control method for a movable platform, the movable platform includes a photographing device, and the control method includes:

acquiring position information of the shooting target of the shooting device;

obtaining the location information of obstacles in the environment where the movable platform is located;

A path of movement of the movable platform around the obstacle is determined based at least in part on the location information of the photographed target and the location information of the obstacle.

The present disclosure also provides a control method for a movable platform, the movable platform includes a photographing device, and the control method includes:

acquiring the first pixel area corresponding to the following target and the second pixel area corresponding to the background target in the image captured by the photographing device;

The posture of the photographing device is adjusted according to the relative positional relationship between the first pixel area and the second pixel area.

The present disclosure also provides a control method for a movable platform, the control method comprising:

Obtain the location information of the no-fly zone in the environment where the movable platform is located;

According to the position information of the obstacle and the position information of the no-fly zone, a moving path of the movable platform around the obstacle and the no-fly zone is determined.

The present disclosure also provides a control device for a movable platform, the movable platform includes a photographing device, and the control device includes:

memory for storing executable instructions;

A processor for executing the executable instructions stored in the memory to perform the following operations:

acquiring position information of the shooting target of the shooting device;

memory for storing executable instructions;

The present disclosure also provides a control device for a movable platform, the control device comprising:

memory for storing executable instructions;

The present disclosure also provides a computer-readable storage medium storing executable instructions, which, when executed by one or more processors, can cause the one or more processors to execute the above control method .

The present disclosure also provides a movable platform, comprising: a photographing device, and a movable carrier; the movable carrier includes: the above-mentioned control device.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present disclosure more clearly, the accompanying drawings required in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

FIG. 1 is a flowchart of a control method for a movable platform provided by an embodiment of the present disclosure.

FIG. 2 is a schematic diagram of an application scenario of an unmanned aerial vehicle provided by an embodiment of the present disclosure.

FIG. 3 is a schematic diagram of determining a movement path according to position information of a shooting target and position information of an obstacle according to an embodiment of the present disclosure.

FIG. 4 is a schematic diagram of determining a movement path according to a distance between a waypoint and an obstacle according to an embodiment of the present disclosure.

FIG. 5 is a schematic diagram of determining a movement path according to path smoothness according to an embodiment of the present disclosure.

FIG. 6 shows a first posture adjustment strategy provided by an embodiment of the present disclosure when the first pixel area and the second pixel area do not overlap.

FIG. 7 shows an example of a second posture adjustment strategy in which the first pixel area is located in the second pixel area provided by an embodiment of the present disclosure.

FIG. 8 shows another example of a second posture adjustment strategy in which the first pixel area is located in the second pixel area provided by an embodiment of the present disclosure.

FIG. 9 shows a third attitude adjustment strategy for a linear second pixel region provided by an embodiment of the present disclosure.

FIG. 10 is a schematic diagram of determining a movement path according to location information of a shooting target and location information of a no-fly zone provided by an embodiment of the present disclosure.

FIG. 11 is a flowchart of a control method of a movable platform according to another embodiment of the present disclosure.

FIG. 12 is a flowchart of a control method of a movable platform according to still another embodiment of the present disclosure.

FIG. 13 is a schematic diagram of determining a movement path according to the position information of an obstacle and the position information of a no-fly zone according to an embodiment of the present disclosure.

FIG. 14 is a schematic structural diagram of a control device of a movable platform provided by an embodiment of the present disclosure.

FIG. 15 is a schematic structural diagram of a movable platform provided by an embodiment of the present disclosure.

detailed description

Although in the prior art, the movable platform can move along the moving path and use the photographing device to track the target during the motion, when planning the moving path in the prior art, only the environment in which the movable platform is located is usually considered. The location of the obstacle basically does not consider other factors that may affect the shooting effect and safety, which will affect the shooting effect of target tracking and the safety of the movable platform. At the same time, in the process of imaging the target, the photographing device usually only pays attention to the tracked target, and the entire composition process lacks attention to the environment where the target is located and the target objects in the environment, which will also affect the shooting effect of target tracking.

The present disclosure provides a control method, device, computer-readable storage medium and movable platform for a movable platform, by acquiring the position information of the photographing target of the photographing device and the position information of obstacles in the environment where the movable platform is located , and determine the moving path of the movable platform around the obstacle according to the position information of the shooting target and the position information of the obstacle at least in part, so as to improve the shooting effect of target tracking.

The technical solutions of the present disclosure will be clearly and completely described below with reference to the embodiments and the accompanying drawings in the embodiments. Obviously, the described embodiments are only some, but not all, embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present disclosure.

An embodiment of the present disclosure provides a control method for a movable platform. The movable platform includes a photographing device. As shown in FIG. 1 , the control method includes:

S101: Acquire position information of a photographing target of the photographing device;

S102: Acquire position information of obstacles in the environment where the movable platform is located;

S103: Determine a movement path of the movable platform around the obstacle based at least in part on the location information of the photographed target and the location information of the obstacle.

The control method of this embodiment can be applied to various movable platforms. The movable platform may be, for example, an aerial movable platform. Aerial movable platforms may include, but are not limited to, unmanned aerial vehicles, fixed-wing aircraft, rotary-wing aircraft, and the like. The movable platform can also be, for example, a ground movable platform. Ground movable platforms may include, but are not limited to, unmanned vehicles, robots, manned vehicles, and the like. The movable platform can also be, for example, a handheld device or a mobile device. Handheld setups may include, but are not limited to, handheld gimbals, PTZ cameras; mobile devices may include, but are not limited to: remote controls, smart phones/mobile phones, tablets, laptops, desktops, media content players, video game stations/ systems, virtual reality systems, augmented reality systems, wearable devices, etc.

For the convenience of description, the control method of this embodiment is described below by taking a movable platform such as an unmanned aerial vehicle as an example. Referring to FIG. 2 , the drone 100 includes: a drone body 110 , a gimbal 140 and a photographing device 130 .

The drone body 110 may include the drone body 105 , and one or more propulsion units 150 . Propulsion unit 150 may be configured to generate lift for drone 100 . Propulsion unit 150 may include rotors. The drone 100 can fly in three-dimensional space, and can rotate along at least one of a pitch axis, a yaw axis, and a roll axis.

Unmanned aerial vehicle 110 may include one or more sensors. These sensors include, for example, image sensors, distance sensors, height sensors, position sensors, and the like. As an example of an image sensor, the drone 100 may include one or more cameras 130 . In this embodiment, the photographing device 130 can be a visible light camera, an infrared camera, an ultraviolet camera, etc., and the photographing device 130 can photograph the target 160 within its field of view 170 .

The photographing device 130 is supported by the unmanned aerial vehicle 110 . The camera 130 may be directly supported by the UAV 110 , or may be supported by the UAV 110 via the vehicle 140 . When the photographing device 130 is supported by the UAV 110 via the carrier 140 , the photographing device 130 may be installed on the carrier 140 . The vehicle 140 may allow the camera 130 to rotate about at least one of a pitch axis, a yaw axis, and a roll axis to adjust the orientation of the camera 130 . Vehicle 140 may include a single-axis gimbal, a dual-axis gimbal, or a three-axis gimbal.

The drone 100 can be controlled by the remote controller 120 . The remote controller 120 may communicate with at least one of the drone body 110 , the gimbal 140 , and the photographing device 130 . The remote control 120 includes a display. The display is used to display images captured by the capturing device 130 . The remote control 120 also includes an input device. An input device may be used to receive input from a user.

The position information of the photographing target of the photographing device is acquired through S101 .

During the flight of the drone, the shooting device can shoot various shooting targets in the surrounding environment. In this embodiment, the shooting target not only includes the object that the drone wants to track. In fact, the shooting target may include any object in the surrounding environment.

The object that the drone wants to track may be referred to as a following target in this embodiment, and the following target usually refers to a movable object. In some examples, following targets may include: people, animals, vehicles (air, ground, surface, underwater), and the like. In some examples, the person may be the user operating the drone itself, or someone other than the user; the vehicle may include: a motor vehicle, a non-motor vehicle, an unmanned aerial vehicle, a manned aerial vehicle, a watercraft, and the like.

In this embodiment, the shooting target may also include a background target, and the background target may include any object other than the following target. Background objects often reflect the scene in which the following objects are located. In some examples, the background objects may include mountains, rivers, lakes, forests, seas, beaches, grasslands, various buildings, horizons, etc., and may also include sky, sun, moon, clouds, morning glow, sunset glow, and the like.

When the shooting device shoots the shooting target in its surrounding environment, the captured image can be sent to the remote control through the communication link between the remote control and the drone body, the gimbal or the shooting device, and the remote control on the monitor to display the captured image. In some examples, the photographing apparatus may send the photographed image to the remote controller in real time for the user to view, or may also store the photographed image locally in the photographing apparatus, and then send the image to the remote control for display.

In some examples, the user may manually select a follow target in the image. For the following target, the user may select one or more objects displayed in the image as the following target through the input device. The input device may be a button, a joystick, or a knob of the remote control; when the display of the remote control is a touch screen, the user can also select a follow target through the display. When the following target is selected, the UAV will track the following target during the next flight.

In other examples, the user may not need to manually select the follow target, but the follow target may be automatically selected by one of the drone, gimbal, camera, and remote control. For example, when the following target is automatically selected by the remote controller, the remote controller may select one or more objects in the image as the following target according to a preset rule. The preset rules can be: color, size, shape, etc. That is, when an object in the image conforms to a preset color, size, or shape, the object is identified as a following target. Of course, in some examples, a user confirmation step may also be added. The following target automatically selected by the remote control is used as an alternative target and displayed on the display, and the user performs a confirmation operation through the input device. Only after the user confirms a certain/some candidate target, the candidate target will be used as a follow target.

The selection process of the following target has been introduced above. For the background target, the selection process is similar to that of the following target, and will not be repeated here.

After the shooting target is selected, the position information of the shooting target can be obtained by various means, and the position information can be the position coordinates relative to a certain coordinate system.

In some examples, the location information of the shooting target can be obtained through the three-dimensional reconstruction model. The three-dimensional reconstruction model can reflect the correspondence between the pixels in the image of the photographing device and the spatial points corresponding to the pixels. The 3D reconstructed model can look like this:

Among them, (u, v) represents the coordinates of the pixels in the image in the pixel coordinate system; (X _W , Y _W , Z _W ) represents the spatial point corresponding to the pixel (u, v) in the UAV coordinate system. Coordinates; Z _C represents the distance between the spatial point corresponding to the pixel (u, v) and the optical center of the camera; A represents the internal parameter matrix of the camera; [RT] represents the external parameter matrix of the camera.

For example, for the following target, the pixels corresponding to the following target in the image captured by the photographing device can be substituted into the above-mentioned three-dimensional reconstruction model, so as to obtain the coordinates of the following target in the UAV coordinate system. Among them, Z _C can be obtained in various ways. For example, a drone or a photographing device can be installed with a distance sensor, and the distance between the following target and the optical center of the photographing device can be measured by the distance sensor; A and [RT] can be obtained by calibrated. The position information obtained above is the coordinates relative to the coordinate system of the UAV, of course, this embodiment is not limited to this. The UAV is usually equipped with a position sensor and an attitude sensor. Through the position sensor and the attitude sensor, the transformation relationship between the UAV coordinate system and the geodetic coordinate system can be obtained, so that the coordinates of the shooting target in the geodetic coordinate system can be obtained.

For the background target, in some examples, the position information of the background target can be obtained by means similar to the following target. In other examples, the drone, the camera or the remote control may store map data, and use the map data to determine the location information of the shooting target. The map data records geographic information of various areas, and these areas include at least the activity area of the drone. The geographic information may include: the type, name, location coordinates, etc. of each object in the activity area. When the user manually selects the background target or one of the drone, the gimbal, the photographing device and the remote controller automatically selects the background target, the position coordinates of the background target can be obtained by querying the map data.

After obtaining the position information of the shooting target, the position information of obstacles in the environment where the movable platform is located is obtained through S102.

During the flight, the UAV can detect obstacles through image sensors, radar, etc., or directly obtain obstacles and their location information through remote devices (such as cloud servers). Various objects in the surrounding environment can be detected, and one of the drone, gimbal, camera and remote control will recognize the object. When the recognition result shows that the object is an obstacle, the object is marked as an obstacle. This embodiment does not limit the method for identifying obstacles, and various image recognition methods can be used to identify obstacles, such as but not limited to methods based on feature detection, feature matching, and methods based on artificial neural networks.

After the obstacle is identified, the process of acquiring the location information of the obstacle may be basically similar to the process of acquiring the location information of the photographing target in S101. That is, the position information of the obstacle can be obtained through the three-dimensional reconstruction model, and the position information of the obstacle can also be obtained through the query of the map data.

In some examples, the obstacle may be, for example, an object with obvious features, such as a utility pole, street light, outdoor sign, outdoor billboard, and the like. In other examples, the obstacle may not be significantly different from the background target, or some objects may be the background target in some cases and the obstacle in others, which may depend more on Follow the position of one of the target, the background target, and the obstacle in the image, or at least the relative positions of both in the image. For trees, for example, if there are a lot of trees in the surrounding environment, it may indicate that the drone is located in a forest environment, and the trees can be considered as background objects at this time. Trees can be considered obstacles if only one or a few trees appear in the image and are in close proximity to the drone. For buildings, if some buildings are farther away from the drone or farther away from the drone than the following target, these buildings can be considered as background targets. Follow the target closer to the drone, at which point these structures can be considered obstacles.

After obtaining the position information of the shooting target and the position information of the obstacle, the moving path of the UAV around the obstacle can be determined through S103. In this embodiment, when determining the movement path of the UAV, not only the position information of the obstacle, but also the position information of the shooting target is considered. Determining the movement path of the UAV may include: establishing a loss function based at least in part on the position information of the photographed target and the position information of the obstacle, and minimizing the loss function to determine the movement path of the UAV around the obstacle.

The moving path of the UAV can be generated in real time during the target tracking process, and the moving path is usually composed of a series of waypoints, and the UAV flies along the series of waypoints. The loss function is used to determine the cost of some factors in the moving path, and the quantified cost value can be obtained. When the moving path is determined according to the position information of the shooting target and the position information of the obstacle, the loss function may include a first loss function as shown below:

Cost=cost_1(target, obstacle)

Among them, Cost represents the loss function, cost_1 represents the first loss function, target represents the location information of the shooting target, and obstacle represents the location information of the obstacle.

In the first loss function, the less the connecting line between the waypoint and the shooting target on the moving path passes through the area where the obstacle is, the smaller the value of the first loss function is. If the connection line between the waypoint and the shooting target does not pass through the area where the obstacle is located, it means that the obstacle does not appear between the shooting target and the shooting device, so that the shooting target will not be blocked by the obstacle in the captured image; otherwise, If the connection line between the waypoint and the photographing target passes through the area where the obstacle is located, it means that the obstacle appears between the photographing target and the photographing device, so that the photographing target will be blocked by the obstacle in the photographed image. That is, the principle of the first loss function is to determine whether there are obstacles between the waypoint and the shooting target. If there are more path points without occlusion, the cost value of the first loss function is smaller, so try to choose this path. That is to say, when the UAV flies along the moving path determined according to the first loss function, the photographed target in the photographed image will not be blocked by obstacles as much as possible or even completely. As shown in Figure 3, in the dotted line path on the left, a considerable part of the connection between the waypoints and the background target passes through obstacles, while in the solid line path on the right, there is almost no connection with the background target. The path point where the line passes through the obstacle, so the first loss function value of the solid line path is smaller. Under the condition that other factors are not considered, or the influence of other factors is basically the same, a solid line path with a smaller first loss function value will be selected. The first loss function is a loss function that reflects the shooting effect. When the drone flies along the solid line path, in the captured image, the background target will not be blocked by obstacles, which ensures the shooting effect.

Figure 3 illustrates the background target as an example, and the above process is similar for the following target. That is, the present application can also follow the target in the photographed image without being blocked by obstacles when the drone is flying, thereby ensuring the photographing effect. When both the following target and the background target appear in the image, in some examples, only the cost value of the following target or the background target may be considered to determine the movement path. In other examples, both the cost of following the target and the background target need to be considered. At this time, the following target and the background target can be regarded as the same type of target. If the connection between the target and the target passes through the obstacles, the more path points, the greater the cost value of the first loss function, so try not to choose this path. If the connection between the target and the target passes through the obstacles, the fewer path points, the smaller the cost value of the first loss function, and the path is selected as much as possible.

In some examples, the loss function may further include a second loss function as follows:

Cost=cost_2(distance)

Among them, cost_2 represents the second loss function, and distance represents the distance between the path point and the obstacle in the moving path. The larger the distance between the waypoint and the obstacle, the smaller the value of the second loss function.

The second loss function is a loss function reflecting the safety of the path, and the safety of the path is used to characterize whether the distance between the path point and the obstacle satisfies the safety index. When the drone flies along the moving path, the distance between the drone and the obstacle cannot be too close, so as to ensure that the drone does not collide with the obstacle and improve the flight safety of the drone. Through the second loss function, if the distance between the path point in a path and the obstacle is greater, the path is more likely to be selected as the moving path of the UAV, and vice versa, the less likely it is to be selected as the UAV's moving path. movement path. As shown in Figure 4, for the two-day paths of the solid line and the dashed line, the distance between the path point of the solid line path and the obstacle is closer. If the UAV flies according to the solid line path, the UAV may collide with the obstacle. Compared with the solid line path, the distance between the path point of the dotted line path and the obstacle is farther. If the UAV flies according to the dotted line path, it is basically impossible for the UAV to collide with the obstacle. Without considering other factors, or the influence of other factors is basically the same, the dashed line path can be used as the movement path of the UAV.

In some examples, the loss function further includes a third loss function as follows:

Cost=cost_3(curv)

Among them, cost_3 represents the third loss function, and curv represents the curvature of the moving path and/or the rate of change of the curvature, and the smaller the rate of change of the curvature and/or the curvature, the smaller the value of the third loss function.

The third loss function is a loss function reflecting the smoothness of the path, and the smoothness of the path is used to characterize the degree of bending of the path, and the degree of bending can usually be represented by the curvature and/or the rate of change of the curvature of the path. If the path is less curved, the path smoothness is better, and if the path is more curved, the path smoothness is poorer. As shown in Fig. 5, the dashed path has more bends than the solid line path, so the smoothness of the solid line path is better than that of the dashed line path. Without considering other factors, or the influence of other factors is basically the same, the solid line path is more likely to be used as the movement path of the UAV. Through the third loss function, the planned movement path can be made smoother, and the bending in the path can be minimized. The flight mileage of the small UAV and the drastic degree of the UAV attitude change save the UAV's flight time and energy consumption.

In some examples, the loss function further includes a fourth loss function as follows:

Cost=cost_4(range)

Among them, cost_4 represents the fourth loss function, and range represents the distance between the path point on the moving path and the following target, and the closer the distance is to the preset tracking distance, the smaller the value of the fourth loss function.

The fourth loss function is a loss function that reflects the distance between the photographing device and the following target. In the process of tracking the following target by the drone, in order to make the pixel area corresponding to the following target in the image captured by the photographing device at a preset position or having a preset size, the distance between the photographing device and the following target is usually determined. With certain constraints, this distance generally cannot be too close or too far, and it should be within a preset tracking distance range, otherwise the size and/or position of the pixel area corresponding to the tracking target will be affected. The preset tracking distance may be set by the user through the remote controller, or automatically set by at least one of the drone, the photographing device and the remote controller. Through the fourth loss function, the distance between the planned path point and the following target is as close as possible to the preset tracking distance, so as to ensure the shooting effect of target tracking.

The first, second, third and fourth loss functions are described above, but it does not mean that the loss function can only include one of these loss functions. In practice, the loss function may comprise at least one of the first, second, third and fourth loss functions. In one example, the loss function can look like this:

Cost=a*cost_1(target, obstacle)+b*cost_2(distance)+c*cost_3(curv)+d*cost_4(range)

Among them, a, b, c, d represent the weighting coefficients of the first, second, third and fourth loss functions, respectively. Through the above loss function, when determining the moving path of the UAV, it can also consider whether the connection between the moving path and the shooting target passes through the area where the obstacle is located, the distance between the path point in the moving path and the obstacle, The smoothness of the path and the distance between the path points on the moving path and the following target, so as to obtain a moving path that minimizes the cost of the loss function Cost. The four weighting coefficients a, b, c, and d can be set separately. The higher the weighting coefficient of the loss function, the greater the proportion of the factor corresponding to the loss function when determining the moving path. On the contrary, the proportion of factors corresponding to the loss function is lighter. The user may manually set the weighting coefficient through the remote controller, or at least one of the drone, the photographing device and the remote controller may automatically set the above-mentioned weighting coefficient. By assigning different values to the weighting coefficients, the importance of different factors in determining the moving path can be reflected. For example, if the user is more concerned about whether the shooting target will be blocked by obstacles, the weighting coefficient a of the first loss function can be set larger, and the other weighting coefficients can be set smaller. In this way, when planning the moving path, we will first try to ensure that the shooting target is not blocked by obstacles, and then consider several other factors on this basis.

The above is just one example of the loss function, and in other examples, the loss function may also be a weighted sum of the first loss function and one or both of the second, third and fourth loss functions. For example, when the user only cares about whether the shooting target will be blocked by obstacles and whether the distance between the waypoint and the obstacle is appropriate, the loss function can be a weighted sum of the first and second four loss functions.

It can be seen that the control method of this embodiment takes the relationship between the shooting target and the obstacle into consideration when determining the moving path, establishes a loss function based on the factor of whether the shooting target will be blocked by the obstacle, and plans the corresponding Therefore, when the UAV tracks the target along the moving path, the shooting target can be prevented from being blocked by obstacles as much as possible or even completely, thereby improving the shooting effect. At the same time, other factors such as the safety and smoothness of the path, and the distance between the path point and the following target can also be considered when determining the moving path, so that the planned path can guarantee a certain good effect in all aspects.

The control method of this embodiment further includes: determining the environment type of the environment where the drone is located, and determining the movement parameters of the drone according to the environment type.

Since the following target is usually in a moving state, it may appear in various scenes, or switch between various scenes. When the following target is tracked, the environment in which the drone is located will also change accordingly. Background objects in the scene can reflect the type of environment the drone is in.

In some examples, the environment types may include obstacle-dense types and obstacle-sparse types. The "dense" and "sparse" are used to characterize the density of obstacles in the environment, and the degree of density will affect the flight state of the UAV to a certain extent. That is to say, in an environment with sparse obstacles and in an environment with dense obstacles, the UAV should adopt different flight states to ensure the shooting effect and the safety of the UAV.

In some examples, the environment type of the environment in which the drone is located is determined according to the background target in the image captured by the photographing device. As mentioned above, background objects may include mountains, rivers, lakes, forests, seas, beaches, grasslands, various buildings, horizons, etc., and may also include sky, sun, moon, clouds, morning glow, sunset glow, and the like. At least one of the type, position, number, size, and relative positional relationship of the background target can determine the type of environment in which the drone is located. For example, if the background targets are lakes, seas, beaches, grasslands, and the sky, it can be considered that the environment in which the drone is located is sparse with obstacles. If the background target is a forest, it is necessary to further judge factors such as the number and size of trees. When the number of trees in the image is limited and the size is large, it can be considered that the UAV is in the forest and the environment is dense with obstacles; when the number of trees in the image is large and the size is small, it can be considered that the UAV is located Outside the forest, the environment is of sparse obstacle type. For buildings, the way of judging the type of environment is similar to the case of forests.

In other examples, the environment type of the environment where the drone is located is determined according to the detection data of the detection device carried by the drone. The detection device can be a variety of sensors carried by the UAV, and these sensors can perceive the external environment and obtain detection data. Sensors such as, but not limited to, image sensors, distance sensors, elevation sensors, and the like. By analyzing the detection data of the sensor, it can be determined whether the type of the environment is a dense type of obstacles or a type of sparse obstacles.

The flight state of the UAV is usually characterized by movement parameters. In some examples, the movement parameter of the UAV includes the speed of the UAV. If the environment type is a dense obstacle type, the speed of the UAV is determined to be the first speed; if the environment type is a sparse obstacle type, it is determined The speed of the drone is the second speed, and the second speed is greater than the first speed. That is to say, when the drone is in an environment with dense obstacles, such as forests and buildings, the flying speed of the drone should be small, so as to avoid collision with obstacles and ensure that no one The safety of the camera is ensured, and the stability of the shooting device is ensured, and the shooting effect is improved. And when the following target is a person or an animal, it can avoid the frightening of the following target due to the sudden appearance of the drone at a relatively fast speed, and ensure a good tracking effect. When the UAV is in an environment with sparse obstacles, such as beaches, grasslands, lakes, etc., the flying speed of the UAV can be larger, which provides greater flexibility for the UAV to fly.

In some other examples, the movement parameters of the UAV also include the angular velocity of the photographing device. If the environment type is a dense obstacle type, the angular velocity of the photographing device is determined to be the first angular velocity; if the environment type is a sparse obstacle type, it is determined The angular velocity of the photographing device is the second angular velocity, and the second angular velocity is greater than the first angular velocity. The angular velocity of the photographing device may include an angular velocity around at least one of a pan axis, a roll axis, and a pitch axis, and may reflect the attitude change rate of the photographing device. The greater the angular velocity, the more severe the change in the posture of the photographing device, and vice versa, the more gentle the change in the posture. When the drone is in an environment with dense obstacles, the angular velocity of the shooting device can be small, which can ensure the stability of the shooting device and improve the shooting effect. When the UAV is in an environment with sparse obstacles, the angular velocity of the photographing device can be larger to provide greater flexibility for the flight of the UAV.

During the process of the UAV circumnavigating the obstacle along the moving path, the control method of this embodiment further includes: adjusting the posture of the photographing device according to the first pixel area and/or the second pixel area. The first pixel area is the pixel area corresponding to the following target in the image captured by the photographing device; the second pixel area is the pixel area corresponding to the background target in the image captured by the photographing device.

When photographing, the photographing apparatus of this embodiment not only considers the position of the following target, but also takes the position of the background target into consideration. The posture of the photographing device may be determined according to the position of the pixel area corresponding to the following target, the position of the pixel area corresponding to the background target, or the relative positions of the pixel areas corresponding to the following target and the background target. In some cases, the above process may be commonly referred to as composition.

In some examples, the posture of the photographing device may be adjusted according to the relative positional relationship between the first pixel area and the second pixel area. The relative positional relationship may refer to the distance between the first pixel area and the second pixel area, the pixel coordinate range of the two, the size of the two, and the like. If the relative positional relationship is the first positional relationship, the posture of the photographing device is adjusted based on the first posture adjustment strategy; if the relative positional relationship is the second positional relationship, the posture of the photographing device is adjusted based on the second posture adjustment strategy; the second posture adjustment The strategy is different from the first posture adjustment strategy.

In some examples, the relative positional relationship may include: whether the first pixel area and the second pixel area overlap. When the first positional relationship is that the first pixel area and the second pixel area do not overlap, the first posture adjustment strategy includes: positioning the center of the image between the first pixel area and the second pixel area.

As shown in Fig. 6, the following target and the background target appear in the image at the same time, and the entire following target completely appears in the image. The size of the background object is small, or the distance from the photographing device is far away, and the whole of it also fully appears in the image. There is no overlap between the following target and the background target. In this case, by adjusting the posture of the photographing device, the center of the image is located between the background target and the following target. In some examples, the attitude adjustment amount of the camera can be calculated by the following formula:

Error(pitch)=0–(a*y(target)+b*y(background))/2 (1)

Error(yaw)=0–(a*x(target)+b*x(background))/2 (2)

Among them, pitch represents the pitch angle, yaw represents the heading angle; Error() represents the attitude adjustment amount of the shooting device; x(target), y(target) respectively represent the position coordinates of the following target in the predetermined coordinate system; x(background), y(background) respectively represents the position coordinates of the background target in the same predetermined coordinate system; a and b are the weighting coefficients, which indicate the weight of keeping the following target and the background target at the center of the screen, and the center of the calculated image is located between the background target and the following target. When the specific position is between, the weighting coefficient represents the proportion of the position of the following target and the background target. The predetermined coordinate system may be a pixel coordinate system or an image coordinate system.

For the example shown in FIG. 6 , the user may manually set the weighting coefficient through the remote controller, or at least one of the drone, the photographing device and the remote controller may automatically set the above-mentioned weighting coefficient. By assigning different values to the weighting coefficients, the importance of different objects in the composition can be reflected. For example, if the user prefers to make the following target closer to the center of the image, the weighting coefficient a may be set larger than the weighting coefficient b. Conversely, if the user prefers to make the background object closer to the center of the image, the weighting coefficient b may be set larger than the weighting coefficient a.

In the above description, the image center is located between the first pixel area and the second pixel area, but the embodiment is not limited thereto. Other designated positions of the image may also be located between the first pixel area and the second pixel area. The other designated positions may be positions manually set by the user through the remote controller, or automatically set by at least one of the drone, the camera, and the remote controller.

When the second positional relationship is that the first pixel area is located in the second pixel area, two situations can be considered at this time. When the second pixel area occupies the entire image, the second posture adjustment strategy may include: positioning the first pixel area at a designated position in the image. When the second pixel area does not occupy the entire image, the second posture adjustment strategy may include: making the distance between the edge pixels of the second pixel area and the edge of the image smaller than a preset distance threshold.

When the background object is relatively close to the photographing device or has a larger size, the second pixel area corresponding to the background object is likely to occupy the entire image. In this case, only the position of the pixel area corresponding to the following target in the image can be considered. As shown in Figure 7, the pixel area corresponding to the background target occupies the entire image. In this case, the attitude adjustment amount of the photographing device can be calculated by the following formula:

Error(pitch)=0–y(target) (3)

Error(yaw)=0–x(target) (4)

Among them, please refer to formula (1) and (2) for the meaning of each symbol.

When the designated position is the center of the image, the camera rotates the attitude adjustment amount obtained by the above formula in the heading and pitch directions respectively, and the first pixel area corresponding to the following target can be located at the center of the image.

In other cases, the photographing device may have photographed part of the edge of the background object, and at this time, the second pixel area does not occupy the entire image. In this case, on the premise that the following target appears in the image, the distance between the edge pixels of the second pixel area and the edge of the image can be smaller than the preset distance threshold, that is, try to make more background targets appear. in the image. The attitude adjustment amount of the camera can be calculated by the following formula:

Error(pitch)=a*(0-y(target))+b*(h-y(boundary)) (5)

Error(yaw)=a*(0–x(target))+b*(w–x(boundary)) (6)

Among them, h and w represent the height and width of the image, respectively, and the meanings of the remaining symbols are shown in formulas (1) and (2).

As shown in Figure 8, the background object does not occupy the entire image. The attitude adjustment amount is calculated by the above formula, so that while the first pixel area corresponding to the following target is located in the image, the distance between the edge pixels of the second pixel area corresponding to the background target and the edge of the image is smaller than the preset distance threshold . This allows the background objects to appear in the image as much as possible. The preset distance threshold may be manually set and adjusted by a user through a remote control, or automatically set or adjusted by at least one of the drone, the photographing device and the remote control.

It should be understood by those skilled in the art that although the first positional relationship and the second positional relationship, as well as the corresponding first attitude adjustment strategy and the second attitude adjustment strategy are described above, this is only an exemplary illustration, and the embodiment is not limited thereto. . The relative positional relationship between the first pixel area and the second pixel area can be various, and corresponding to different attitude adjustment strategies.

In other examples, the posture of the photographing device may be adjusted according to the shape of the second pixel area. If the shape of the second pixel area is of the first type, the posture of the photographing device is adjusted based on the third posture adjustment strategy; if the shape of the second pixel area is of the second type, the posture of the photographing device is adjusted based on the fourth posture adjustment strategy, The fourth attitude adjustment strategy is different from the third attitude adjustment strategy.

In some examples, the first type is a line type, and the third posture adjustment strategy includes: passing the second pixel area through a designated position in the image, and the first pixel area coincides with the designated position in at least one direction of the image.

For some background objects, the corresponding second pixel area is a straight line or a curve. For example, horizon. The horizon can usually be the boundary between the sea, grassland, and desert and the sky. As shown in Figure 9, for a linear background target similar to the horizon, making the second pixel area pass through the designated position in the image means: the camera rotates the pitch angle adjustment amount in the direction of the pitch axis, so that the horizon passes through the center of the image; the first pixel The area coincides with the designated position in at least one direction of the image means: the camera rotates the heading angle adjustment amount in the direction of the heading axis, so that the center of the following target coincides with the center of the image in the heading direction. If the second pixel area is a curve, the midpoint of the curve along the horizontal direction of the image passes through the center of the image. In this way, the horizon is located at the center of the vertical direction of the image, and the following target is located at the center of the horizontal direction of the image, and the following target is presented in front of the horizon and the backgrounds on both sides, which conforms to aesthetic habits and improves the shooting effect of the shooting device.

In other examples, the second type is a face shape, and the fourth posture adjustment strategy includes: if the first pixel area and the second pixel area do not overlap, adjusting the posture of the photographing device so that the center of the image is located at the first pixel between the area and the second pixel area; if the first pixel area is located in the second pixel area, adjust the posture of the photographing device so that the first pixel area is located at a designated position in the image; and/or, make the second pixel area The distance between the edge pixel and the edge of the image is less than the preset distance threshold.

There are many kinds of background objects in which the second pixel area is a surface, for example, the aforementioned mountains, lakes, seas, beaches, grasslands, and the sky. For such a target, the attitude adjustment method can be determined according to whether the first pixel area and the second pixel area overlap. In some examples, when the first pixel area and the second pixel area do not overlap, adjusting the posture of the photographing device so that the center of the image is located between the first pixel area and the second pixel area is the same as the aforementioned method according to the first pixel area. The relative positional relationship with the second pixel area is similar in that there is no overlap. Similarly, when the first pixel area is located in the second pixel area, adjust the posture of the photographing device so that the first pixel area is located at a designated position in the image; and/or, make the edge pixels of the second pixel area and the edge of the image The distance between them is smaller than the preset distance threshold, which is similar to the aforementioned manner in which the first pixel area is located in the second pixel area according to the relative positional relationship between the first pixel area and the second pixel area.

In other examples, the pose of the camera may also be adjusted according to the type of background object. If the type of the background target is the first background type, the posture of the photographing device is adjusted based on the fifth posture adjustment strategy; if the type of the background target is the second background type, the posture of the photographing device is adjusted based on the sixth posture adjustment strategy. The attitude adjustment strategy is different from the fifth attitude adjustment strategy.

The first background type may be a surface type, and the fifth posture adjustment strategy may include: the first pixel area is located in the upper half area of the image width direction, and the second pixel area is located in the lower half area in the image width direction; the second background type may be For non-surface types, the sixth attitude adjustment strategy includes: making the first pixel area located in the lower half area in the width direction of the image, and the second pixel area in the upper half area in the width direction of the image.

In some examples, the surface types may include: mountains, rivers, lakes, forests, seas, beaches, grasslands, buildings and other background objects rooted in the ground, and the non-surface types may include: sky, sun, moon, clouds and other background targets.

When the first background type is a surface type, the following target may be located above the background target in some cases. For example, when the background targets are grasslands, seas or lakes, and birds or drones are used as the following targets, these following targets usually move over these background targets, at this time, the first pixel area can be located in the width direction of the image. Half area, the second pixel area is located in the lower half area in the width direction of the image, which is more in line with the user's observation habits and is conducive to the improvement of the shooting effect.

It can be seen that in this embodiment, when determining the position of the following target in the image, not only the following target itself, but also the background target, the relative positional relationship between the background target and the following target, the shape of the pixel area corresponding to the background target, etc. are considered. Compared with the prior art, which only considers following the target in the composition, the content of the images shot in the present application is richer, the sense of hierarchy is stronger, the shooting effect is improved, and it has strong adaptability and adaptability to various scenes. Universality. At the same time, the present application can flexibly determine the positions of the following target and the background target in the image by adjusting the weighting coefficient, which brings more flexibility and convenience to the composition of the photographing device.

The above describes the process of determining the moving path of the movable platform around the obstacle according to the position information of the shooting target and the position information of the obstacle. As mentioned above, the obstacles mentioned therein may be utility poles, street lights, outdoor signs, outdoor billboards, and background targets in some cases. Such obstacles can be considered as obstacles in the usual sense, or a solid obstacle. However, in some cases, it may not be enough to only consider such obstacles in the process of determining the moving path, which may affect the target tracking effect of the UAV to a certain extent, especially the flight safety and shooting of the UAV. device stability. Therefore, in this embodiment, the no-fly zone in the environment where the UAV is located can also be used as an obstacle, and according to the position information of the shooting target and the position information of the no-fly zone, it is determined that the UAV bypasses the obstacle and the The movement path of the no-fly zone.

In this embodiment, the no-fly zone can be regarded as a kind of virtual obstacle, which can be implemented in multiple ways. In some examples, the no-fly zone is determined by an electronic fence device. Electronic fence devices, also known as geo-fencing devices, are provided at a location and claim one or more electronic fence boundaries that enclose the no-fly zone and control the movement of drones within the electronic fence boundaries .

The electronic fence device can be fixed at a certain location, or it can be easily movable and/or portable. The electronic fence device can be any type of device, eg visual identification device, audio identification device, radio identification device. The visual identification device can be a device that can be sensed by the optical sensor of the drone; the audio identification device can be a device that can be sensed by the audio collection device of the drone; the radio identification device can include: mobile terminals, desktop computers, portable A computer, etc., it can also be another drone or a docking station for a drone.

An electronic fence device is usually associated with one or more sets of flight controls, which are used to indicate the control content of a no-fly zone, such as prohibiting drones from flying, restricting drones from flying under certain conditions, and so on. There may be different flight controls for different drones, different users operating drones, and different electronic fence equipment.

The location information of the no-fly zone around the drone can be obtained by searching the location information of the drone itself. In some examples, the location of the no-fly zone is available to the drone itself. The UAV can store map data locally, and the map data includes the location information of the no-fly zones within various geographical ranges. After the UAV obtains its own position information through the position sensor, it can obtain the position information of the no-fly zone within the preset range around the UAV by querying the locally stored map data. Alternatively, the UAV may not store the map data locally, but obtain the map data through a remote device (for example, a cloud server), and query the obtained map data to obtain the information of the no-fly zone within the preset range around the UAV. location information.

The manner of determining the movement path according to the position information of the photographed target and the position information of the no-fly zone is similar to the foregoing manner of determining the movement path according to the position information of the photographed target and the position information of obstacles. When determining the moving path, treat the no-fly zone as an obstacle, or treat the no-fly zone as a type of obstacle. All the aforementioned steps and operations applicable to obstacles are applicable to the no-fly zone. For example, the larger the distance between the waypoint on the moving path and the no-fly zone, the smaller the value of the second loss function.

As shown in FIG. 10 , the solid line with arrows indicates the movement path. If there is a no-fly zone around the UAV, when determining the movement path, the position information of the shooting target, and the position information of the no-fly zone, or the position information of both the no-fly zone and the obstacle are used to determine the movement path. waypoint. Due to the consideration of the no-fly zone, the area between the no-fly zone and the obstacle can be regarded as a passable area, and the planned movement path passes through the communicable area without entering the no-fly zone.

When determining the moving path of target tracking, the traditional method usually only considers the influence of obstacles and does not consider the influence of the no-fly zone, so that the UAV often accidentally breaks into the no-fly zone when flying along the moving path, which will lead to no flight. Humans and aircraft violate relevant flight controls, which brings a series of safety problems. In some cases, corresponding actions will be taken when the drone enters the no-fly zone or is very close to the no-fly zone. These actions generally include: immediate landing, sudden changes in the flight direction of the drone, speed and attitude of the drone Such flight parameters will change drastically, which is very detrimental to the smoothness of the drone's flight and the stability of the shooting device. In this embodiment, the moving path can be determined by the location information of the photographed target and the location information of the no-fly zone, so that when the drone flies along the moving path, if there is a no-fly zone around it, the drone can operate in the same way as an obstacle. Before reaching the no-fly zone, decelerate in advance, or change the flight direction in advance to bypass the no-fly zone, so as to avoid entering the no-fly zone and improve flight safety. The change of flight parameters is smoother, which improves the smoothness of the drone flight and the stability of the shooting device.

Another embodiment of the present disclosure provides a control method for a movable platform, where the movable platform includes a photographing device. As shown in FIG. 11 , the control method includes:

S1101: Acquire a first pixel area corresponding to a following target and a second pixel area corresponding to a background target in the image captured by the photographing device;

S1102: Adjust the posture of the photographing device according to the relative positional relationship between the first pixel area and the second pixel area.

The movable platform of this embodiment is similar to the movable platform of the previous embodiment. The control method of this embodiment will be described below by taking a movable platform such as an unmanned aerial vehicle as an example.

When the drone flies along the moving path and the shooting device shoots the shooting target, not only the position of the following target, but also the position of the background target is considered. The posture of the photographing device may be determined according to the relative positions of the first pixel area corresponding to the following target and the second pixel area corresponding to the background target. Part of the steps, operations, and flow of the control method in this embodiment may be similar to the corresponding parts in the previous embodiment. The relative positional relationship may refer to the distance between the first pixel area and the second pixel area, the pixel coordinate range of the two, the size of the two, and the like.

In some examples, if the shape of the second pixel area is of the first type, the step of adjusting the posture of the photographing device according to the relative positional relationship between the first pixel area and the second pixel area is performed.

When the first type is a face type, if the relative positional relationship is the first positional relationship, then adjust the posture of the photographing device based on the first posture adjustment strategy; if the relative positional relationship is the second positional relationship, adjust based on the second posture adjustment strategy The posture of the photographing device; the second posture adjustment strategy is different from the first posture adjustment strategy.

Error(pitch)=0–(a*y(target)+b*y(background))/2 (1)

Error(yaw)=0–(a*x(target)+b*x(background))/2 (2)

Among them, pitch represents the pitch angle, yaw represents the heading angle; Error() represents the attitude adjustment amount of the shooting device; x(target), y(target) respectively represent the position coordinates of the following target in the predetermined coordinate system; x(background), y(background) respectively represents the position coordinates of the background target in the same predetermined coordinate system; a and b are the weighting coefficients, indicating the weight of keeping the following target and the background target at the center of the screen, and the center of the calculated image is located between the background target and the following target. When the specific position between the two, the weighting coefficient represents the proportion of the position of the following target and the background target. The predetermined coordinate system may be a pixel coordinate system or an image coordinate system.

Error(pitch)=0–y(target) (3)

Error(yaw)=0–x(target) (4)

Among them, please refer to formula (1) and (2) for the meaning of each symbol.

Error(pitch)=a*(0-y(target))+b*(h-y(boundary)) (5)

Error(yaw)=a*(0–x(target))+b*(w–x(boundary)) (6)

As shown in FIG. 8 , the second pixel area corresponding to the background object is roughly triangular in shape, and is located in the lower left area of the image, and does not occupy the entire image. The follower target is above and to the right of the background target in the image. The attitude adjustment amount is calculated by the above formula, so that the photographing device is rotated as far as possible to the left in the heading direction and to the right in the pitch direction, so that the first pixel area corresponding to the following target is located in the upper right corner of the image, and the first pixel area corresponding to the background target is located at the same time. The distance between the edge pixels of the two-pixel area and the edge of the image is less than a preset distance threshold. In FIG. 8 , the edge pixels of the second pixel area may be pixels on the hypotenuse of the second image area of the triangle. The edges of the image can be the top and right edges of the image. This allows the background objects to appear in the image as much as possible. The preset distance threshold may be manually set and adjusted by a user through a remote control, or automatically set or adjusted by at least one of the drone, the photographing device and the remote control.

In other examples, if the shape of the second pixel area is of the second type, the posture of the photographing device is adjusted according to the positions of the first pixel area and the second pixel area. The second type may be a line type. The step of adjusting the posture of the photographing device according to the positions of the first pixel area and the second pixel area includes: causing the second pixel area to pass through a designated position in the image, and the first pixel area coincides with the designated position in at least one direction of the image .

Yet another embodiment of the present disclosure provides a control method for a movable platform, where the movable platform includes a photographing device. As shown in FIG. 12 , the control method includes:

S1201: Acquire position information of obstacles in the environment where the movable platform is located;

S1202: Obtain location information of a no-fly zone in the environment where the movable platform is located;

S1203: Determine, according to the position information of the obstacle and the position information of the no-fly zone, the moving path of the movable platform to circumvent the obstacle and the no-fly zone.

The movable platform of this embodiment is similar to the movable platform of the above-mentioned embodiment. The control method of this embodiment will be described below by taking a movable platform such as an unmanned aerial vehicle as an example.

Usually, the process of determining the moving path of the movable platform around the obstacle can be determined according to the position information of the obstacle. The obstacles can be utility poles, street lights, outdoor signs, outdoor billboards, and background targets in some cases. However, in some cases, it may not be enough to only consider obstacles in the process of determining the movement path, which may affect the flight safety of the UAV to a certain extent. In this embodiment, the no-fly zone in the environment where the UAV is located can also be used as a factor for determining the moving path, and the UAV is determined to bypass the obstacle according to the position information of the obstacle and the position information of the no-fly zone. and the movement path of the no-fly zone.

In this embodiment, the no-fly zone may be implemented in various manners. In some examples, the no-fly zone is determined by an electronic fence device. Electronic fence devices, also known as geo-fencing devices, are provided at a location and claim one or more electronic fence boundaries that enclose the no-fly zone and control the movement of drones within the electronic fence boundaries .

The role of the no-fly zone in this embodiment is similar to that of an obstacle. Determining the movement path of the UAV may include: establishing a loss function based at least in part on the location information of the obstacle and the location information of the obstacle, and minimizing the loss function to determine the movement path of the UAV around the obstacle and the no-fly zone.

The moving path of the UAV can be generated in real time during the target tracking process, and the moving path is usually composed of a series of waypoints, and the UAV flies along the series of waypoints. The loss function is used to determine the cost of some factors in the moving path, and the quantified cost value can be obtained. When the movement path is determined according to the position information of the no-fly zone and the position information of the obstacle, the loss function may include a fifth loss function as shown below:

Cost=cost_5(obstacle,no_fly_zone) (7)

Among them, Cost represents the loss function, cost_5 represents the fifth loss function, obstacle represents the distance between the waypoint in the moving path and the obstacle, no_fly_zone represents the distance between the waypoint in the moving path and the no-fly zone, and the waypoint and The larger the distance of the no-fly zone or the obstacle, the smaller the value of the fifth loss function.

The fifth loss function is a loss function reflecting the safety of the path, and the safety of the path is used to characterize whether the distance between the path point and the obstacle or the no-fly zone satisfies the safety index. When the drone flies along the moving path, the distance between the drone and the obstacle cannot be too close, so as to ensure that the drone does not collide with the obstacle and improve the flight safety of the drone. At the same time, drones cannot enter the no-fly zone, so as to avoid violating relevant flight controls. Through the fifth loss function, if the distance between the path point in a path and the obstacle and the no-fly zone is greater, the path is more likely to be selected as the moving path of the UAV, otherwise, the less likely it is to be selected as the UAV's moving path The movement path of the drone. As shown in FIG. 13 , the solid line with arrows indicates the movement path. If there are no-fly zones and obstacles around the UAV, when determining the movement path, the waypoints of the movement path are determined based on the position information of both the obstacles and the no-fly zone. Due to the consideration of the no-fly zone, the area between the no-fly zone and the obstacle can be regarded as a passable area, and the planned movement path passes through the communicable area without entering the no-fly zone.

When determining the moving path of target tracking, the traditional method usually only considers the influence of obstacles and does not consider the influence of the no-fly zone, so that the UAV often accidentally breaks into the no-fly zone when flying along the moving path, which will lead to no flight. Humans and aircraft violate relevant flight controls, which brings a series of safety problems. In some cases, corresponding actions will be taken when the drone enters the no-fly zone or is very close to the no-fly zone. These actions generally include: immediate landing, sudden changes in the flight direction of the drone, speed and attitude of the drone Such flight parameters will change drastically, which is very detrimental to the smoothness of the drone's flight and the stability of the shooting device. In this embodiment, the moving path can be determined according to the position information of the obstacle and the position information of the no-fly zone, so that when the drone flies along the moving path, if there is a no-fly zone around it, the drone can fly in the same way as an obstacle. Before reaching the no-fly zone, decelerate in advance, or change the flight direction in advance to bypass the no-fly zone, so as to avoid entering the no-fly zone and improve flight safety. The change of flight parameters is smoother, which improves the smoothness of the drone flight and the stability of the shooting device.

Yet another embodiment of the present disclosure further provides a control device for a movable platform. The movable platform includes a photographing device. As shown in FIG. 14 , the control device includes:

memory for storing executable instructions;

acquiring position information of the shooting target of the shooting device;

The control device of this embodiment can basically perform various operations corresponding to the steps of the control method of the above-mentioned embodiment.

In some examples, the processor is further configured to perform the following operations: establish a loss function based at least in part on the location information of the photographed target and the location information of the obstacle; minimize the loss function to determine the possible The mobile platform circumvents the movement path of the obstacle.

In some examples, the loss function includes a first loss function, and the connection between the fewer waypoints on the moving path and the shooting target passes through the area where the obstacle is located, the first loss function the smaller the value.

In some examples, the loss function includes a second loss function, and the greater the distance between the path point on the moving path and the obstacle, the smaller the value of the second loss function.

In some examples, the loss function includes a third loss function, and the smaller the curvature and/or the rate of change of the curvature of the moving path, the smaller the value of the third loss function.

In some examples, the shooting target includes a following target, the loss function includes a fourth loss function, and the closer the distance between the path point on the moving path and the following target is to a preset tracking distance, the fourth loss function The value of the loss function is smaller.

In some examples, the processor is further configured to perform the following operations: determine an environment type of the environment in which the movable platform is located; and determine movement parameters of the movable platform according to the environment type.

In some examples, the environment types include obstacle-dense types and obstacle-sparse types.

In some examples, the processor is further configured to perform the following operation: determine an environment type of the environment in which the movable platform is located according to a background object in the image captured by the photographing device.

In some examples, the processor is further configured to perform the following operation: according to the detection data of the detection device carried on the movable platform, determine the environment type of the environment where the movable platform is located.

In some examples, the processor is further configured to perform the following operations: if the environment type is a dense obstacle type, determining the speed of the movable platform to be the first speed; if the environment type is a sparse obstacle type type, the speed of the movable platform is determined to be a second speed, and the second speed is greater than the first speed.

In some examples, the processor is further configured to perform the following operations: if the environment type is a type with dense obstacles, determine the angular velocity of the photographing device to be the first angular velocity; if the environment type is a type with sparse obstacles , then it is determined that the angular velocity of the photographing device is a second angular velocity, and the second angular velocity is greater than the first angular velocity.

In some examples, the shooting target includes a following target and a background target.

In some examples, the processor is further configured to perform the following operations: adjust the posture of the photographing device according to the first pixel area and/or the second pixel area; wherein the first pixel area is where the following target is located The pixel area corresponding to the image captured by the photographing device; the second pixel area is the pixel area corresponding to the background object in the image captured by the photographing device.

In some examples, the processor is further configured to perform the following operation: adjust the posture of the photographing device according to the relative positional relationship between the first pixel area and the second pixel area.

In some examples, the processor is further configured to perform the following operations: if the relative positional relationship is a first positional relationship, adjust the posture of the photographing device based on a first posture adjustment strategy; if the relative positional relationship is For the second positional relationship, the posture of the photographing device is adjusted based on a second posture adjustment strategy; the second posture adjustment strategy is different from the first posture adjustment strategy.

In some examples, the first positional relationship is no coincidence, and the first posture adjustment strategy includes: positioning the center of the image between the first pixel area and the second pixel area; the first The two positional relationship is that the first pixel area is located in the second pixel area, and the second posture adjustment strategy includes: making the first pixel area located at a specified position in the image; and/or making the first pixel area The distance between the edge pixels of the two-pixel area and the edge of the image is less than a preset distance threshold.

In some examples, the processor is further configured to perform the following operation: adjust the posture of the photographing device according to the shape of the second pixel area.

In some examples, the processor is further configured to perform the following operations: if the shape of the second pixel region is of the first type, adjust the posture of the photographing device based on a third posture adjustment strategy;

If the shape of the second pixel area is of the second type, the posture of the photographing device is adjusted based on a fourth posture adjustment strategy, and the fourth posture adjustment strategy is different from the third posture adjustment strategy.

In some examples, the first type is a line type, and the third posture adjustment strategy includes: causing the second pixel area to pass through a designated position in the image, the first pixel area in the image Coincidence with the designated position in at least one direction.

In some examples, the second type is a face type, and the fourth posture adjustment strategy includes: if the first pixel area and the second pixel area do not overlap, adjusting the posture of the photographing device to make The center of the image is located between the first pixel area and the second pixel area; if the first pixel area is located within the second pixel area, adjust the posture of the photographing device so that the The first pixel area is located at a specified position in the image; and/or the distance between the edge pixel point of the second pixel area and the edge of the image is smaller than a preset distance threshold.

In some examples, a no-fly zone in the environment in which the movable platform is located is used as the obstacle.

In some examples, the processor is further configured to perform the following operation: determine, according to the location information of the photographing target and the location information of the no-fly zone, that the movable platform can circumvent the obstacle and the prohibited area. The movement path of the fly zone.

Another embodiment of the present disclosure further provides a control device for a movable platform, where the movable platform includes a photographing device. As shown in FIG. 14 , the control device includes:

memory for storing executable instructions;

In some examples, the processor is further configured to perform the following operation: if the shape of the second pixel area is of a first type, perform the relative operation according to the first pixel area and the second pixel area Position relationship, the steps of adjusting the posture of the photographing device.

In some examples, the first type includes: face type.

In some examples, if the shape of the second pixel area is of the second type, the posture of the photographing device is adjusted according to the positions of the first pixel area and the second pixel area.

In some examples, the second type includes: line type.

In some examples, the processor is further configured to: cause the second pixel area to pass through a designated location in the image, the first pixel area and the image in at least one direction The specified positions coincide.

memory for storing executable instructions;

obtaining the position information of obstacles in the environment where the movable platform is located;

In some examples, the processor is further configured to: establish a loss function based at least in part on the no-fly zone location information and the obstacle location information; minimize the loss function to determine the A movable platform circumvents the obstacle and the no-fly zone's movement path.

In some examples, the loss function includes a fifth loss function, and the greater the distance between the path point on the moving path and the obstacle and the no-fly zone, the smaller the value of the fifth loss function is .

In some examples, the loss function includes a second loss function, and the smaller the curvature and/or the rate of change of the curvature of the moving path, the smaller the value of the second loss function.

In some examples, the no-fly zone is determined by electronic fence, flight control information.

In some examples, the processor is further configured to obtain location information for the no-fly zone from at least one of a remote device local to the movable platform and a remote device that can communicate with the movable platform.

In some examples, the processor is further configured to measure the location information of the obstacle via sensors of the movable platform; and/or obtain from a remote device communicable with the movable platform location information of the obstacle.

Still another embodiment of the present disclosure also provides a computer-readable storage medium storing executable instructions that, when executed by one or more processors, can cause the one or more processors to The control methods described in the above embodiments are executed.

A computer-readable storage medium, for example, can be any medium that can contain, store, communicate, propagate, or transmit instructions. For example, a readable storage medium may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, apparatus, device, or propagation medium. Specific examples of readable storage media include: magnetic storage devices, such as magnetic tapes or hard disks (HDDs); optical storage devices, such as compact disks (CD-ROMs); memories, such as random access memory (RAM) or flash memory; and/or wired /Wireless communication link.

Still another embodiment of the present disclosure further provides a movable platform, as shown in FIG. 15 , including: a photographing device, and a movable carrier; the movable carrier includes: the control device of the above-mentioned embodiment. In some examples, the movable carrier includes a drone, an unmanned vehicle, an unmanned boat, or a robot.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, only the division of the above functional modules is used for illustration. The internal structure is divided into different functional modules to complete all or part of the functions described above. For the specific working process of the apparatus described above, reference may be made to the corresponding process in the foregoing method embodiments, and details are not described herein again.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, but not to limit them; although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions described in the foregoing embodiments can still be modified, or some or all of the technical features thereof can be equivalently replaced; in the case of no conflict, the features in the embodiments of the present disclosure can be combined arbitrarily; and these modifications or replacements , does not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present disclosure.

Claims

A control method of a movable platform, wherein the movable platform includes a photographing device, wherein the control method comprises:

acquiring position information of the shooting target of the shooting device;

obtaining the location information of obstacles in the environment where the movable platform is located;

A path of movement of the movable platform around the obstacle is determined based at least in part on the location information of the photographed target and the location information of the obstacle.
The control method of claim 1, wherein the movement of the movable platform around the obstacle is determined at least in part based on the position information of the photographed target and the position information of the obstacle path, including:

establishing a loss function based at least in part on the location information of the photographed target and the location information of the obstacle;

The loss function is minimized to determine a path of movement of the movable platform around the obstacle.
The control method according to claim 2, wherein the loss function comprises a first loss function, and the connection line between the fewer path points on the moving path and the shooting target passes through the location of the obstacle. area, the smaller the value of the first loss function is.
The control method according to claim 2, wherein the loss function includes a second loss function, and the greater the distance between the path point on the moving path and the obstacle, the greater the distance of the second loss function. the smaller the value.
The control method according to claim 2, wherein the loss function includes a third loss function, and the smaller the curvature and/or the rate of change of the curvature of the moving path, the higher the value of the third loss function. small.
The control method according to any one of claims 2-5, wherein the shooting target includes a following target, the loss function includes a fourth loss function, and the path point on the moving path is the same as the following target The closer the distance is to the preset tracking distance, the smaller the value of the fourth loss function.
The control method of claim 1, further comprising:

determining the environment type of the environment in which the mobile platform is located;

Movement parameters of the movable platform are determined according to the type of environment.
The control method according to claim 7, wherein the environment types include a dense obstacle type and a sparse obstacle type.
The control method according to claim 8, wherein the determining the environment type of the environment where the movable platform is located comprises:

The environment type of the environment in which the movable platform is located is determined according to the background target in the image captured by the photographing device.
The control method according to claim 8, wherein the determining the environment type of the environment where the movable platform is located comprises:

According to the detection data of the detection device mounted on the movable platform, the environment type of the environment where the movable platform is located is determined.
The control method according to claim 8, wherein the movement parameter of the movable platform includes the speed of the movable platform, and the determining the movement parameter of the movable platform according to the environment type includes:

If the environment type is a type of dense obstacles, determining the speed of the movable platform as the first speed;

If the type of the environment is a type with sparse obstacles, the speed of the movable platform is determined to be a second speed, and the second speed is greater than the first speed.
The control method according to claim 8, wherein the movement parameter of the movable platform comprises the angular velocity of the photographing device, and the determining the movement parameter of the movable platform according to the environment type comprises:

If the environment type is a type of dense obstacles, determining the angular velocity of the photographing device as the first angular velocity;

If the environment type is a type of sparse obstacles, it is determined that the angular velocity of the photographing device is a second angular velocity, and the second angular velocity is greater than the first angular velocity.
The control method according to claim 1, wherein the shooting target includes a following target and a background target.
The control method according to claim 13, wherein, in the process of the movable platform detouring the obstacle along the moving path, further comprising:

Adjust the posture of the photographing device according to the first pixel area and/or the second pixel area;

in,

The first pixel area is a pixel area corresponding to the following target in the image captured by the photographing device;

The second pixel area is a pixel area corresponding to the background object in the image captured by the capturing device.
The control method according to claim 14, wherein the adjusting the posture of the photographing device according to the first pixel area and the second pixel area comprises:

The posture of the photographing device is adjusted according to the relative positional relationship between the first pixel area and the second pixel area.
The control method according to claim 15, wherein,

If the relative positional relationship is the first positional relationship, adjusting the posture of the photographing device based on the first posture adjustment strategy;

If the relative positional relationship is the second positional relationship, the posture of the photographing device is adjusted based on a second posture adjustment strategy; the second posture adjustment strategy is different from the first posture adjustment strategy.
The control method according to claim 15, wherein,

The first positional relationship is not overlapping, and the first attitude adjustment strategy includes:

centering the image between the first pixel area and the second pixel area;

The second positional relationship is that the first pixel area is located in the second pixel area, and the second posture adjustment strategy includes:

positioning the first pixel area at a specified position in the image;

and / or,

The distance between the edge pixels of the second pixel area and the edge of the image is less than a preset distance threshold.
The control method according to claim 14, wherein the adjusting the posture of the photographing device according to the first pixel area and/or the second pixel area comprises:

The posture of the photographing device is adjusted according to the shape of the second pixel area.
The control method of claim 18, wherein:

If the shape of the second pixel area is of the first type, adjusting the posture of the photographing device based on a third posture adjustment strategy;

If the shape of the second pixel area is of the second type, the posture of the photographing device is adjusted based on a fourth posture adjustment strategy, and the fourth posture adjustment strategy is different from the third posture adjustment strategy.
The control method according to claim 19, wherein the first type is linear, and the third attitude adjustment strategy comprises:

The second pixel area is caused to pass through a designated position in the image, and the first pixel area is coincident with the designated position in at least one direction of the image.
The control method according to claim 19, wherein the second type is a face shape, and the fourth attitude adjustment strategy comprises:

If the first pixel area and the second pixel area do not overlap, adjusting the posture of the photographing device so that the center of the image is located between the first pixel area and the second pixel area;

If the first pixel area is located in the second pixel area, adjust the posture of the photographing device so that the first pixel area is located at a designated position in the image; and/or, make the second pixel area The distance between the edge pixel point of the pixel area and the edge of the image is less than a preset distance threshold.
The control method according to claim 2, wherein a no-fly zone in the environment where the movable platform is located is used as the obstacle.
23. The control method according to claim 22, wherein the movement of the movable platform around the obstacle is determined at least in part based on the position information of the photographed target and the position information of the obstacle path, including:

According to the location information of the photographed target and the location information of the no-fly zone, a moving path of the movable platform to circumvent the obstacle and the no-fly zone is determined.
A control method of a movable platform, wherein the movable platform includes a photographing device, wherein the control method comprises:

acquiring the first pixel area corresponding to the following target and the second pixel area corresponding to the background target in the image captured by the photographing device;

The posture of the photographing device is adjusted according to the relative positional relationship between the first pixel area and the second pixel area.
The control method according to claim 24, wherein,

If the shape of the second pixel area is of the first type, the step of adjusting the posture of the photographing device according to the relative positional relationship between the first pixel area and the second pixel area is performed.
The control method of claim 25, wherein the first type comprises: a face type.
The control method according to claim 25 or 26, wherein,

If the relative positional relationship is the first positional relationship, adjusting the posture of the photographing device based on the first posture adjustment strategy;

If the relative positional relationship is the second positional relationship, the posture of the photographing device is adjusted based on a second posture adjustment strategy; the second posture adjustment strategy is different from the first posture adjustment strategy.
The control method according to claim 27, wherein,

The first positional relationship is not overlapping, and the first attitude adjustment strategy includes:

centering the image between the first pixel area and the second pixel area;

The second positional relationship is that the first pixel area is located in the second pixel area, and the second posture adjustment strategy includes:

The first pixel area is located at a specified position in the image; and/or the distance between the edge pixel point of the second pixel area and the edge of the image is smaller than a preset distance threshold.
The control method according to claim 24, wherein,

If the shape of the second pixel area is of the second type, the posture of the photographing device is adjusted according to the positions of the first pixel area and the second pixel area.
The control method of claim 29, wherein the second type comprises: a line type.
The control method according to claim 30, wherein the step of adjusting the posture of the photographing device according to the positions of the first pixel area and the second pixel area comprises:

The second pixel area is caused to pass through a designated position in the image, and the first pixel area is coincident with the designated position in at least one direction of the image.
A control method for a movable platform, characterized in that the control method comprises:

obtaining the location information of obstacles in the environment where the movable platform is located;

Obtain the location information of the no-fly zone in the environment where the movable platform is located;

According to the position information of the obstacle and the position information of the no-fly zone, a moving path of the movable platform around the obstacle and the no-fly zone is determined.
The control method according to claim 32, characterized in that, according to the position information of the obstacle and the position information of the no-fly zone, it is determined that the movable platform detours the obstacle and the no-fly zone. The movement path of the fly zone, including:

establishing a loss function based at least in part on the no-fly zone location information and the obstacle location information;

The loss function is minimized to determine a path of movement of the movable platform around the obstacle and the no-fly zone.
The control method according to claim 33, wherein the loss function includes a fifth loss function, and the greater the distance between the path point on the moving path and the obstacle and the no-fly zone, the greater the distance between the path point and the no-fly zone. The value of the fifth loss function is smaller.
The control method according to claim 33, wherein the loss function comprises a second loss function, and the smaller the curvature and/or the rate of change of the curvature of the moving path, the higher the value of the second loss function. small.
The control method according to any one of claims 32 to 35, wherein the no-fly zone is determined by electronic fence and flight control information.
The control method according to any one of claims 32 to 35, wherein the acquiring the position information of the no-fly zone in the environment where the movable platform is located comprises:

The location information of the no-fly zone is obtained from at least one of the remote devices local to the movable platform and communicable with the movable platform.
The control method according to any one of claims 32 to 35, wherein the acquiring position information of obstacles in the environment where the movable platform is located comprises:

The location information of the obstacle is measured by sensors of the movable platform; and/or the location information of the obstacle is obtained from a remote device that can communicate with the movable platform.
A control device for a movable platform, the movable platform includes a photographing device, wherein the control device includes:

memory for storing executable instructions;

A processor for executing the executable instructions stored in the memory to perform the following operations:

acquiring the location information of the photographing target of the photographing device;

obtaining the position information of obstacles in the environment where the movable platform is located;

A path of movement of the movable platform around the obstacle is determined based at least in part on the location information of the photographed target and the location information of the obstacle.
The control device of claim 39, wherein the processor is further configured to perform the following operations:

establishing a loss function based at least in part on the location information of the photographed target and the location information of the obstacle;

The loss function is minimized to determine a path of movement of the movable platform around the obstacle.
The control device according to claim 40, wherein the loss function includes a first loss function, and the connecting line between the fewer waypoints on the moving path and the shooting target passes through the location of the obstacle. area, the smaller the value of the first loss function is.
The control device according to claim 40, wherein the loss function includes a second loss function, and the greater the distance between the path point on the moving path and the obstacle, the greater the distance of the second loss function. the smaller the value.
The control device according to claim 40, wherein the loss function includes a third loss function, and the smaller the curvature and/or the rate of change of the curvature of the moving path, the higher the value of the third loss function. small.
The control device according to any one of claims 40-43, wherein the shooting target includes a following target, the loss function includes a fourth loss function, and the path point on the moving path is the same as the following target The closer the distance is to the preset tracking distance, the smaller the value of the fourth loss function.
The control device of claim 39, wherein the processor is further configured to perform the following operations:

determining the environment type of the environment in which the movable platform is located;

Movement parameters of the movable platform are determined according to the type of environment.
46. The control device of claim 45, wherein the environment types include a dense obstacle type and a sparse obstacle type.
The control device of claim 46, wherein the processor is further configured to perform the following operations:

The environment type of the environment in which the movable platform is located is determined according to the background target in the image captured by the photographing device.
The control device of claim 46, wherein the processor is further configured to perform the following operations:

According to the detection data of the detection device mounted on the movable platform, the environment type of the environment where the movable platform is located is determined.
The control device of claim 46, wherein the processor is further configured to perform the following operations:

If the environment type is a dense type of obstacles, determining that the speed of the movable platform is the first speed;

If the type of the environment is a type with sparse obstacles, the speed of the movable platform is determined to be a second speed, and the second speed is greater than the first speed.
The control device of claim 46, wherein the processor is further configured to perform the following operations:

If the environment type is a type of dense obstacles, determining the angular velocity of the photographing device as the first angular velocity;

If the type of the environment is a type with sparse obstacles, it is determined that the angular velocity of the photographing device is a second angular velocity, and the second angular velocity is greater than the first angular velocity.
The control device of claim 39, wherein the shooting target includes a following target and a background target.
The control device of claim 51, wherein the processor is further configured to perform the following operations:

Adjust the posture of the photographing device according to the first pixel area and/or the second pixel area;

in,

The first pixel area is a pixel area corresponding to the following target in the image captured by the photographing device;

The second pixel area is a pixel area corresponding to the background object in the image captured by the capturing device.
The control device of claim 52, wherein the processor is further configured to perform the following operations:

The posture of the photographing device is adjusted according to the relative positional relationship between the first pixel area and the second pixel area.
The control device of claim 53, wherein the processor is further configured to perform the following operations:

If the relative positional relationship is the first positional relationship, adjusting the posture of the photographing device based on the first posture adjustment strategy;

If the relative positional relationship is the second positional relationship, the posture of the photographing device is adjusted based on a second posture adjustment strategy; the second posture adjustment strategy is different from the first posture adjustment strategy.
The control device of claim 53, wherein:

The first positional relationship is not overlapping, and the first attitude adjustment strategy includes:

centering the image between the first pixel area and the second pixel area;

The second positional relationship is that the first pixel area is located in the second pixel area, and the second posture adjustment strategy includes:

positioning the first pixel area at a specified position in the image;

and / or,

The distance between the edge pixels of the second pixel area and the edge of the image is less than a preset distance threshold.
The control device of claim 52, wherein the processor is further configured to perform the following operations:

The posture of the photographing device is adjusted according to the shape of the second pixel area.
The control device of claim 56, wherein the processor is further configured to perform the following operations:

If the shape of the second pixel area is of the first type, adjusting the posture of the photographing device based on a third posture adjustment strategy;

If the shape of the second pixel area is of the second type, the posture of the photographing device is adjusted based on a fourth posture adjustment strategy, and the fourth posture adjustment strategy is different from the third posture adjustment strategy.
The control device of claim 57, wherein the first type is linear, and the third attitude adjustment strategy comprises:

The second pixel area is caused to pass through a designated position in the image, and the first pixel area is coincident with the designated position in at least one direction of the image.
The control device of claim 57, wherein the second type is a face type, and the fourth posture adjustment strategy comprises:

If the first pixel area and the second pixel area do not overlap, adjusting the posture of the photographing device so that the center of the image is located between the first pixel area and the second pixel area;

If the first pixel area is located in the second pixel area, adjust the posture of the photographing device so that the first pixel area is located at a designated position in the image; and/or, make the second pixel area The distance between the edge pixel point of the pixel area and the edge of the image is less than a preset distance threshold.
The control device of claim 40, wherein a no-fly zone in the environment where the movable platform is located is used as the obstacle.
The control device of claim 60, wherein the processor is further configured to perform the following operations:

According to the position information of the photographed target and the position information of the no-fly zone, a moving path of the movable platform to circumvent the obstacle and the no-fly zone is determined.
A control device for a movable platform, the movable platform includes a photographing device, wherein the control device includes:

memory for storing executable instructions;

A processor for executing the executable instructions stored in the memory to perform the following operations:

acquiring the first pixel area corresponding to the following target and the second pixel area corresponding to the background target in the image captured by the photographing device;

The posture of the photographing device is adjusted according to the relative positional relationship between the first pixel area and the second pixel area.
The control device of claim 62, wherein the processor is further configured to perform the following operations:

If the shape of the second pixel area is of the first type, the step of adjusting the posture of the photographing device according to the relative positional relationship between the first pixel area and the second pixel area is performed.
64. The control device of claim 63, wherein the first type comprises: a face type.
The control device according to claim 63 or 64, wherein the processor is further configured to perform the following operations:

If the relative positional relationship is the first positional relationship, adjusting the posture of the photographing device based on the first posture adjustment strategy;

If the relative positional relationship is the second positional relationship, the posture of the photographing device is adjusted based on a second posture adjustment strategy; the second posture adjustment strategy is different from the first posture adjustment strategy.
The control device of claim 65, wherein:

The first positional relationship is not overlapping, and the first attitude adjustment strategy includes:

centering the image between the first pixel area and the second pixel area;

The second positional relationship is that the first pixel area is located in the second pixel area, and the second posture adjustment strategy includes:

The first pixel area is located at a specified position in the image; and/or the distance between the edge pixel point of the second pixel area and the edge of the image is smaller than a preset distance threshold.
The control device of claim 62, wherein:

If the shape of the second pixel area is of the second type, the posture of the photographing device is adjusted according to the positions of the first pixel area and the second pixel area.
68. The control device of claim 67, wherein the second type comprises: a linear type.
The control device of claim 68, wherein the processor is further configured to perform the following operations:

The second pixel area is caused to pass through a designated position in the image, and the first pixel area is coincident with the designated position in at least one direction of the image.
A control device for a movable platform, characterized in that the control device comprises:

memory for storing executable instructions;

A processor for executing the executable instructions stored in the memory to perform the following operations:

obtaining the location information of obstacles in the environment where the movable platform is located;

Obtain the location information of the no-fly zone in the environment where the movable platform is located;

According to the position information of the obstacle and the position information of the no-fly zone, a moving path of the movable platform around the obstacle and the no-fly zone is determined.
The control device of claim 70, wherein the processor is further configured to perform the following operations:

establishing a loss function based at least in part on the no-fly zone location information and the obstacle location information;

The loss function is minimized to determine a path of movement of the movable platform around the obstacle and the no-fly zone.
The control device of claim 71, wherein the loss function includes a fifth loss function, and the greater the distance between the waypoint on the moving path and the obstacle and the no-fly zone, the greater the The value of the fifth loss function is smaller.
The control device according to claim 71, wherein the loss function includes a second loss function, and the smaller the curvature and/or the rate of change of the curvature of the moving path, the higher the value of the second loss function. small.
The control device according to any one of claims 70 to 73, wherein the no-fly zone is determined by electronic fence and flight control information.
The control device according to any one of claims 70 to 74, wherein the processor is further configured to perform the following operations:

The location information of the no-fly zone is obtained from at least one of the remote devices local to the movable platform and communicable with the movable platform.
The control device according to any one of claims 70 to 74, wherein the processor is further configured to perform the following operations:

The location information of the obstacle is measured by sensors of the movable platform; and/or the location information of the obstacle is obtained from a remote device that can communicate with the movable platform.
A computer-readable storage medium, characterized in that it stores executable instructions, and when executed by one or more processors, the executable instructions can cause the one or more processors to execute as claimed in claim 1 The control method of any one of claims 38 to 38.
A movable platform, characterized in that it comprises: a photographing device and a movable carrier; the movable carrier comprises: the control device according to any one of claims 39-76.
The movable platform of claim 78, wherein the movable carrier comprises: an unmanned aerial vehicle, an unmanned vehicle, an unmanned ship or a robot.