WO2021217403A1

WO2021217403A1 - Method and apparatus for controlling movable platform, and device and storage medium

Info

Publication number: WO2021217403A1
Application number: PCT/CN2020/087423
Authority: WO
Inventors: 刘洁; 周游; 陈希
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-04-28
Filing date: 2020-04-28
Publication date: 2021-11-04
Also published as: CN113853559A

Abstract

A method and apparatus for controlling a movable platform, a device, and a storage medium. The method is used for controlling a movable platform to encircle a target object, the movable platform comprises a photographing apparatus, and the method comprises: acquiring first feature points in a target region of a reference image photographed by the photographing apparatus, wherein the reference image is a previous image frame of a current image, and the target region is an image region corresponding to the target object; determining second feature points corresponding to the first feature points within the current image; when the number of second feature points meets a preset condition, moving the target region of the reference image so as to obtain a target region of the current image; and controlling the photographing apparatus to be oriented towards three-dimensional space points corresponding to the target region of the current image. Thus, the photographing apparatus of the movable platform can continuously photograph the target object. Moreover, the described procedure is an image-based processing procedure that does not require a pre-established complicated three-dimensional model, the processing means is quick and simple, and the user experience is good.

Description

Control method, device, equipment and storage medium of movable platform

Technical field

The embodiments of the present application relate to the field of control, and in particular, to a control method, device, device, and storage medium of a movable platform.

Background technique

Surround shooting is a common shooting scheme. During the shooting process, the movable platform moves around the target, and the shooting device on the movable platform can be used to shoot the target during this process. If you need to achieve surround shooting, you not only need to control the movable platform to surround the target, but also adjust the orientation of the shooting device. When the operator manually completes this kind of shooting, he needs high operating skills.

In order to facilitate the operator to complete the surround shooting, the autonomous "point of interest surround" function came into being, referred to as POI (English full name: Point of Interest). The vision-based POI solution in the prior art is generally aimed at small targets that are easy to see the whole picture, but for larger targets, the effect is relatively poor. This is because the movable platform often only sees the larger targets. Part of the target, it is difficult to observe the full picture of the target, and the limitation is strong. A POI solution for a huge target is based on a pre-established three-dimensional model of the target, but this requires the operator to establish a three-dimensional model of the target before the surround shooting, which is cumbersome and has an unfriendly user experience.

Summary of the invention

The embodiments of the present application provide a control method, device, device, and storage medium for a movable platform. By updating the target area, the shooting device of the movable platform can always shoot the target object, which is conducive to surrounding shooting of a relatively large target.

The first aspect of the embodiments of the present application is to provide a method for controlling a movable platform, which is used to control the movable platform to surround a target object, and the movable platform includes a photographing device, including:

Acquiring the current image taken by the photographing device;

Acquiring a first feature point in a target area of a reference image, where the reference image is an image of a previous frame of the current image, and the target area is an image area corresponding to the target object;

Determining a second feature point corresponding to the first feature point in the current image;

When the number of the second feature points meets a preset condition, moving the target area of the reference image to obtain the target area of the current image;

Controlling the photographing device to face the three-dimensional space point corresponding to the target area of the current image.

A second aspect of the embodiments of the present application is to provide a control device for a movable platform, the control device is used to control the movable platform to surround a target object, the movable platform includes a photographing device, and the control device includes: Memory, processor;

The memory is used to store program code;

The processor calls the program code, and when the program code is executed, is used to perform the following operations:

Acquiring the current image taken by the photographing device;

The third aspect of the embodiments of the present application is to provide a movable platform, including:

body;

The power system is installed on the fuselage to provide power;

Camera

And the control device as described in the second aspect.

The fourth aspect of the embodiments of the present application is to provide a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the method as described in the first aspect.

The control method, device, device, and storage medium of the movable platform provided in this embodiment determine the location of the first feature point mapped to the next frame of image based on the first feature point in the target area of the previous frame of image, Then obtain the second feature point of the next frame of image; when the number of second feature points meets the preset condition, for example, when the number of second feature points is less than the number of first feature points, move the target of the previous frame of image Area, obtain the target area of the next frame of image, thereby update the target area of the image, and control the camera to face the three-dimensional space point corresponding to the updated target area; in turn, the camera of the movable platform can always capture the target, which is beneficial to Surround shooting of larger targets. In addition, the above process does not require the establishment of a three-dimensional model in advance, the processing method is fast and simple, and the user experience is better.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

FIG. 1 is a flowchart of a method for controlling a movable platform provided by an embodiment of the application;

Figure 2 is a schematic diagram of an application scenario provided by an embodiment of the application;

FIG. 3 is a first schematic diagram of feature points provided by an embodiment of this application;

FIG. 4 is a second schematic diagram of feature points provided by an embodiment of this application;

FIG. 5 is a flowchart of a method for controlling a movable platform according to another embodiment of the application;

FIG. 6 is a first schematic diagram of a grid area provided by an embodiment of this application;

FIG. 7 is a second schematic diagram of a grid area provided by an embodiment of this application;

FIG. 8 is a third schematic diagram of a grid area provided by an embodiment of this application;

FIG. 9 is a fourth schematic diagram of a grid area provided by an embodiment of this application;

FIG. 10 is a third schematic diagram of feature points provided by an embodiment of this application;

FIG. 11 is a schematic diagram of movement of a movable platform provided by an embodiment of the application;

FIG. 12 is a flowchart of a method for controlling a movable platform provided by another embodiment of this application;

FIG. 13 is a first schematic diagram of an initial image provided by an embodiment of this application;

FIG. 14 is a second schematic diagram of an initial image provided by an embodiment of this application;

FIG. 15 is a third schematic diagram of an initial image provided by an embodiment of this application;

FIG. 16 is a structural diagram of a control device for a movable platform provided by an embodiment of the application;

FIG. 17 is a structural diagram of a movable platform provided by an embodiment of the application.

Reference signs:

20: UAV;

21: Camera;

22: PTZ;

23: Wireless communication interface;

24: Control device;

31: target audience;

41: current image;

42: The target area of the reference image;

51, 52, 53: image;

160: control device;

161: memory;

162: processor;

163: communication interface;

170: UAV;

171: Motor;

172: Propeller;

173: Electronic governor;

174: Camera;

175: Sensing system;

176: Communication system;

177: Supporting equipment;

178: Control device.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

It should be noted that when a component is referred to as being "fixed to" another component, it can be directly on the other component or a centered component may also exist. When a component is considered to be "connected" to another component, it can be directly connected to the other component or there may be a centered component at the same time.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of this application. The terms used in the specification of the application herein are only for the purpose of describing specific embodiments, and are not intended to limit the application. The term "and/or" as used herein includes any and all combinations of one or more related listed items.

Hereinafter, some embodiments of the present application will be described in detail with reference to the accompanying drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

In the surround shooting technology, in order to facilitate the operator to complete the surround shooting, the autonomous "point of interest surround" function came into being. The existing vision-based POI solutions are aimed at small targets that are easy to see the whole picture. For larger targets, the effect is relatively poor. This is because the movable platform often only sees the target when it surrounds the larger target. Part of it is difficult to observe the full picture of the target and has strong limitations. A POI solution for large targets is to perform three-dimensional modeling of the large target before the movable platform surrounds the large target for shooting; then, based on the pre-established three-dimensional model of the target, the surround shooting is performed. However, this method requires the operator to establish a three-dimensional model of the target before the surround shooting, which is cumbersome and the algorithm is complicated; the user experience is not friendly.

The mobile platform control method, device, equipment and storage medium provided in the embodiments of the present application can solve the above-mentioned problems.

FIG. 1 is a flowchart of a method for controlling a movable platform provided by an embodiment of the application. The method for controlling a movable platform provided in this embodiment is used to control the movable platform to surround a target object, and the movable platform includes a camera. As shown in Figure 1, the method provided in this embodiment may include:

Step S101: Acquire a current image captured by the photographing device.

Exemplarily, the movable platform of this embodiment may specifically be an unmanned aerial vehicle, an unmanned ground robot, an unmanned ship, a mobile robot, and the like. For the convenience of explanation, the movable platform is used as a drone for schematic illustration. It is understandable that the drone in this application can be equally replaced with a movable platform.

FIG. 2 is a schematic diagram of an application scenario provided by an embodiment of the application; as shown in FIG. 2, the drone 20 is equipped with a photographing device 21, and the photographing device 21 may specifically be a camera, a video camera, or the like. Specifically, the camera 21 can be mounted on the drone 20 via a pan/tilt 22, or the camera 21 can be fixed on the drone 20 via other fixing devices. The camera 21 can take real-time shooting to obtain video data or image data, and send the video data or image data to the control device 24 through the wireless communication interface 23 of the drone 20. The control device 24 may specifically be a remote control corresponding to the drone 20, or a user terminal; among them, the user terminal may be a smart phone, a tablet computer, and the like. In addition, the drone 20 may also include a control device, and the control device may include a general-purpose or special-purpose processor. It should be noted that this is only a schematic description, and does not limit the specific structure of the UAV.

In the process of capturing images by the camera, the movable platform can obtain the current image captured by the camera in real time.

Step S102: Acquire the first feature point in the target area of the reference image, the reference image is the previous frame of the current image, and the target area is the image area corresponding to the target object.

Exemplarily, in the process of photographing the target object by the photographing device on the movable platform, the photographing device outputs the current image; the movable platform determines that the previous frame of the image is a reference image. The movable platform determines the target area of the reference image, which is the image area corresponding to the target object; then, extracts the first feature point in the image area (in order to facilitate the distinction, the feature point of the reference image is called the first Feature points). For example, the movable platform obtains the first frame of image output by the camera, determines the target area on the first frame of image, and extracts the first feature point on the target area of the first frame of image. Then the movable platform obtains the second frame image output by the shooting device, and it can be determined that the first frame image is the reference image of the second frame image.

Among them, the target object may be a large-scale target, for example, the target object is a building, or the target object is a group of buildings.

In an example, as shown in FIG. 2, the image captured by the photographing device 21 includes the target object 31 as shown in FIG. 2.

In an example, when extracting the first feature point of the reference image, a corner detection algorithm (full English name: Corner Detection Algorithm) may be used to detect the first feature point on the target area of the reference image. Among them, corner detection algorithms, for example, are FAST (English full name: Features from Accelerated Segment Test) algorithm, SUSAN (English full name: Small Univalue Segment Assimilating Nucleus) algorithm, and Harris operator algorithm.

Step S103: Determine a second feature point corresponding to the first feature point in the current image.

Exemplarily, during the movement of the movable platform, as the position of the movable platform changes, the area that can be photographed by the camera on the movable platform also changes, and the image output by the camera also changes. For the current image, it is necessary to determine the feature points on the current image (in order to facilitate the region, the feature points of the current image are referred to as second feature points).

After acquiring the first feature point of the reference image, it is necessary to determine the position of the first feature point in the current image, and then obtain the second feature point in the current image.

In an example, since the first feature point corresponds to a three-dimensional space point on the target object, the position of the three-dimensional space point on the current image can be determined according to the three-dimensional space point, and then the second feature point in the current image can be obtained. For example, map the three-dimensional space point corresponding to the first feature point to the current image, and then obtain the second feature point corresponding to the first feature point.

In another example, after the first feature point of the reference image is acquired, the tracking algorithm is used to track the first feature point in the target area to determine that the first feature point in the target area of the reference image is in the current image s position. The tracking algorithm, for example, is the KLT (full English name: Kanade-Lucas-Tomasi Feature Tracker) feature tracking algorithm.

For example, FIG. 3 is the first schematic diagram of the feature points provided by the embodiment of the application. As shown in FIG. 3, the reference image 40 has the target area 42 and the target area 42 has the target object 31; A feature point, the first feature point includes feature point A, feature point B, and feature point C, for example. During the process of the movable platform surrounding the target object, the camera outputs the current image 41. It can be seen that by using a tracking algorithm, the position of the first feature point (for example, feature point A, feature point B, and feature point C) in the target area 42 of the reference image 40 in the current image 41 can be determined. Wherein, the reference image 40 and the current image 41 may be adjacent images or non-adjacent images. FIG. 3 is only a schematic illustration, and does not limit the type of target object or the number of feature points.

For example, FIG. 4 is the second schematic diagram of the feature points provided by the embodiment of this application. As shown in FIG. 4, the movable platform moves from left to right (the direction shown by the arrow shown in FIG. 4). 31 represents the target object, 51 and 52 represent the images output by the camera during the process of moving the camera around the target object 31 in the direction shown by the arrow (from left to right). It can be understood that the three-dimensional points on the target 31 can be Mapped to the

images

51 and 52, the mapping point of the three-dimensional space point in the image 51 may specifically be a feature point on the target area of the image 51, and the mapping point of the three-dimensional space point in the image 52 may specifically be the target of the image 52 Feature points on the area.

Point A and point B are three-dimensional space points on the target object 31. The point a1 and the point b1 represent feature points in the image 51, the point a1 corresponds to the point A, and the point b1 corresponds to the point B. The point a2 and the point b2 represent feature points in the image 52, the point a2 corresponds to the point A, and the point b2 corresponds to the point B.

In the process of moving the movable platform from left to right, it is necessary to know the location of point A mapped to point a1 in image 51, and the location of point B mapped to feature point b1 in image 51; after the movable platform moves, point A is mapped to the position of the feature point a2 in the image 52, and the point B is mapped to the location of the point feature b2 in the target image 52; then it is learned that the feature point a1 is mapped to the feature point a2 in the image 52, and the feature point b1 is mapped to the image 52 The feature point in b2.

Step S104: When the number of second feature points meets the preset condition, move the target area of the reference image to obtain the target area of the current image.

Before moving the target area of the reference image, it is necessary to determine whether the number of second feature points in the current image meets a preset condition; for example, the preset condition is that the number of second feature points is less than the number of first feature points, that is The number of feature points is decreasing.

Step S105: Control the shooting device to face the three-dimensional space point corresponding to the target area of the current image.

Exemplarily, after step S104, the target area of the current image has been obtained. The target area of the current image is an image area. The image area may correspond to a three-dimensional space point on the target object, so that the camera can be controlled to face the current image. The target area corresponds to the three-dimensional space point, so that the shooting device can always shoot the target object.

Moreover, step S101 to step S105 may be a process of repeated processing.

In this embodiment, by providing a method for controlling the movable platform to surround the target object, the current image captured by the photographing device is obtained; the first feature point in the target area of the reference image is obtained, and the reference image is the previous frame image of the current image , The target area is the image area corresponding to the target object; determine the second feature point corresponding to the first feature point in the current image; when the number of second feature points meets the preset condition, move the target area of the reference image to obtain the current image The target area; control the camera to face the three-dimensional space point corresponding to the target area of the current image. Based on the first feature point in the target area of the previous frame of image, the position where the first feature point is mapped to the next frame of image can be determined, and then the second feature point of the next frame of image can be obtained; When the number satisfies the preset condition, for example, when the number of second feature points is small, the target area of the reference image is moved to obtain the target area of the next frame of image. Thus, based on the distribution of feature points, the target area of the image is updated. The shooting device is controlled to face the three-dimensional space point corresponding to the target area of the current image; furthermore, the shooting device of the movable platform can always shoot the target object. In addition, the above process is an image-based processing process, no complicated three-dimensional model needs to be established in advance, the processing method is fast and simple, and the user experience is better.

FIG. 5 is a flowchart of a method for controlling a movable platform provided by another embodiment of the application. The method provided in this embodiment is used to control the movable platform to surround the target object, and the movable platform includes a camera. As shown in Figure 5, the method provided in this embodiment may include:

Step S201: Acquire a current image captured by the photographing device.

For the application scenario of this embodiment, refer to FIG. 2.

For this step, refer to step S101 in FIG. 1 and will not be repeated here.

Step S202: Divide the target area of the reference image into multiple grid areas.

In an example, step S202 specifically includes: dividing the target area of the reference image into multiple grid areas according to the direction in which the movable platform surrounds the target object.

Exemplarily, before extracting the first feature point of the target area of the reference image, in order to reduce the algorithm complexity and the amount of data calculation, calculation is not performed for every pixel in the target area of the reference image. The target area of the reference image may be divided into a plurality of grid areas; then, in step S203, feature points less than a preset number are extracted from each grid area.

When dividing the grid area, the grid area may be divided according to the direction in which the movable platform surrounds the target object, so as to divide the target area of the reference image into multiple grid areas.

For example, FIG. 6 is a first schematic diagram of a grid area provided by an embodiment of the application. As shown in FIG. 6, a movable platform surrounds a target object on a horizontal plane, and the target area can be divided into grids by horizontal lines to obtain multiple grids. Grid area.

For another example, if the movable platform surrounds the target object on a vertical plane, the grid can be divided by facing the target area vertically to obtain multiple grid areas.

For another example, FIG. 7 is the second schematic diagram of the grid area provided by the embodiment of the application. As shown in FIG. 7, the movable platform surrounds the target object on the inclined surface, and the determination is made according to the inclination direction indicated by the inclined surface. Corresponding slanted line; the target area is divided into grids according to the slanted line to obtain multiple grid areas. The tilt direction is not limited to the direction shown in FIG. 7.

Step S203: Acquire a preset number of feature points in each grid area as the first feature point in the target area of the reference image. The reference image is the previous frame of the current image, and the target area is the image area corresponding to the target object.

Exemplarily, in each grid area of the target area of the reference image, when the first feature point is extracted, the corner detection algorithm can be used to detect the feature point in each grid area, and limit the number of feature points in the detection process. Then, the preset number of feature points in each grid area are extracted, and the preset number of feature points in each grid area are used as the first feature point in the target area of the reference image.

Step S204: Determine a second feature point corresponding to the first feature point in the current image.

In an example, step S204 specifically includes:

Based on the first feature point, the tracking algorithm is used to obtain the feature point obtained by tracking in the current image; the feature point obtained by tracking is filtered based on the epipolar constraint condition to obtain the second feature point.

Exemplarily, referring to the introduction of step S103 shown in 1, the tracking algorithm is used to track the first feature point in the target area, and then the feature point obtained by tracking in the current image is obtained. However, due to the calculation error of the tracking algorithm itself or other factors, the feature points obtained by the tracking may be inaccurate, so the feature points obtained by the tracking need to be filtered to obtain the second feature point.

In an example, the feature points obtained by tracking can be filtered based on the epipolar constraint (English full name: Epipolar constraint) condition to obtain the second feature point. In another example, the feature points obtained by tracking may be filtered based on the motion relationship of the feature points of the two images before and after. It should be noted that the method of filtering the tracked feature points based on the epipolar constraint condition can be combined with the method of filtering the tracked feature points based on the movement relationship of the feature points of the two frames of image before and after, so as to obtain more accurate results. The second feature point.

Step S205: When the number of second feature points meets the preset condition, move the target area of the reference image to obtain the target area of the current image.

In an example, the preset condition is that the number of first feature points located in the grid area on the border among the multiple grid areas of the reference image corresponds to zero in the current image. At this time, step S205 specifically includes: moving the target area of the reference image in a direction away from the grid area of the boundary.

In another example, the preset condition is that the number of second feature points is less than the number of first feature points.

Exemplarily, during the process of moving the movable platform around the target object, the position of the movable platform relative to the target object is constantly changing. For example, when the drone is flying around, the multiple grid areas of the reference image The first feature point in the grid area at the boundary of the middle may be lost, that is, the corresponding second feature point cannot be found in the current image. Therefore, the target area of the reference image needs to be moved to obtain the target area of the current image .

As shown in FIG. 8, FIG. 8 is the third schematic diagram of the grid area provided by the embodiment of the application. The diagram a in FIG. 8 is each grid area of the target area of the reference image, and the diagram a includes a plurality of first feature points. ; Figure b in Figure 8 is the target area of the current image. The target area of the reference image is divided into multiple grid areas; the first feature point is extracted from each grid area. In the process of moving the movable platform around the target object, the first feature point (the solid point in Figure a) on the left boundary of Figure a in Figure 8 may be lost; thus, in the grid area determined to be located on the boundary When the number of corresponding second feature points in the current image is zero or less than a certain number, the target area of the reference image is moved to obtain the target area of the current image as shown in b in FIG. 8. At this time, it is to move the target area of the reference image in the direction away from the grid area of the boundary; where the moving distance can be the width of the grid area occupied by the missing feature points, for example, Figure a in Figure 8 As shown, the missing feature points in the grid area occupy a row of grids, and relative to the target area of the reference image, move the distance represented by the "column of grids".

In an example, if the first feature point of the grid area of the left border in the reference image is lost (that is, the number of second feature points corresponding to the first feature point in the current image is zero), then the target of the reference image Area, moving toward the right edge of the image.

Or, if the first feature point of the grid area of the right boundary in the reference image is lost (that is, the number of second feature points corresponding to the first feature point in the current image is zero), then the target area of the reference image is Move towards the left edge of the image.

Or, if the first feature point of the grid area of the upper boundary in the reference image is lost (that is, the number of second feature points corresponding to the first feature point in the current image is zero), then the target area of the reference image is changed to Move towards the lower border of the image.

Or, if the first feature point of the grid area of the upper left boundary in the reference image is lost (that is, the number of second feature points corresponding to the first feature point in the current image is zero), then the target area of the reference image is Move towards the lower right border of the image.

Step S206: Acquire a third feature point in the target area of the current image, where the number of third feature points is greater than or equal to the number of second feature points.

Exemplarily, after step S205, the target area of the current image includes at least one second feature point, and the second feature point is a feature point corresponding to the first feature point of the reference image. Further, candidate feature points of the target area of the current image can be extracted. If no candidate feature points are extracted, the above-mentioned second feature points are determined to be all feature points (that is, third feature points) in the target area of the current image. At this time, the number of third feature points is equal to the number of second feature points. If the candidate feature points are extracted, the above-mentioned second feature point and the newly extracted candidate feature points are used as all the feature points (ie, the third feature points) in the target area of the current image. At this time, the number of third feature points is greater than the number of second feature points.

9 is a schematic diagram of the grid area provided by an embodiment of the application. As shown in FIG. 9, the graph a in FIG. 9 is each grid area of the target area of the reference image, and the graph a includes a plurality of first feature points. ; Figure b in Figure 9 is the target area of the current image. As shown in Figure a in Figure 9, the first feature point (the solid point in Figure a) on the left boundary of Figure a in Figure 9 is lost in the current image; the target area will be moved one column to the right. Grid; Then for the target area of the current image, new candidate feature points (the solid points in Figure b) are extracted.

In an example, step 206 specifically includes: obtaining candidate feature points in the target area of the current image; filtering the candidate feature points according to the depth value of the candidate feature points and/or the semantic information of the candidate feature points to determine The third feature point.

The candidate feature points may have feature points that do not belong to the target object, that is, the three-dimensional space points corresponding to the candidate feature points are not three-dimensional space points on the target object, but other targets. At this time, the candidate feature points need to be filtered. After filtering, all candidate feature points may be filtered out. At this time, the number of third feature points is equal to the number of second feature points. After filtering, part of the candidate feature points may be filtered out. At this time, the number of third feature points is greater than the number of second feature points.

When filtering the candidate feature points, the following implementation manners are included.

In one embodiment, the depth value of the candidate feature point can be calculated; where the depth value represents the distance between the three-dimensional space point corresponding to the feature point and the reference point (for example, the optical center) of the imaging device. If the depth value of the candidate feature point belongs to the preset depth value range, the candidate feature point is determined to be the feature point in the target area of the current image, and the feature point does not need to be filtered; if the depth value of the candidate feature point does not belong to the preset depth value range, Set the depth value range to determine to filter the feature point. Among them, the "preset depth value range" can be an empirical value; the "preset depth value range" represents the value range of the depth value of the target area, or the "preset depth value range" represents the three-dimensional value on the target object. The depth value range of the feature point corresponding to the spatial point.

In another implementation manner, the semantic information of the candidate feature points can be obtained; wherein the semantic information represents the category of the three-dimensional space point corresponding to the feature point, for example, buildings, sky, and grass. Then, when it is determined that the semantic information of the candidate feature point corresponds to the target object, it is determined that the candidate feature point is a feature point in the target area of the current image, and the feature point does not need to be filtered. When it is determined that the semantic information of the candidate feature point does not correspond to the target object, the feature point is filtered. For example, if the target object is a building, but the semantic information of a candidate feature point indicates that the candidate feature point corresponds to the grass or the sky behind the building, the feature point needs to be filtered out.

It should be noted that when the candidate feature points are filtered, the depth value and semantic information of the candidate feature points can be combined to determine whether to filter the candidate feature points. For example, if the depth value of the candidate feature point belongs to the preset depth value range, and the semantic information of the candidate feature point corresponds to the target object, the candidate feature point is determined to be the feature point in the target area of the current image. The feature point needs to be filtered. Otherwise, filter the feature point.

For example, FIG. 10 is the third schematic diagram of the feature points provided by the embodiment of the application. As shown in FIG. 10, the movable platform moves from left to right (the direction shown by the arrow shown in FIG. 10). 31 represents the target object, 51, 52, 53 represent the images output by the camera during the process of moving around the target object 31 in the direction indicated by the arrow (from left to right). It can be understood that the three-dimensional space on the target 31 The points can be mapped to the

images

51, 52, and 53, the mapping points of the three-dimensional space points in the image 51 may specifically be feature points on the target area of the image 51, and the mapping points of the three-dimensional spatial points in the image 52 may specifically be A feature point on the target area of the image 52, and the mapping point of the three-dimensional space point in the image 53 may specifically be a feature point on the target area of the image 53.

Point A, point B, and point C are three-dimensional space points on the target object 31. The point a1 and the point b1 represent feature points in the image 51, the point a1 corresponds to the point A, and the point b1 corresponds to the point B. Point a2, point b2, and point c2 represent feature points in image 52, point a2 corresponds to point A, point b2 corresponds to point B, and point c2 corresponds to point C. Point a3, point b3, and point c3 represent feature points in the image 53, point a3 corresponds to point A, point b3 corresponds to point B, and point c3 corresponds to point C.

In the process of moving the movable platform from left to right, it is necessary to know the location of the point A mapped to the point a1 in the image 51 and the location of the point B mapped to the feature point b1 in the image 51. After the movable platform moves, point A is mapped to the position of the feature point a2 in the image 52, and point B is mapped to the location of the point feature b2 in the target image 52; then it is learned that the feature point a1 is mapped to the feature point a2 in the image 52 , The feature point b1 is mapped to the feature point b2 in the image 52, and a new feature point c2 is added to the target area of the image 52 (the new feature point obtained in this step S206 is adopted). After the movable platform moves again, point A is mapped to the position of the feature point a2 in the image 52, point B is mapped to the location of the point feature b2 in the target image 52, and point C is mapped to the location of the point feature c2 in the target image 52 Position; and then know that the feature point a2 is mapped to the feature point a3 in the image 53, the feature point b2 is mapped to the feature point b3 in the image 53, and the feature point c2 is mapped to the feature point c3 in the image 53. By incorporating new feature points, the camera can always capture the surface of the target object 31, ensuring that the target object will not be lost during the surrounding process.

Step S207: Determine a three-dimensional space point corresponding to the target area of the current image according to the third feature point in the target area of the current image.

In an example, step S207 specifically includes: weighted average of the depth value of the third feature point in the target area of the current image; according to the weighted average and the shooting direction of the current image taken by the shooting device, it is determined that the target area of the current image corresponds to Points in three-dimensional space.

In an example, the weight corresponding to the third feature point close to the center of the target area of the current image is greater than the weight corresponding to the third feature point far away from the center of the target area of the current image.

Exemplarily, during the movement of the movable platform, the orientation of the camera on the movable platform needs to be adjusted.

After step S206, the target area of the current image is obtained, and if the number of third feature points in the target area of the current image is not zero, the camera can be controlled to face the target area of the current image according to the target area of the current image Corresponding three-dimensional space point.

In one embodiment, there are multiple third feature points in the target area of the current image, and the depth value of each third feature point can be calculated; where the depth value represents the three-dimensional space corresponding to the feature point The distance between the point and the reference point (such as the optical center) of the imaging device; then the depth value of each third feature point in the target area of the current image can be calculated by weighted average to obtain the weighted average. At the same time, the movable platform can learn the shooting direction when the camera is shooting the current image. Furthermore, the movable platform determines the three-dimensional space point corresponding to the target area of the current image based on the above-mentioned weighted average value and the shooting direction. Among them, because the third feature point close to the center of the target area of the current image is adjacent to the center of the target area of the current image; thus, the weight corresponding to the third feature point close to the center of the target area of the current image is greater than that far away from the current image. The weight of the third feature point corresponding to the center of the target area.

In another implementation manner, a third feature point close to the center of the target area of the current image can be selected. Furthermore, the movable platform determines the three-dimensional space point corresponding to the target area of the current image based on the depth value and the shooting direction of the third feature point close to the center of the target area of the current image.

Step S208: Control the shooting device to face the three-dimensional space point corresponding to the target area of the current image.

By controlling the camera to face the three-dimensional point corresponding to the target area of the current image, the camera of the movable platform can observe the surface of the target object corresponding to the target area.

For example, FIG. 11 is a schematic diagram of the movement of the movable platform provided by an embodiment of the application. As shown in FIG. 11, the movable platform is, for example, a drone; when the drone is at the starting point A, the target of the image collected The area corresponds to the area a of the target object in three-dimensional space; during the flight to point B, the target area is continuously updated, and then when the drone is at point B, the target area of the collected image corresponds to the target object in the three-dimensional space的区b.

In an example, step S208 specifically includes: when the number of third feature points in the target area of the current image is greater than a preset threshold, controlling the camera to face the three-dimensional space point corresponding to the target area of the current image.

Exemplarily, it has been determined in step S207 that the number of third feature points in the target area of the current image is multiple. In this step, the number of third feature points can be further determined; when the number of third feature points is greater than the preset threshold, it is determined that the number of feature points in the target area of the current image is large, and you can Continuously update the target area. Furthermore, it is determined that the shooting device of the movable platform can face the three-dimensional space point corresponding to the target area of the current image determined in step 207.

Step S209: When the number of third feature points in the target area of the current image is less than or equal to the preset threshold, the camera is controlled to face the three-dimensional space point corresponding to the target area of the reference image.

Exemplarily, after step S206, if the number of third feature points is less than or equal to the preset threshold, it is determined that the number of feature points in the target area of the current image is small. At this time, according to the reference image (ie , The feature points of the target area of the previous frame of the current image, determine the three-dimensional space point corresponding to the target area of the reference image, and then control the shooting device toward the three-dimensional space point corresponding to the target area of the reference image.

Exemplarily, when the number of feature points in the target area of the current image is small, the movable platform can also be controlled to return home; or, the shooting mode of the shooting device of the movable platform can be switched, for example, step S201- can no longer be used. The shooting mode of the S208 large target object needs to be switched to another shooting mode.

Or, when the number of third feature points in the target area of the current image is less than or equal to the preset threshold, and the number of third feature points in the target area of subsequent frames of images is less than or equal to the preset threshold, the shooting is controlled The device faces the three-dimensional space point corresponding to the target area of the reference image.

In this embodiment, on the basis of the above-mentioned embodiment, when extracting the first feature points of the target area of the reference image, the feature points that are less than the preset number in the grid area of the reference image are extracted, and the reference image is not targeted. Each pixel or feature point in the target area of the image is calculated, thereby reducing the complexity of the algorithm and the amount of data calculation. According to the first feature point of the target area of the reference image, the second feature point corresponding to the first feature point in the current image is obtained, and the target area of the reference image is moved to obtain the target area of the current image; thereby the target area is updated. In addition, analyze the current image, enrich the feature points in the target area of the current image, and adjust the orientation of the camera on the movable platform according to the feature points in the target area of the current image, and control the camera to face the target of the current image The three-dimensional space point corresponding to the area, and further, the camera of the movable platform can always observe the target object and shoot the surface corresponding to the target area; at the same time, it provides the exit logic, that is, the feature points in the target area of the current image are more In a short period of time, the camera is controlled to face the three-dimensional space point corresponding to the target area of the reference image, or the movable platform is controlled to return home to end the current shooting, or the camera of the movable platform can continue to perform shooting tasks in other shooting modes.

FIG. 12 is a flowchart of a method for controlling a movable platform provided by another embodiment of the application. The method provided in this embodiment is used to control the movable platform to surround the target object, and the movable platform includes a photographing device. As shown in Figure 12, the method provided in this embodiment may include:

Step S301: Send the initial image captured by the photographing device to the control device of the movable platform, so that the control device displays the initial image.

For the application scenario of this embodiment, refer to FIG. 2.

The initial image captured by the photographing device is sent to the control device of the movable platform, so that the control device displays the initial image.

Before the movable platform moves around, an initial image is captured by the camera. Then the movable platform sends the initial image to the control device of the movable platform, and then the control device displays the initial image, and the user can view the initial image. Wherein, the control device may specifically be a remote controller corresponding to the movable platform, or a user terminal; where the user terminal is, for example, a smart phone, a tablet computer, and the like.

Step S302: Obtain the user's instruction information for the target object. The instruction information is generated according to the user's click operation or frame selection operation on the initial image displayed by the control device.

The user can input instruction information into the control device by means of touch, or gesture control, or voice information; the instruction information is used to indicate the target object in the initial image.

For example, the user clicks a location point of the initial image through an operating medium (for example, a finger or a stylus); and then the control device receives a click operation input by the user.

Or, the user uses an operating medium (for example, a finger or a stylus) to frame select an area of the initial image; and then the control device receives the frame selection operation input by the user.

For example, FIG. 13 is a first schematic diagram of a reference image provided by an embodiment of the application. As shown in FIG. 13, the user selects an area on the initial image by using a finger frame.

Step S303: Determine the target area of the initial image according to the instruction information.

In an example, step S303 specifically includes the following steps:

The first step is to perform image segmentation on the initial image to obtain multiple segmented regions.

The second step is to determine the target area of the initial image according to the segmented area and the instruction information.

Wherein, the second step specifically includes: when the proportion of the image area in the initial image indicated by the indication information in the target segmentation area is greater than the preset ratio, then determining the target area of the initial image according to the target segmentation area, and the target segmentation area It is at least one of a plurality of divided regions.

In one example, image segmentation is performed on the initial image, for example, image segmentation processing based on clustering is performed to obtain a segmentation result; wherein the segmentation result includes multiple segmented regions, and pixels in each segmented region have similar features.

Then, the ratio value of the image area indicated by the instruction information (ie, the image area in the initial image) on the target segmented area is calculated, where the target segmented area is at least one of the plurality of segmented areas. If it is determined that the ratio value is greater than the preset ratio, the target area of the initial image can be determined according to the target segmentation area to ensure that the target area contains the complete target object.

For example, the image area indicated by the indication information is A1; after image segmentation is performed on the initial image, segmentation areas B1, B2, B3, and B4 are obtained. If it is determined that the proportion of the image area A1 in the segmented area B1 is greater than the preset ratio, the target area of the initial image can be determined according to the segmented area B1.

For example, FIG. 14 is the second schematic diagram of the reference image provided by the embodiment of the application. As shown in FIG. 14, the target object selected by the user frame is a small house; image segmentation is performed on the entire image, and the segmented image is A number of divided areas are divided; further, the small house in Figure a in Figure 14 is divided into one divided area. Then, according to the user's frame selection and the result of image segmentation, all the image information of the small house can be placed in the target area, as shown in Figure 15.

Step S304: Acquire the current image captured by the camera.

Exemplarily, this step may refer to the embodiment shown in FIG. 1 and FIG. 5, and details are not described herein again.

Step S305: Acquire the first feature point in the target area of the reference image, the reference image is the previous frame of the current image, and the target area is the image area corresponding to the target object.

Step S306: Determine a second feature point corresponding to the first feature point in the current image.

Step S307: Identify the target type of the target object; if the target type is a preset type, execute the step of moving the target area of the reference image to obtain the target area of the current image.

Exemplarily, before moving the target area of the reference image, it is necessary to identify whether the target object in the target area is a preset target type. For example, it is necessary to identify whether the target object in the target area is a large building.

If yes, step S308 is executed. If not, there is no need to perform step S308, and the camera can be controlled to face the three-dimensional space point corresponding to the target area of the reference image.

Step S308: When the number of second feature points meets the preset condition, move the target area of the reference image to obtain the target area of the current image.

Step S309: Control the shooting device to face the three-dimensional space point corresponding to the target area of the current image.

Step S310: Control the movement of the movable platform so that the distance between the shooting device and the three-dimensional space point corresponding to the target area of the current image is the surrounding radius.

For example, in step S309, the orientation of the camera is controlled. Since the camera is carried on the movable platform, it is also necessary to control the movement of the movable platform.

For example, take the drone as an example; if the distance value calculated based on the third feature point in the target area is less than the surrounding radius, the drone is controlled to fly backward; “backward” refers to the target and the drone The first direction of the line between the two, the first direction is the direction in which the target object points to the drone. If the distance value is greater than the surrounding radius, control the drone to fly forward; "forward" refers to the second direction of the line between the target and the drone, the first direction is that the drone points to the target object Direction. In this way, the distance between the drone and the surface of the target object can always be equal to the surrounding radius through the control of the drone. Wherein, the surrounding radius may be input by the user through the control device, or the distance between the movable platform and the three-dimensional space point corresponding to the target area of the initial image may be used as the surrounding radius.

In the above control process, the distance between the shooting device and the three-dimensional space point corresponding to the target area of the current image is always the above-mentioned surrounding radius. That is to say, when the movable platform surrounds the building group to shoot, not only the surface of the building group can always be photographed, but also a certain distance can be kept from the surface of the building group.

In another example, the three-dimensional space point corresponding to the target area of the initial image may be used as the center point of the movable platform surrounding the target object.

Since the target area of the initial image corresponds to a three-dimensional space point (for example, the three-dimensional space point corresponding to the weighted average of each feature point in the target area of the initial image; or, the center position of the target area of the initial image corresponds to 3D space point), the 3D space point corresponding to the target area of the initial image can be used as the center point around the movable platform. At this time, as the movable platform moves, the center point of the surround is unchanged. That is to say, when the movable platform is shooting around the building group, the camera always shoots the surface of the building group. However, the center of the orbiting track of the movable platform is always the target area of the initial image corresponding to a three-dimensional space point.

16 is a structural diagram of a control device for a movable platform provided by an embodiment of the application. The control device is used to control the movable platform to surround the target object. The movable platform includes a photographing device 160. As shown in 16, the control device 160 includes: a memory 161, the processor 162;

The memory 162 is used to store program codes;

The processor 162 calls the program code. When the program code is executed, it is used to perform the following operations: obtain the current image captured by the camera; obtain the first feature point in the target area of the reference image, the reference image being the front of the current image One frame of image, the target area is the image area corresponding to the target object; the second feature point corresponding to the first feature point in the current image is determined; when the number of second feature points meets the preset condition, the target area of the reference image is moved to Obtain the target area of the current image; control the shooting device to face the three-dimensional space point corresponding to the target area of the current image.

In an example, before acquiring the first feature point in the target area of the reference image, the processor is further configured to: divide the target area of the reference image into multiple grid areas.

In an example, when the processor divides the target area of the reference image into multiple grid areas, it is specifically configured to: divide the target area of the reference image into multiple grid areas according to the direction in which the movable platform surrounds the target object.

In an example, when the processor obtains the first feature points in the target area of the reference image, it is specifically configured to: obtain a preset number of feature points in each grid area as the first feature points.

In an example, the preset condition includes: the number of first feature points in the grid area on the border among the multiple grid areas of the reference image corresponding to the second feature points in the current image is zero; the processor moves the reference image When the target area is specifically used to: move the target area of the reference image in a direction away from the grid area of the boundary.

In an example, the number of second feature points meets the preset condition, including: the number of second feature points is less than the number of first feature points.

In one example, when the processor determines the second feature point corresponding to the first feature point in the current image, it is specifically used to: based on the first feature point, use a tracking algorithm to obtain the feature points tracked in the current image; and based on the epipolar constraint The condition filters the tracked feature points to obtain the second feature point.

In an example, before the processor controls the camera to face the three-dimensional space point corresponding to the target area of the current image, it is also used to: obtain the third feature point in the target area of the current image, and the number of third feature points is greater than or equal to the second The number of feature points.

In an example, when the processor controls the camera to face the three-dimensional space point corresponding to the target area of the current image, it is specifically used to: determine the three-dimensional space point corresponding to the target area of the current image according to the third feature point in the target area of the current image ; Control the camera to face the three-dimensional point corresponding to the target area of the current image.

In an example, when the processor determines the three-dimensional space point corresponding to the target area of the current image according to the third characteristic point in the target area of the current image, it is specifically used to: determine the depth of the third characteristic point in the target area of the current image The values are weighted and averaged; according to the weighted average and the shooting direction of the shooting device when shooting the current image, the three-dimensional space point corresponding to the target area of the current image is determined.

In an example, the weight corresponding to the third feature point close to the center of the target area of the current image is greater than the weight corresponding to the third feature point far from the center of the target area of the current image.

In an example, when the processor obtains the third feature point in the target area of the current image, it is specifically used to:

Obtain candidate feature points in the target area of the current image; filter the candidate feature points according to the depth value of the candidate feature points and/or the semantic information of the candidate feature points to determine the third feature point.

In an example, when the processor controls the camera to face the three-dimensional space point corresponding to the target area of the current image, it is specifically used to: when the number of third feature points in the target area of the current image is greater than the preset threshold, control the camera to face forward The three-dimensional point corresponding to the target area of the image.

In an example, the processor is further configured to: when the number of third feature points in the target area of the current image is less than or equal to the preset threshold, control the camera to face the three-dimensional space point corresponding to the target area of the reference image.

In an example, the processor is further configured to: control the movement of the movable platform so that the distance between the shooting device and the three-dimensional space point corresponding to the target area of the current image is the surrounding radius.

In an example, before the processor moves the target area of the reference image to obtain the target area of the current image, it is also used to: identify the target type of the target object; if the target type is a preset type, execute moving the target area of the reference image to obtain The step of the target area of the current image.

In an example, the control device 160 further includes a communication interface 163, which is connected to the processor. The processor is also used for:

The initial image taken by the camera is sent to the control device of the movable platform, so that the control device displays the initial image; the user's instruction information to the target object is obtained through the communication interface 163, and the instruction information is based on the user's initial image displayed on the control device According to the instruction information, determine the target area of the initial image.

In an example, when the processor determines the target area of the initial image according to the instruction information, it is specifically configured to: perform image segmentation on the initial image to obtain multiple segmented areas; and determine the target area of the initial image according to the segmented areas and the instruction information.

In one example, when the processor determines the target area of the initial image according to the segmented area and the instruction information, it is specifically used to: when the proportion of the image area in the initial image indicated by the instruction information in the target segmented area is greater than the preset ratio , The target area of the initial image is determined according to the target segmentation area, and the target segmentation area is at least one of the plurality of segmentation areas.

In one example, the three-dimensional space point corresponding to the target area of the initial image is taken as the center point of the movable platform surrounding the target object, and the distance between the movable platform and the three-dimensional space point corresponding to the target area of the initial image is taken as the surrounding radius.

The specific principles and implementation manners of the control device for the movable platform provided in the embodiments of the present application are similar to the foregoing embodiments, and will not be repeated here.

In this embodiment, based on the first feature point in the target area of the previous frame of image, the position where the first feature point is mapped to the next frame of image is determined, and then the second feature point of the next frame of image is obtained; When the number of the second feature points meets the preset condition, for example, when the number of the second feature points is small, the target area of the reference image is moved to obtain the target area of the next frame of image. Thus, based on the distribution of feature points, the target area of the image is updated. The shooting device is controlled to face the three-dimensional space point corresponding to the target area of the image; furthermore, the shooting device of the movable platform can always shoot the target object. In addition, the above process is an image-based processing process, no complicated three-dimensional model needs to be established in advance, the processing method is fast and simple, and the user experience is better.

The embodiment of the present application provides a movable platform, and the movable platform may specifically be an unmanned aerial vehicle. FIG. 17 is a structural diagram of a movable platform provided by an embodiment of the application. As shown in FIG. 17, the movable platform 170 includes a body, a power system, a camera 174, and a control device 178. The power system includes at least one of the following: The motor 171, the propeller 172 and the electronic governor 173, the power system is installed on the fuselage to provide power; the specific principle and implementation of the control device 178 are similar to the foregoing embodiment, and will not be repeated here.

In addition, as shown in FIG. 17, the movable platform 170 also includes: a sensing system 175, a communication system 176, and a supporting device 177. The supporting device 177 may specifically be a pan/tilt, and the camera 174 is mounted on the movable platform through the supporting device 177. On platform 170.

In some embodiments, the control device 178 may specifically be a flight controller of the movable platform 170.

The specific principles and implementation manners of the movable platform provided in the embodiments of the present application are similar to the foregoing embodiments, and will not be repeated here.

The embodiments of the present application also provide a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to realize the control device of the movable platform as described above.

In the several embodiments provided in this application, it should be understood that the disclosed device and method may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware, or may be implemented in the form of hardware plus software functional units.

The above-mentioned integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The above-mentioned software functional unit is stored in a storage medium, and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute the method described in each embodiment of the present application. Part of the steps. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code .

Those skilled in the art can clearly understand that for the convenience and conciseness of the description, only the division of the above-mentioned functional modules is used as an example. In practical applications, the above-mentioned functions can be allocated by different functional modules as required, that is, the device The internal structure is divided into different functional modules to complete all or part of the functions described above. For the specific working process of the device described above, reference may be made to the corresponding process in the foregoing method embodiment, which is not repeated here.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application, not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or equivalently replace some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present application. Scope.

Claims

A control method of a movable platform, which is used to control the movable platform to surround a target object, the movable platform includes a photographing device, and is characterized in that it includes:

Acquiring the current image taken by the photographing device;

Acquiring a first feature point in a target area of a reference image, where the reference image is an image of a previous frame of the current image, and the target area is an image area corresponding to the target object;

Determining a second feature point corresponding to the first feature point in the current image;

When the number of the second feature points meets a preset condition, moving the target area of the reference image to obtain the target area of the current image;

Controlling the photographing device to face the three-dimensional space point corresponding to the target area of the current image.
The method according to claim 1, wherein before said acquiring the first feature point in the target area of the reference image, the method further comprises:

The target area of the reference image is divided into a plurality of grid areas.
The method according to claim 2, wherein the dividing the target area of the reference image into multiple grid areas comprises:

According to the direction in which the movable platform surrounds the target object, the target area of the reference image is divided into a plurality of grid areas.
The method according to claim 2, wherein said obtaining the first feature point in the target area of the reference image comprises:

A preset number of feature points are acquired in each grid area as the first feature points.
The method according to claim 2, wherein the preset condition comprises: the first feature point located in the grid area of the boundary among the multiple grid areas of the reference image is in the current image The number of corresponding second feature points in is zero;

The moving the target area of the reference image includes:

Move the target area of the reference image in a direction away from the grid area of the boundary.
The method according to claim 1, wherein the number of the second feature points meets a preset condition, comprising:

The number of the second feature points is less than the number of the first feature points.
The method according to claim 1, wherein the determining a second feature point corresponding to the first feature point in the current image comprises:

Based on the first feature point, using a tracking algorithm to obtain the feature point obtained by tracking in the current image;

Filtering the feature points obtained by the tracking based on the epipolar constraint condition to obtain the second feature point.
The method according to claim 1, wherein before the controlling the photographing device to face the three-dimensional space point corresponding to the target area of the current image, the method further comprises:

Acquire a third feature point in the target area of the current image, where the number of the third feature points is greater than or equal to the number of the second feature points.
The method according to claim 8, wherein the controlling the photographing device to face the three-dimensional space point corresponding to the target area of the current image comprises:

Determine the three-dimensional space point corresponding to the target area of the current image according to the third feature point in the target area of the current image;

Controlling the photographing device to face the three-dimensional space point corresponding to the target area of the current image.
The method according to claim 9, wherein the determining a three-dimensional space point corresponding to the target area of the current image according to a third characteristic point in the target area of the current image comprises:

Performing a weighted average on the depth value of the third feature point in the target area of the current image;

Determine the three-dimensional space point corresponding to the target area of the current image according to the weighted average value and the shooting direction when the shooting device shoots the current image.
The method according to claim 10, wherein a weight corresponding to a third feature point close to the center of the target area of the current image is greater than a weight corresponding to a third feature point far from the center of the target area of the current image .
The method according to claim 8, wherein said acquiring a third characteristic point in the target area of the current image comprises:

Acquiring candidate feature points in the target area of the current image;

According to the depth value of the candidate feature point and/or the semantic information of the candidate feature point, filtering the candidate feature point to determine the third feature point.
The method according to claim 8, wherein the controlling the photographing device to face the three-dimensional space point corresponding to the target area of the current image comprises:

When the number of third feature points in the target area of the current image is greater than a preset threshold, controlling the photographing device to face the three-dimensional space point corresponding to the target area of the current image.
The method according to claim 8, wherein when the number of third feature points in the target area of the current image is less than or equal to the preset threshold, the camera is controlled to face the reference image The three-dimensional space point corresponding to the target area.
The method according to any one of claims 1-14, further comprising:

The movement of the movable platform is controlled so that the distance between the photographing device and the three-dimensional space point corresponding to the target area of the current image is a circle radius.
The method according to any one of claims 1-14, wherein before the moving the target area of the reference image to obtain the target area of the current image, the method further comprises:

Identifying the target type of the target object;

If the target type is a preset type, the step of moving the target area of the reference image to obtain the target area of the current image is performed.
The method according to any one of claims 1-14, further comprising:

Sending the initial image captured by the photographing device to the control device of the movable platform, so that the control device displays the initial image;

Acquiring instruction information of the user for the target object, the instruction information being generated according to the user's click operation or frame selection operation on the initial image displayed by the control device;

According to the instruction information, the target area of the initial image is determined.
The method according to claim 17, wherein the determining the target area of the initial image according to the instruction information comprises:

Performing image segmentation on the initial image to obtain multiple segmented regions;

Determine the target area of the initial image according to the segmented area and the instruction information.
The method according to claim 18, wherein the determining the target area of the initial image according to the segmented area and the indication information comprises:

When the proportion of the image area in the initial image indicated by the indication information in the target segmentation area is greater than the preset ratio, the target area of the initial image is determined according to the target segmentation area, and the target The divided area is at least one of the plurality of divided areas.
The method according to claim 17, wherein the three-dimensional space point corresponding to the target area of the initial image is used as the center point of the movable platform surrounding the target object, and the movable platform is connected to the target object. The distance between the three-dimensional space points corresponding to the target area of the initial image is taken as the surrounding radius.
A control device for a movable platform, the control device is used to control the movable platform to surround a target object, the movable platform includes a photographing device, and is characterized in that the control device includes: a memory and a processor;

The memory is used to store program code;

The processor calls the program code, and when the program code is executed, is used to perform the following operations:

Acquiring the current image taken by the photographing device;

Acquiring a first feature point in a target area of a reference image, where the reference image is an image of a previous frame of the current image, and the target area is an image area corresponding to the target object;

Determining a second feature point corresponding to the first feature point in the current image;

When the number of the second feature points meets a preset condition, moving the target area of the reference image to obtain the target area of the current image;

Controlling the photographing device to face the three-dimensional space point corresponding to the target area of the current image.
The control device according to claim 21, wherein the processor is further configured to: before the acquiring the first feature point in the target area of the reference image:

The target area of the reference image is divided into a plurality of grid areas.
The control device according to claim 22, wherein when the processor divides the target area of the reference image into multiple grid areas, it is specifically configured to:

According to the direction in which the movable platform surrounds the target object, the target area of the reference image is divided into a plurality of grid areas.
The control device according to claim 22, wherein when the processor obtains the first feature point in the target area of the reference image, it is specifically configured to:

A preset number of feature points are acquired in each grid area as the first feature points.
The control device according to claim 22, wherein the preset condition comprises: the first feature point located in the grid area of the boundary among the multiple grid areas of the reference image is in the current The number of corresponding second feature points in the image is zero;

When the processor moves the target area of the reference image, it is specifically configured to:

Move the target area of the reference image in a direction away from the grid area of the boundary.
The control device according to claim 21, wherein the number of the second characteristic points meets a preset condition, comprising:

The number of the second feature points is less than the number of the first feature points.
The control device according to claim 21, wherein when the processor determines the second feature point corresponding to the first feature point in the current image, it is specifically configured to:

Based on the first feature point, using a tracking algorithm to obtain the feature point obtained by tracking in the current image;

Filtering the feature points obtained by the tracking based on the epipolar constraint condition to obtain the second feature point.
22. The control device according to claim 21, wherein before the processor controls the photographing device to face the three-dimensional space point corresponding to the target area of the current image, it is further configured to:

Acquire a third feature point in the target area of the current image, where the number of the third feature points is greater than or equal to the number of the second feature points.
The control device according to claim 28, wherein when the processor controls the photographing device to face the three-dimensional space point corresponding to the target area of the current image, it is specifically configured to:

Determine the three-dimensional space point corresponding to the target area of the current image according to the third feature point in the target area of the current image;

Controlling the photographing device to face the three-dimensional space point corresponding to the target area of the current image.
The control device according to claim 29, wherein the processor determines the three-dimensional space point corresponding to the target area of the current image according to the third feature point in the target area of the current image, specifically using At:

Performing a weighted average on the depth value of the third feature point in the target area of the current image;

Determine the three-dimensional space point corresponding to the target area of the current image according to the weighted average value and the shooting direction when the shooting device shoots the current image.
The control device according to claim 30, wherein the weight corresponding to the third feature point close to the center of the target area of the current image is greater than the weight corresponding to the third feature point far away from the center of the target area of the current image. Weights.
The control device according to claim 28, wherein when the processor acquires the third feature point in the target area of the current image, it is specifically configured to:

Acquiring candidate feature points in the target area of the current image;

According to the depth value of the candidate feature point and/or the semantic information of the candidate feature point, filtering the candidate feature point to determine the third feature point.
The control device according to claim 28, wherein when the processor controls the photographing device to face the three-dimensional space point corresponding to the target area of the current image, it is specifically configured to:

When the number of third feature points in the target area of the current image is greater than a preset threshold, the camera is controlled to face the three-dimensional space point corresponding to the target area of the current image.
The control device according to claim 28, wherein the processor is further configured to: when the number of third feature points in the target area of the current image is less than or equal to the preset threshold, control The photographing device faces the three-dimensional space point corresponding to the target area of the reference image.
The control device according to any one of claims 21-34, wherein the processor is further configured to:

The movement of the movable platform is controlled so that the distance between the photographing device and the three-dimensional space point corresponding to the target area of the current image is a circle radius.
The control device according to any one of claims 21-34, wherein the processor is further configured to: before moving the target area of the reference image to obtain the target area of the current image:

Identifying the target type of the target object;

If the target type is a preset type, the step of moving the target area of the reference image to obtain the target area of the current image is performed.
The control device according to any one of claims 21-34, wherein the processor is further configured to:

Sending the initial image captured by the photographing device to the control device of the movable platform, so that the control device displays the initial image;

Acquiring instruction information of the user for the target object, the instruction information being generated according to the user's click operation or frame selection operation on the initial image displayed by the control device;

According to the instruction information, the target area of the initial image is determined.
The control device according to claim 37, wherein the processor is specifically configured to: when determining the target area of the initial image according to the instruction information:

Performing image segmentation on the initial image to obtain multiple segmented regions;

Determine the target area of the initial image according to the segmented area and the instruction information.
The control device according to claim 38, wherein the processor is specifically configured to:

When the proportion of the image area in the initial image indicated by the indication information in the target segmentation area is greater than the preset ratio, the target area of the initial image is determined according to the target segmentation area, and the target The divided area is at least one of the plurality of divided areas.
The control device according to claim 37, wherein the three-dimensional space point corresponding to the target area of the initial image is used as the center point of the movable platform surrounding the target object, and the movable platform is connected to the center point of the target object. The distance between the three-dimensional space points corresponding to the target area of the initial image is taken as the surrounding radius.
A movable platform, characterized in that it comprises:

body;

The power system is installed on the fuselage to provide power;

Camera

And the control device according to any one of claims 21-40.
The movable platform of claim 41, wherein the movable platform comprises an unmanned aerial vehicle.
A computer-readable storage medium, characterized in that a computer program is stored thereon, and the computer program is executed by a processor to implement the method according to any one of claims 1-20.