WO2021056501A1

WO2021056501A1 - Feature point extraction method, movable platform and storage medium

Info

Publication number: WO2021056501A1
Application number: PCT/CN2019/108856
Authority: WO
Inventors: 高文良; 周游; 叶长春
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-09-29
Filing date: 2019-09-29
Publication date: 2021-04-01
Also published as: CN112154479A

Abstract

A feature point extraction method, a movable platform and a storage medium. The method comprises: acquiring the current frame image photographed by a photographing device (S101); tracking, in the current frame image, a first feature point in a previous frame image so as to acquire second feature points successfully tracked in the current image frame (S102); determining the number of the second feature points (S103); and determining whether to extract a new feature point from the current frame image according to the number of the second feature points (S104).

Description

Method for extracting feature points, movable platform and storage medium

Technical field

This application relates to the field of computer vision technology, and in particular to a method for extracting feature points, a removable platform and a storage medium.

Background technique

At present, in the field of machine vision, feature points in images are often used for target recognition and tracking. The traditional feature point extraction method needs to enumerate all the pixels of the entire picture through a specific feature point operator, which requires a lot of repeated calculation operations and a large amount of calculation. At the same time, for a strong texture environment, when enumerating 300,000 pixels, hundreds of thousands of feature points may be generated, which requires a large amount of memory.

Summary of the invention

Based on this, the present application provides a method, a removable platform, and a storage medium for extracting feature points.

In the first aspect, the present application provides a method for extracting feature points, which is applied to a movable platform including a camera, and includes:

Acquiring the current frame image taken by the photographing device;

Tracking the first feature point in the previous frame image of the current frame image in the current frame image to obtain the second feature point that is successfully tracked in the current image frame;

Determining the number of the second feature points;

Determine whether to extract new feature points in the current frame image according to the number of the second feature points.

In the second aspect, this application provides a movable platform, including: a camera, a processor, and a memory;

The photographing device is used for photographing images;

The memory is used to store a computer program;

The processor is used to execute the computer program and when executing the computer program, implement the following steps:

Acquiring the current frame image taken by the photographing device;

Determining the number of the second feature points;

In a third aspect, the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor realizes the extraction of feature points as described above Methods.

The embodiment of the application provides a method for extracting feature points, a movable platform, and a storage medium. The first feature point in the previous frame image of the current frame image is tracked in the current frame image to obtain the current frame image. The second feature point that is successfully tracked; according to the number of second feature points, determine whether to extract new feature points in the current frame image; because the first feature point is tracked in the previous frame image, according to the second feature point that is successfully tracked The number of points determines whether to extract new feature points in the current frame image, instead of directly enumerating all the pixels of the current frame image, repeating the calculation, or directly enumerating a fixed number of pixels, by comparing the previous frame image According to the tracking results, it can avoid repeated selection of feature points in the same area, which can reduce the amount of calculation and memory resources; when the number of successfully tracked second feature points meets the preset number threshold, it is not necessary Extracting new feature points can avoid repetition, a large number of calculations, and is fast; when the number of successfully tracked second feature points is less than the preset number threshold, only the remaining number of new feature points can be extracted to the preset number threshold. Reduce the extraction of new feature points and reduce the memory usage, so there is no need to provide a large memory space, and the speed is fast.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. Ordinary technicians can obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic flowchart of an embodiment of a method for extracting feature points according to the present application;

2 is a schematic flowchart of another embodiment of the method for extracting feature points according to the present application;

3 is a schematic flowchart of another embodiment of the method for extracting feature points according to the present application;

4 is a schematic diagram of the position prediction of the first feature point in the method for extracting feature points according to the present application;

FIG. 5 is a schematic diagram of the position tracking of the first feature point in the method for extracting feature points according to the present application;

FIG. 6 is a schematic flowchart of another embodiment of the method for extracting feature points according to the present application;

FIG. 7 is a schematic flowchart of another embodiment of the method for extracting feature points according to the present application;

FIG. 8 is a schematic diagram of the current frame image after rasterization processing in the first application of the method for extracting feature points of the present application;

9 is a schematic diagram of the actual positions of the second feature points that are successfully tracked in the multiple raster images of FIG. 8 and the raster images that do not need to extract new feature points;

FIG. 10 is a schematic diagram of the multiple raster images of FIG. 8 divided into a central area and an area outside the central area;

11 is a schematic diagram of a raster image corresponding to a second feature point successfully tracked in the multiple raster images of FIG. 9 and subsequent extraction of new feature points;

Fig. 12 is a schematic structural diagram of an embodiment of a movable platform of the present application.

detailed description

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The flowchart shown in the drawings is only an example, and does not necessarily include all contents and operations/steps, nor does it have to be executed in the described order. For example, some operations/steps can also be decomposed, combined or partially combined, so the actual execution order may be changed according to actual conditions.

The traditional method of extracting feature points from an image needs to enumerate all the pixels of the entire image, which requires a lot of repeated calculation operations and a large amount of calculation. At the same time, for a strong texture environment, when 300,000 pixels need to be enumerated, hundreds of thousands of feature points may be generated, which requires a large amount of memory. In the embodiment of the present application, the first feature point in the previous frame image of the current frame image is tracked in the current frame image to obtain the second feature point successfully tracked in the current image frame; the determination is made according to the number of second feature points Whether to extract new feature points in the current frame image; since the first feature point is tracked in the previous frame image, the number of second feature points that have been successfully tracked is used to determine whether to extract new feature points in the current frame image instead of Directly enumerate all pixels of the current frame image, repeat the calculation, or directly enumerate a fixed number of pixels, by tracking the first feature point in the previous frame image, it can avoid being in the same area according to the tracking result The internal repetitive selection of feature points can reduce the amount of calculation and memory resources; when the number of successfully tracked second feature points meets the preset number threshold, no new feature points can be extracted, which can avoid repetition, a large number of calculations, and is fast; When the number of successfully tracked second feature points is less than the preset number threshold, only the remaining number of new feature points can be extracted to the preset number threshold, which can reduce the extraction of new feature points and reduce the memory usage, so there is no need to provide a larger number Memory space, and fast.

This application applies to a movable platform that includes a camera. The movable platform refers to various platforms that can move automatically or under controlled conditions, such as PTZ (for example, PTZ camera, etc.), unmanned aerial vehicles , Vehicles, unmanned vehicles, ground robots, etc.

Hereinafter, some embodiments of the present application will be described in detail with reference to the accompanying drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

Referring to Fig. 1, Fig. 1 is a schematic flowchart of an embodiment of a method for extracting feature points according to the present application. The method according to an embodiment of the present application is applied to a movable platform including a camera, and the method includes:

Step S101: Acquire a current frame image taken by the photographing device.

Step S102: Track the first feature point in the previous frame image of the current frame image in the current frame image to obtain the second feature point successfully tracked in the current frame image.

In this embodiment, the first feature point tracking refers to searching for the first feature point (ie target feature point) selected in the previous frame image in the next current frame image (ie real-time frame image) The best position of the point, where the first feature point selected in the last frame of image can be obtained by the method of automatic target detection and recognition, or it can be selected by the method of manual participation.

The first feature point tracking uses a feature point tracking algorithm. Tracking based on feature points mainly includes two aspects: feature point extraction and feature point matching. The first feature point selected in the last frame of image mainly includes color, texture, edge, block feature, optical flow feature, perimeter, area, centroid, corner point, etc. The purpose of extracting the first feature point is to match the first feature point between frames and track the first feature point with the best match. Common tracking algorithms based on feature point matching include: tracking based on binary target image matching, tracking based on edge feature matching or corner feature matching, tracking based on target gray feature matching, tracking based on target color feature matching, etc. . Among them, the KLT tracking algorithm is a widely used tracking algorithm based on feature points. Because the feature points are distributed on the entire target, even if a part is occluded, another part of the feature points can still be tracked. This is also the advantage of the KLT tracking algorithm. .

In general, the advantage of the tracking algorithm based on feature points is that it is not sensitive to changes in the scale, deformation, and brightness of the moving target. Even if a certain part of the target is occluded, as long as a part of the feature can be seen, it can be completed. Tracking task; In addition, this method is used in conjunction with Kalman filter, and it also has a good tracking effect.

Step S103: Determine the number of second feature points.

Step S104: Determine whether to extract new feature points in the current frame image according to the number of second feature points.

Determine whether to extract new feature points in the current frame image according to the number of second feature points, instead of necessarily extracting new feature points in the current frame image, which can reduce the amount of calculation and memory resources.

In the embodiment of the present application, the first feature point in the previous frame image of the current frame image is tracked in the current frame image to obtain the second feature point successfully tracked in the current image frame; the determination is made according to the number of second feature points Whether to extract new feature points in the current frame image; since the first feature point is tracked in the previous frame image, the number of second feature points that have been successfully tracked is used to determine whether to extract new feature points in the current frame image instead of Directly enumerate all pixels of the current frame image, repeat the calculation, or directly enumerate a fixed number of pixels, by tracking the first feature point in the previous frame image, it can avoid being in the same area according to the tracking result The internal repetitive selection of feature points can reduce the amount of calculation and memory resources; when the number of successfully tracked second feature points meets the preset number threshold, no new feature points can be extracted, which can avoid repetition, a large number of calculations, and is fast; When the number of successfully tracked second feature points is less than the preset number threshold, only the remaining number of new feature points can be extracted to the preset number threshold, which can reduce the extraction of new feature points and reduce the memory usage, so there is no need to provide a larger number Memory space, and fast.

The specific details of the tracking of the first feature point in step S102 are described in detail below.

In an embodiment, in order to avoid tracking in the entire image and reduce the tracking range, first determine the tracking area, that is, step S102 may include: sub-step S1021 and sub-step S1022, as shown in FIG. 2.

Sub-step S1021: Determine, in the current frame of image, the tracking area of the first feature point in the previous frame of image in the current frame of image.

Sub-step S1022: Track the first feature point in the previous frame of image within the tracking area in the current frame of image.

In this embodiment, a conventional feature point tracking method is adopted. When the first feature point is tracked in the current frame of image, the tracking starting point is the first feature point itself of the previous frame of image, and the tracking area is usually relatively large. For example, the tracking area uses the first feature point as the origin, and a circular area with R (the size of R is larger) as the radius as the tracking area. According to the relative moving speed of the camera, the difference between the shooting time of the previous frame of image and the shooting time of the current frame of image, the size of R can be roughly estimated. Then track the first feature point in the previous frame image in the tracking area of the current frame image. Compared with tracking the first feature point in the previous frame image in the entire current frame image, the tracking area is reduced a lot , Can improve the tracking speed.

In order to further reduce the tracking area, predict the position of the first feature point in the previous frame image in the current frame image, and determine the tracking area according to the predicted position, that is, sub-step S1021, which may also include: sub-step S1021a and sub-step S1021b, As shown in Figure 3.

Sub-step S1021a: predict the position of the spatial point corresponding to the first feature point in the current frame of image.

As shown in Figure 4, the black circle indicates the position of the first feature point in the previous frame, the gray circle indicates the predicted position of the first feature point in the current frame, and the gray arrow indicates the prediction process of the first feature point. .

Sub-step S1021b: Determine the tracking area of the first feature point in the previous frame of image in the current frame of image according to the predicted position.

Referring to Figure 5, the black dot A represents the position of the first feature point in the previous frame of image, the gray dot B represents the predicted position of the first feature point in the current frame of image, and the gray and black dot C represents the first feature point. The actual position of the successful tracking in the current frame image; the gray arrow 1 (from A to B) represents the prediction process of the first feature point, and the gray arrow 2 (from B to C) represents the tracking process of the first feature point; three The large circles represent the tracking area when the feature point is tracked by the traditional method, and the three small circles represent the tracking area determined according to the predicted position of the first feature point.

Because by predicting the position of the first feature point of the previous frame of image in the current frame of image, tracking around the predicted position, the tracking area can be greatly reduced, so that the tracking speed can be further improved.

Wherein, the sub-step S1021a may specifically include: predicting the position of the spatial point corresponding to the first feature point in the current frame of the image according to the pose information of the last frame of image captured by the camera and the position of the spatial point corresponding to the first feature point .

This embodiment uses a triangulation measurement algorithm. For the feature points that have been calculated and three-dimensionally reconstructed in the previous frame, the position of the spatial point corresponding to the first feature point (ie, three-dimensional coordinates) has been calculated through the triangulation algorithm; for the first frame that has just been extracted but not three-dimensionally reconstructed For a feature point (newly extracted feature point), the average depth of the first feature point in the previous frame is used as a rough depth value to predict the location of the corresponding spatial point (ie, three-dimensional coordinates).

The pose information of the last frame of the image taken by the camera includes the rotational motion information and translation motion information of the last frame of the image taken by the camera. The pose information of the last frame of the image taken by the camera is known information that has been estimated. According to the posture information of the last frame of the image taken by the camera and the movement information of the movable platform equipped with the camera, it can be estimated that the camera is taking the current frame of image posture information, and further based on the estimated camera’s taking the current frame of image The pose information and the position of the space point corresponding to the first feature point can predict the position of the space point corresponding to the first feature point in the current frame image.

_{_{p i = π (RP i +}} t), where, π representing the projector function, P _i is the i-dimensional coordinates of first feature points corresponding to spatial point, p _i is the spatial point corresponding to a first feature points on the current frame image The pixel coordinates of R and t represent the rotation and translation motion information of the current frame image taken by the camera.

The specific details of whether to extract new feature points in the current frame of image in step S104 will be specifically described below.

In an embodiment, in order to further determine whether to extract new feature points in the current frame image, and to avoid repetition and a large number of calculations, step S104 may further include: sub-step S1041, sub-step S1042, and sub-step S1043, as shown in FIG. 6 shown.

Sub-step S1041: Determine whether the number of second feature points is greater than or equal to a first preset number threshold.

Sub-step S1042: if the number of second feature points is greater than or equal to the first preset number threshold, it is determined not to extract new feature points in the current frame image.

Sub-step S1043: If the number of second feature points is less than the first preset number threshold, it is determined to extract new feature points in the current frame of image.

The first preset number threshold is determined according to specific applications and specific requirements. If the number of second feature points that are successfully tracked is relatively large and meets the requirements, and reaches the first preset number threshold and above, there is no need to extract new feature points in the current frame of image. If the number of second feature points that are successfully tracked is relatively small, does not meet the requirements, and does not reach the first preset number threshold, new feature points need to be extracted from the current frame image.

In an application, the first preset number threshold is related to the number of feature points extracted in the first frame, and the first preset number threshold is less than or equal to the number of feature points extracted in the first frame.

For example, by default, the number of feature points extracted from the first frame of image is 120, and the first preset number threshold is set to 100. The number of the first feature points of the previous frame image is 110. If the number of the second feature points that are successfully tracked in the current frame image is 100, it is considered that the number of points is sufficient, then the current frame image does not need to extract new feature points; if the current frame image The number of second feature points that are successfully tracked is 90, so new feature points need to be extracted from the current frame of image.

In this embodiment, when the number of successfully tracked second feature points meets or exceeds the first preset number threshold, no new feature points may be extracted. When the number of successfully tracked second feature points is less than the first preset number threshold, Then extract new feature points; in this way, repetition, a large number of calculations can be avoided, and the speed is fast.

Further, the number of new feature points extracted in the current frame image is determined according to the difference between the first preset number threshold and the number of second feature points. In some embodiments, the number of new feature points extracted in the current frame of image is the difference between the first preset number threshold and the number of second feature points. In this way, it is possible to reduce the extraction of new feature points and reduce the memory occupation, so there is no need to provide a large memory space and the speed is fast.

For example, by default, the number of feature points extracted from the first frame of image is 120, and the first preset number threshold is set to 100. The number of first feature points in the previous frame of image is 110. If the number of second feature points that are successfully tracked in the current frame image is 90, then new feature points need to be extracted in the current frame of image, and the number of new feature points extracted can be 10 There can be more than ten. In general, the number of new feature points to be extracted can be far less than 100, which can greatly reduce the number of new feature points to be extracted and reduce the memory usage. Therefore, there is no need to provide a large memory space and the speed is fast.

If the current frame image needs to extract new feature points, how to extract the new feature points will be described in detail below.

In an actual application, in order to ensure uniform distribution of feature points, the current frame image is divided into multiple raster images. Referring to Figure 7, the method may further include:

Step S201: Perform rasterization processing on the current frame image according to the first preset size to obtain multiple raster images. For example: as shown in Figure 8, the current frame image is rasterized to obtain 20 raster images.

The first preset size is determined according to specific applications and specific requirements. For example: it is determined according to the number of feature points extracted from the first frame of image; or it is determined according to the first preset number threshold, and so on.

Dividing the current frame image into multiple raster images helps reduce the tracking range of the first feature point in the previous frame image in the current frame image, and helps locate the second feature point that has been successfully tracked. The most important thing is to provide technical support to ensure the uniform distribution of feature points.

It should be noted that the range, shape, and number of raster images in this embodiment are not limited.

Step S202: Determine a target raster image from a plurality of raster images, where the target raster image does not include the second feature point. According to the tracking result of step S102, the target raster image that does not include the second feature point can be determined from the plurality of raster images.

Step S203: When it is determined that a new feature point is to be extracted from the current frame image, the new feature point is extracted from the target raster image, wherein at most one new feature point is extracted from the target raster image. According to the sub-step S1043, when it is determined that a new feature point is to be extracted from the current frame image, the new feature point can be extracted from the target raster image. A good sparse feature point set should be uniformly distributed, and each target raster image can extract at most one new feature point as a representative to ensure that the feature point distribution is basically uniform.

As shown in Figure 9, the small dots in the left picture indicate the actual position of the second feature point that has been successfully tracked in the current frame of the image, and the black grid in the right picture indicates that the grid does not require new feature point extraction.

In this way, on the one hand, the problem of determining where the new feature points are extracted, on the other hand, can avoid repeated selection of feature points in the same area, and ensure that the feature point distribution is basically uniform.

Further, in order to avoid extracting new feature points with lower edge quality of the image, the method may further include: determining a central area in the current frame of the image according to a second preset size, wherein the target raster image includes the first located in the central area. A target grid image and a second target grid image located outside the central area. At this time, step S203 extracting new feature points from the target raster image may specifically include: extracting new feature points from the first target raster image.

For the visual positioning system, the quality of the feature points at the edge of the image will be low due to possible motion, camera distortion, and other reasons. As shown in Figure 10, in the left image, according to the priority of the new feature point extraction, the current frame image is divided into the central area A area and the B area outside the central area according to the second preset size; the right image, the area in the A area The first target raster image includes: A1 to A6, and the second target raster image in the B area includes: B1 to B14. The probability of new feature points with better quality that can be used in the area A is higher, so the new feature points in the first target raster image from A1 to A6 in the area A are preferentially extracted.

In an embodiment, the method for determining the second preset size may further include:

Step S301: Obtain status information of the movable platform, where the status information of the movable platform includes the motion status parameters of the movable platform.

Specifically, the motion state parameter includes one or more of the velocity, acceleration, angular velocity, angular acceleration of the movable platform, angular velocity of the photographing device, and angular acceleration of the photographing device.

Step S302: Determine a second preset size according to the motion state parameter of the movable platform. Wherein, the second preset size is negatively related to the motion state parameter.

Through the above method, the size of the central area where new feature points are preferentially extracted can be flexibly adjusted according to specific actual conditions.

In order to extract new feature points that meet the requirements, step S203 extracts new feature points from the first target raster image, which may specifically include: sub-step S2031 and sub-step S2032.

Sub-step S2031: respectively detect the pixel points in different first target raster images according to a preset cycle sequence to determine whether the pixel point is a candidate feature point.

Sub-step S2032: Determine the candidate feature point with the highest quality parameter of the candidate feature point in the first target raster image including the candidate feature point and higher than the preset feature point quality threshold as the new feature point of the first target raster image.

In this embodiment, each pixel in the first target raster image is given an opportunity to extract new feature points, so as to make the feature points evenly distributed as much as possible. However, the first condition for extracting new feature points is: the pixel point meets the requirements of candidate feature points after detection; if the pixel points in the first target raster image do not meet the requirements of candidate feature points after detection, then the second No new feature points are extracted from a target raster image.

The specific methods and requirements for detecting pixels are not limited in the embodiment of the present application. For example: to detect the color of the pixel, its location, whether it is a corner, whether it is an intersection, and so on.

Under normal circumstances, the number of candidate feature points is relatively large, usually far more than the number of new feature points that need to be extracted. The second condition for being able to be extracted as a new feature point is: the candidate feature in the first target raster image The quality parameter of the point is the highest and higher than the preset feature point quality threshold, and the candidate feature point in the first target raster image that meets the second condition can be used as the new feature point of the first target raster image. In an application, Harris can be used to evaluate the quality of feature points.

In order to speed up the speed and save memory space as much as possible, in one embodiment, the method further includes: when it is determined that the second preset number of candidate feature points are detected, no longer detecting the pixels in the first target raster image . That is, in this embodiment, not all the pixels in the first target raster image are searched, as long as the number of detected candidate feature points reaches the second preset number, the pixel points in the first target raster image are no longer searched. Perform testing.

If the number of new feature points extracted from the first target raster image is not enough, you can continue to extract feature points from the second target raster image, that is, step S203 extracts new feature points from the first target raster image, and It may include: determining whether the number of new feature points extracted from the first target raster image is greater than or equal to a third preset number threshold; if not, extracting new feature points from the second target raster image.

The third preset number threshold is determined according to the difference between the first preset number threshold and the number of second feature points. In some embodiments, the third preset number threshold is the difference between the first preset number threshold and the number of second feature points.

For example: referring to Figure 11, there are second feature points that have been successfully tracked in A2, A5, and B7, and no new feature points need to be extracted for these areas. Assuming that a total of 7 feature points are needed, except for the three existing feature points of A2, A5, and B7, the remaining 4 new feature points need to be extracted from the regions of A1, A3, A4, and A6. Then the order of searching for detection can be:

The first pixel in the A1 area → the first pixel in the A3 area → the first pixel in the A4 area → the first pixel in the A6 area → the second pixel in the A1 area → the second pixel in the A3 area A pixel →......→the nth pixel in the A6 area. If there are not enough 7 feature points, select from B1→B2....→B14.

Referring to Figure 12, Figure 12 is a schematic structural diagram of an embodiment of the mobile platform of the present application. It should be noted that the mobile platform of this embodiment can execute the steps in the above-mentioned method for extracting feature points, and a detailed description of the relevant content, Please refer to the part of the method of extracting feature points above, which will not be repeated here.

The mobile platform 10 includes: a photographing device 13, a memory 11 and a processor 12; the photographing device 13, the memory 11 and the processor 12 are connected by a bus 14.

Among them, the processor 12 may be a micro control unit, a central processing unit, or a digital signal processor, and so on.

Among them, the memory 11 may be a Flash chip, a read-only memory, a magnetic disk, an optical disk, a U disk, or a mobile hard disk, etc.

The photographing device 13 is used to photograph images; the memory 11 is used to store a computer program; the processor 12 is used to execute the computer program and when the computer program is executed, the following steps are implemented:

Acquire the current frame image taken by the camera; track the first feature point in the previous frame image of the current frame image in the current frame image to obtain the second feature point successfully tracked in the current image frame; determine the second feature point The number of; according to the number of second feature points to determine whether to extract new feature points in the current frame of image.

In the embodiment of the present application, the first feature point in the previous frame image of the current frame image is tracked in the current frame image to obtain the second feature point successfully tracked in the current image frame; the determination is made according to the number of second feature points Whether to extract new feature points in the current frame image; since the first feature point is tracked in the previous frame image, the number of second feature points that have been successfully tracked is used to determine whether to extract new feature points in the current frame image instead of Directly enumerate all pixels of the current frame image, repeat the calculation, or directly enumerate a fixed number of pixels, by tracking the first feature point in the previous frame image, it can avoid being in the same area according to the tracking result The internal repetitive selection of feature points can reduce the amount of calculation and memory resources; when the number of successfully tracked second feature points meets the preset number threshold, no new feature points can be extracted, which can avoid repetition, a large number of calculations, and is fast; When the number of successfully tracked second feature points is less than the preset number threshold, only the remaining number of new feature points can be extracted to the preset number threshold, which can reduce the extraction of new feature points and reduce the memory usage, so there is no need to provide a larger Memory space, and fast.

Wherein, when the processor executes the computer program, the following steps are implemented: if the number of second feature points is greater than or equal to the first preset number threshold, it is determined not to extract new feature points in the current frame image; otherwise, it is determined to be Extract new feature points from the current frame of image.

Wherein, the number of new feature points extracted in the current frame image is determined according to the difference between the first preset number threshold and the number of second feature points.

Wherein, when the processor executes the computer program, it implements the following steps: rasterize the current frame image according to the first preset size to obtain multiple raster images; determine the target raster image from the multiple raster images, Among them, the target raster image does not include the second feature point; when it is determined to extract a new feature point from the current frame image, the new feature point is extracted from the target raster image, where at most one new feature point is extracted from the target raster image .

Wherein, when the processor executes the computer program, the following steps are implemented: determine the central area in the current frame image according to the second preset size, wherein the target raster image includes a first target raster image located in the central area and a first target raster image located in the center. The second target raster image outside the area; new feature points are extracted from the first target raster image.

Wherein, when the processor executes the computer program, the following steps are implemented: according to a preset loop sequence, the pixels in different first target raster images are respectively detected to determine whether the pixel is a candidate feature point; the candidate will be included In the first target raster image of the characteristic points, the candidate characteristic point with the highest quality parameter of the candidate characteristic point and higher than the preset characteristic point quality threshold is determined as the new characteristic point of the target raster image.

Wherein, when the processor executes the computer program, the following steps are implemented: when it is determined that the second preset number of candidate feature points are detected, no more detection is performed on the pixel points in the first target raster image.

Wherein, when the processor executes the computer program, it implements the following steps: determining whether the number of new feature points extracted from the first target raster image is greater than or equal to the third preset number threshold; if not, in the second target raster image Extract new feature points from the image.

Wherein, the third preset number threshold is determined according to the difference between the first preset number threshold and the number of second feature points.

Wherein, when the processor executes the computer program, it implements the following steps: acquiring state information of the movable platform, where the state information of the movable platform includes the movement state parameters of the movable platform; and determining the second step according to the movement state parameters of the movable platform Preset size.

Among them, the motion state parameters include one or more of the speed, acceleration, angular velocity, angular acceleration of the movable platform, the angular velocity of the photographing device, and the angular acceleration of the photographing device.

Wherein, the second preset size is negatively related to the motion state parameter.

Among them, when the processor executes the computer program, it implements the following steps: in the current frame image, determine the tracking area of the first feature point in the previous frame image in the current frame image; align it in the tracking area in the current frame image The first feature point in a frame of image is tracked.

Wherein, when the processor executes the computer program, it implements the following steps: predict the position of the spatial point corresponding to the first feature point in the current frame image; determine the first feature point in the previous frame image according to the predicted position in the current frame image Tracking area in.

Wherein, when the processor executes the computer program, the following steps are implemented: according to the pose information of the last frame of the image taken by the camera and the position of the space point corresponding to the first feature point, predict that the space point corresponding to the first feature point is in the current frame The position in the image.

The present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor implements the method for extracting feature points as described in any of the above items. For a detailed description of the relevant content, please refer to the above method of extracting feature points, which will not be repeated here.

Wherein, the computer-readable storage medium may be an internal storage unit of any of the above-mentioned removable platforms, such as a hard disk or memory of the removable platform. The computer-readable storage medium may also be an external storage device of the removable platform, such as a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, etc., equipped on the removable platform.

It should be understood that the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit the application.

It should also be understood that the term "and/or" used in the specification and appended claims of this application refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

The above are only specific embodiments of this application, but the scope of protection of this application is not limited to this. Any person skilled in the art can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, these modifications or replacements shall be covered within the scope of protection of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

A method for extracting feature points, applied to a movable platform including a camera, is characterized in that it includes:

Acquiring the current frame image taken by the photographing device;

Tracking the first feature point in the previous frame image of the current frame image in the current frame image to obtain the second feature point that is successfully tracked in the current image frame;

Determining the number of the second feature points;

Determine whether to extract new feature points in the current frame image according to the number of the second feature points.
The method according to claim 1, wherein the determining whether to extract new feature points in the current frame image according to the number of the second feature points, further comprises:

If the number of the second feature points is greater than or equal to the first preset number threshold, determining not to extract new feature points in the current frame image;

Otherwise, it is determined to extract new feature points in the current frame image.
The method according to claim 2, wherein the number of new feature points extracted in the current frame image is determined according to the difference between the first preset number threshold and the number of second feature points .
The method according to any one of claims 1-3, wherein the method further comprises:

Performing rasterization processing on the current frame image according to the first preset size to obtain multiple raster images;

Determining a target raster image from the plurality of raster images, wherein the target raster image does not include the second feature point;

When it is determined that a new feature point is to be extracted from the current frame image, a new feature point is extracted from the target raster image, wherein at most one new feature point is extracted from the target raster image.
The method according to claim 4, wherein the method further comprises:

A central area is determined in the current frame image according to a second preset size, wherein the target raster image includes a first target raster image located in the central area and a second target located outside the central area Raster image

The extracting new feature points from the target raster image includes:

Extracting new feature points from the first target raster image.
The method according to claim 5, wherein the extracting new feature points from the first target raster image comprises:

Respectively detecting pixels in different first target raster images according to a preset cycle sequence to determine whether the pixel is a candidate feature point;

The candidate feature point with the highest quality parameter of the candidate feature point in the first target raster image including the candidate feature point and higher than the preset feature point quality threshold is determined as the new feature point of the first target raster image.
The method according to claim 6, wherein the method further comprises:

When it is determined that the second preset number of candidate feature points are detected, the pixel points in the first target raster image are no longer detected.
The method according to claim 5 or 6, wherein the extracting new feature points from the first target raster image further comprises:

Determining whether the number of new feature points extracted from the first target raster image is greater than or equal to a third preset number threshold;

When not, extract new feature points from the second target raster image.
The method according to claim 8, wherein the third preset number threshold is determined according to the difference between the first preset number threshold and the number of the second feature points.
The method according to any one of claims 1-9, wherein the method further comprises:

Acquiring state information of the movable platform, wherein the state information of the movable platform includes the motion state parameters of the movable platform;

The second preset size is determined according to the motion state parameter of the movable platform.
The method according to claim 10, wherein the motion state parameter includes one of the speed, acceleration, angular velocity, angular acceleration of the movable platform, the angular velocity of the photographing device, and the angular acceleration of the photographing device Or multiple.
The method according to claim 10 or 11, wherein the second preset size is negatively related to the motion state parameter.
The method according to any one of claims 1-12, wherein the tracking, in the current frame image, the first feature point in the previous frame image of the current frame image comprises:

Determining, in the current frame of image, the tracking area of the first feature point in the previous frame of image in the current frame of image;

Tracking the first feature point in the previous frame of image in the tracking area in the current frame of image.
The method according to claim 13, wherein the determining in the current frame image the tracking area of the first feature point in the previous frame image in the current frame image comprises:

Predicting the position of the spatial point corresponding to the first feature point in the current frame image;

The tracking area of the first feature point in the previous frame image in the current frame image is determined according to the predicted position.
The method according to claim 14, wherein the predicting the position of the spatial point corresponding to the first feature point in the current frame image comprises:

Predict the position of the spatial point corresponding to the first feature point in the current frame of the image according to the position and orientation information of the previous frame of the image taken by the photographing device and the position of the spatial point corresponding to the first feature point .
A movable platform, characterized in that, the movable platform includes: a photographing device, a processor, and a memory;

The photographing device is used for photographing images;

The memory is used to store a computer program;

The processor is used to execute the computer program and when executing the computer program, implement the following steps:

Acquiring the current frame image taken by the photographing device;

Tracking the first feature point in the previous frame image of the current frame image in the current frame image to obtain the second feature point that is successfully tracked in the current image frame;

Determining the number of the second feature points;

Determine whether to extract new feature points in the current frame image according to the number of the second feature points.
The mobile platform according to claim 16, wherein the processor implements the following steps when executing the computer program:

If the number of the second feature points is greater than or equal to the first preset number threshold, determining not to extract new feature points in the current frame image;

Otherwise, it is determined to extract new feature points in the current frame image.
The mobile platform according to claim 17, wherein the number of new feature points extracted in the current frame image is based on the difference between the first preset number threshold and the number of second feature points definite.
The movable platform according to any one of claims 16-18, wherein the processor implements the following steps when executing the computer program:

Performing rasterization processing on the current frame image according to the first preset size to obtain multiple raster images;

Determining a target raster image from the plurality of raster images, wherein the target raster image does not include the second feature point;

When it is determined that a new feature point is to be extracted from the current frame image, a new feature point is extracted from the target raster image, wherein at most one new feature point is extracted from the target raster image.
The mobile platform according to claim 19, wherein the processor implements the following steps when executing the computer program:

A central area is determined in the current frame image according to a second preset size, wherein the target raster image includes a first target raster image located in the central area and a second target located outside the central area Raster image

Extracting new feature points from the first target raster image.
The mobile platform according to claim 20, wherein the processor implements the following steps when executing the computer program:

Respectively detecting pixels in different first target raster images according to a preset cycle sequence to determine whether the pixel is a candidate feature point;

The candidate feature point with the highest quality parameter of the candidate feature point in the first target raster image including the candidate feature point and higher than the preset feature point quality threshold is determined as the new feature point of the first target raster image.
The mobile platform according to claim 21, wherein the processor implements the following steps when executing the computer program:

When it is determined that the second preset number of candidate feature points are detected, the pixel points in the first target raster image are no longer detected.
The movable platform according to claim 20 or 21, wherein the processor implements the following steps when executing the computer program:

Determining whether the number of new feature points extracted from the first target raster image is greater than or equal to a third preset number threshold;

When not, extract new feature points from the second target raster image.
The movable platform according to claim 23, wherein the third preset number threshold is determined according to the difference between the first preset number threshold and the number of the second feature points.
The movable platform according to any one of claims 16-24, wherein the processor implements the following steps when executing the computer program:

Acquiring state information of the movable platform, wherein the state information of the movable platform includes the motion state parameters of the movable platform;

The second preset size is determined according to the motion state parameter of the movable platform.
The movable platform according to claim 25, wherein the motion state parameters include the speed, acceleration, angular velocity, angular acceleration of the movable platform, the angular velocity of the photographing device, and the angular acceleration of the photographing device. One or more.
The movable platform according to claim 25 or 26, wherein the second preset size is negatively related to the motion state parameter.
The movable platform according to any one of claims 16-27, wherein the processor implements the following steps when executing the computer program:

Determining, in the current frame of image, the tracking area of the first feature point in the previous frame of image in the current frame of image;

Tracking the first feature point in the previous frame of image in the tracking area in the current frame of image.
The mobile platform according to claim 28, wherein the processor implements the following steps when executing the computer program:

Predicting the position of the spatial point corresponding to the first feature point in the current frame image;

The tracking area of the first feature point in the previous frame of image in the current frame of image is determined according to the predicted position.
The mobile platform according to claim 29, wherein the processor implements the following steps when executing the computer program:

Predict the position of the spatial point corresponding to the first feature point in the current frame of the image according to the position and orientation information of the previous frame of the image taken by the photographing device and the position of the spatial point corresponding to the first feature point .
A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor realizes any one of claims 1-15. The method of extracting feature points.