CN112785705B - Pose acquisition method and device and mobile equipment - Google Patents

Abstract

The application discloses a pose acquisition method, a pose acquisition device and mobile equipment, wherein the pose acquisition method comprises the following steps: obtaining a current frame image; obtaining characteristic points with matched characteristic points in a history image contained in a sliding window in a current frame image; respectively obtaining a first pixel model of the matched characteristic points in each current frame image; respectively comparing the first pixel model of the matched characteristic points in each current frame image with the corresponding second pixel model to obtain a model comparison result, wherein the model comparison result represents whether the spatial points corresponding to the matched characteristic points in the current frame image belong to a moving object or not; the second pixel model is a pixel model of the characteristic points matched with the characteristic points in the current frame image in the historical image; screening out target feature points in the current frame image according to the model comparison result, wherein the target feature points are feature points of which the corresponding space points do not belong to a moving object; and obtaining the current pose of the mobile equipment according to the reprojection error corresponding to the target feature point.

Description

A posture acquisition method, device and mobile device

Technical field

The present application relates to the field of smart home technology, and in particular to a posture acquisition method, device and mobile device.

Background technique

Simultaneous Localization and Mapping (SLAM) technology has become a hot topic in current research in the field of robotics. SLAM technology can be divided into laser SLAM and visual SLAM according to different sensors. Among them, visual SLAM is to infer the position and posture of the robot and the surrounding environment through images of continuous motion. Visual SLAM specifically refers to a robot equipped with specific sensors that builds an environment model during movement and estimates its own movement posture without prior information about the environment.

The current algorithms based on SLAM positioning are based on the assumption that the environment is static or quasi-static, that is, the entire scene is static. However, most of the actual robot operating scenes are dynamic scenes, and relatively moving objects will interfere with pose acquisition, so the positioning accuracy may be low.

Therefore, a technical solution that can improve the accuracy of SLAM positioning is urgently needed.

Contents of the invention

In view of this, the present application provides a pose acquisition method, device and mobile device to solve the technical problem of low positioning accuracy of mobile devices in dynamic scenes in the prior art.

This application provides a pose acquisition method, including:

Obtain a current frame image, which is an image collected by an image acquisition device on the mobile device;

Obtain the matched feature points in the current frame image, wherein the matched feature points in the current frame image are those with matching feature points in the historical images included in the sliding window corresponding to the current frame image. Feature points; the sliding window contains multiple frames of historical images, and the historical images are key frame images before the current frame image;

A first pixel model of each matched feature point in the current frame image is obtained respectively, the first pixel model corresponds to a plurality of neighbor feature points of the matched feature point in the current frame image, and the The first pixel model also has a direction vector of the first pixel model and a reprojection error of the matched feature points in the current frame image;

The first pixel model of the matched feature point in each of the current frame images is compared with the corresponding second pixel model to obtain a model comparison result, and the model comparison result characterizes the current frame image. Whether the spatial point corresponding to the matched feature point belongs to a moving object; wherein, the second pixel model is a pixel model of the feature point in the historical image that matches the feature point matched in the current frame image, The second pixel model corresponds to a plurality of neighbor feature points of the matched feature points in the historical image, and the second pixel model also has a direction vector of the second pixel model and a matching feature point in the historical image. The mean value of the reprojection error of the feature points;

According to the model comparison result, target feature points in the current frame image are screened out, and the target feature points are feature points whose corresponding spatial points in the current frame image do not belong to moving objects;

According to the reprojection error corresponding to the target feature point in the current frame image, the current pose of the mobile device is obtained.

The above method, preferably, obtains the current pose of the mobile device based on the reprojection error corresponding to the target feature point in the current frame image, including:

Obtain the weight value corresponding to the target feature point according to the depth value of the spatial point corresponding to the target feature point;

According to the weight value corresponding to the target feature point, the pose transformation matrix corresponding to the minimum reprojection error of the target feature point is obtained. The reprojection error is based on the three-dimensional corresponding three-dimensional image of the target feature point in the historical image. The coordinate value and the two-dimensional coordinate value of the target feature point in the current frame image are obtained;

According to the pose transformation matrix, the current pose of the mobile device is obtained.

The above method, preferably, obtains the first pixel model of the matched feature points in each of the current frame images, including:

Using the matched feature points in each current frame image as the center, obtain multiple neighbor feature points of the matched feature points in the current frame image;

Wherein, the depth value of the matched feature point in the current frame image is different from that of its corresponding neighbor feature point, and the depth value of the neighbor feature point of the matched feature point in the current frame image is greater than the target depth and is different from the target depth. The distance between the matched feature points in the current frame image satisfies the feature points of the distance sorting rule, and the target depth is related to the depth mean of the neighborhood pixels of the matched feature points in the current frame image;

Establish a first pixel model of the matched feature point in the current frame image based on at least a plurality of neighbor feature points of the matched feature point in the current frame image, so that the first pixel model corresponds to the The multiple neighbor feature points of the matched feature point in the current frame image also have the direction vector of the first pixel model and the reprojection error of the matched feature point in the current frame image.

In the above method, the direction vector of the first pixel model is based on the three-dimensional coordinate value of the spatial center point corresponding to the neighbor feature point in the first pixel model and the spatial point corresponding to the matched feature point in the current frame image. Three-dimensional coordinate values are obtained, and the reprojection error of the matched feature points in the current frame image is obtained based on the historical image.

In the above method, preferably, the first pixel model of the matched feature points in each of the current frame images is compared with the corresponding second pixel model to obtain the model comparison result, including:

Obtain the number of matching neighbor feature points in the first pixel model and its corresponding second pixel model;

Obtain the module length of the difference between the direction vector in the first pixel model and its corresponding direction vector in the second pixel model;

Determine whether the reprojection error of the matched feature points in the current frame image in the first pixel model is less than or equal to the reprojection error corresponding to the matched feature points in the historical image in the second pixel model The mean value is the mean value of the cumulative reprojection error of the matched feature points in the historical image in the sliding window to obtain the judgment result;

According to the quantity, the module length and the judgment result, a model comparison result is obtained.

The above method preferably also includes:

Obtain an update request for a pixel model of a feature point in the historical image;

Obtain newly added images within the sliding window;

Perform feature point extraction on the newly added image to obtain feature points in the newly added image;

Match the feature points in the newly added image with the feature points in the historical images in the sliding window to obtain matching features in the newly added image that match the feature points in the historical images. points and non-matching feature points in the newly added image that do not match the feature points in the historical image;

According to the matching feature point, update the pixel model of the feature point in the historical image that matches the matching feature point;

Obtain the pixel model of the non-matching feature point, so that when the newly added image is used as a historical image in the sliding window, the pixel model of the non-matching feature point is used as a new feature point in the historical image. pixel model.

In the above method, preferably, after filtering out the target feature points in the current frame image according to the model comparison result, and obtaining the said target feature points according to the reprojection error corresponding to the target feature point in the current frame image. Before moving the current posture of the device, the method further includes:

Among the target feature points in the current frame image, target feature points that satisfy the epipolar constraint with the feature points in the historical image are screened out.

The above method, preferably, obtains the matched feature points in the current frame image, including:

Extract feature points in the current frame image;

Match the feature points extracted from the current frame image with the feature points extracted from the historical image to obtain feature points in the current frame image that match the feature points in the historical image.

This application also provides a posture acquisition device, including:

An image acquisition unit, used to obtain a current frame image, where the current frame image is an image collected by an image acquisition device on a mobile device;

A feature point processing unit, configured to obtain the matched feature points in the current frame image, where the matched feature points in the current frame image are historical images included in the sliding window corresponding to the current frame image. There are feature points that match the feature points; the sliding window contains multiple frames of historical images, and the historical images are key frame images before the current frame image;

A model building unit configured to obtain a first pixel model of each feature point matched in the current frame image, where the first pixel model corresponds to multiple neighbors of the feature point matched in the current frame image. Feature points, and the first pixel model also has a direction vector of the first pixel model and a reprojection error of the matched feature points in the current frame image;

A model comparison unit, configured to compare the first pixel model of the matched feature points in each of the current frame images with the corresponding second pixel model to obtain a model comparison result. The model comparison The result represents whether the spatial point corresponding to the matched feature point in the current frame image belongs to a moving object; wherein, the second pixel model matches the feature point matched in the current frame image in the historical image. A pixel model of the feature point, the second pixel model corresponds to a plurality of neighbor feature points of the matched feature point in the historical image, and the second pixel model also has a direction vector of the second pixel model and the mean value of the reprojection error of the matched feature points in the historical image;

A feature point screening unit, configured to screen out target feature points in the current frame image based on the model comparison results, where the target feature points are features of corresponding spatial points in the current frame image that do not belong to moving objects. point;

A pose obtaining unit is configured to obtain the current pose of the mobile device based on the reprojection error corresponding to the target feature point in the current frame image.

This application also provides a mobile device, including:

Memory for storing application programs and data generated by the operation of said application programs;

A processor for executing said application to achieve:

Obtain a current frame image, which is an image collected by an image acquisition device on the mobile device;

Obtain the matched feature points in the current frame image, wherein the matched feature points in the current frame image are those with matching feature points in the historical images included in the sliding window corresponding to the current frame image. Feature points; the sliding window contains multiple frames of historical images, and the historical images are key frame images before the current frame image;

A first pixel model of each matched feature point in the current frame image is obtained respectively, the first pixel model corresponds to a plurality of neighbor feature points of the matched feature point in the current frame image, and the The first pixel model also has a direction vector of the first pixel model and a reprojection error of the matched feature points in the current frame image;

The first pixel model of the matched feature point in each of the current frame images is compared with the corresponding second pixel model to obtain a model comparison result, and the model comparison result characterizes the current frame image. Whether the spatial point corresponding to the matched feature point belongs to a moving object; wherein, the second pixel model is a pixel model of the feature point in the historical image that matches the feature point matched in the current frame image, The second pixel model corresponds to a plurality of neighbor feature points of the matched feature points in the historical image, and the second pixel model also has a direction vector of the second pixel model and a matching feature point in the historical image. The mean value of the reprojection error of the feature points;

According to the model comparison result, target feature points in the current frame image are screened out, and the target feature points are feature points whose corresponding spatial points in the current frame image do not belong to moving objects;

According to the reprojection error corresponding to the target feature point in the current frame image, the current pose of the mobile device is obtained.

It can be seen from the above technical solutions that the pose acquisition method, device and mobile device disclosed in this application, after the current frame image collected by the image acquisition device on the mobile device, compares the current frame image with the previous sliding window The feature points that match the feature points in the included historical images are obtained, and then after obtaining the pixel models corresponding to multiple neighbor feature points of the feature points in the current frame image, the pixel model is combined with the sliding window. The pixel models of the feature points in the historical images included are compared, and then based on the model comparison results, the corresponding spatial points in the current frame image can be filtered out that are not target feature points of moving objects. Based on this, based on these targets, The reprojection error of the feature points is used to obtain the current pose of the mobile device. It can be seen that in this application, the pixel model of the matching feature points in the front and rear images is used to eliminate the feature points belonging to the moving object, so as to realize the positioning of the mobile device based on the filtered feature points that do not belong to the moving object. Therefore, It can avoid the interference of the feature points of moving objects on pose acquisition, thereby improving the positioning accuracy of mobile devices in dynamic scenes.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. Those of ordinary skill in the art can also obtain other drawings based on these drawings without exerting creative efforts.

Figure 1 is a flow chart of a pose acquisition method provided in Embodiment 1 of the present application;

Figures 2-6 are example diagrams of embodiments of the present application respectively;

Figures 7-8 are partial flow charts of a posture acquisition method provided in Embodiment 1 of the present application;

Figure 9 is another example diagram of an embodiment of the present application;

Figures 10 and 11 are respectively another part of the flow chart of a posture acquisition method provided in Embodiment 1 of the present application;

Figure 12 is another flow chart of a pose acquisition method provided in Embodiment 1 of the present application;

Figure 13 is a schematic structural diagram of a posture acquisition device provided in Embodiment 2 of the present application;

Figure 14 is another structural schematic diagram of a posture acquisition device provided in Embodiment 2 of the present application;

Figure 15 is a schematic structural diagram of a mobile device provided in Embodiment 3 of the present application;

Figures 16 to 19 are respectively example diagrams of embodiments of the present application being applied to mobile robots.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

Referring to Figure 1, there is a flow chart for implementing a pose acquisition method provided in Embodiment 1 of the present application. This method can be applied to electronic devices that can process images, such as mobile devices with image acquisition devices, such as mobile robots. etc., or it can be a device connected to a mobile device, such as a computer or server. The technical solution in this embodiment is used to improve the accuracy of obtaining the pose of the mobile device.

Specifically, the method in this embodiment may include the following steps:

Step 101: Obtain the current frame image.

Among them, the current frame image is the latest image collected by the image collection device on the mobile device. The image collection device can be a camera or a video recorder, and can collect images within the corresponding collection range of the mobile device, as shown in Figure 2 It shows that the top of the mobile robot is equipped with a camera. During the movement of the mobile robot, the camera can collect images in the front range.

It should be noted that in this embodiment, the current frame image collected by the image acquisition device can be obtained through the connection interface with the image acquisition device, or in this embodiment, the mobile image can be obtained through the communication interface with the mobile device. The current frame image collected by the image acquisition device transmitted by the device.

In addition, the current frame image in this embodiment can be understood as the image that needs to be processed at the current moment.

Step 102: Obtain the matched feature points in the current frame image.

Among them, the matched feature points in the current frame image obtained in step 102 are the feature points in the current frame image that match the feature points in the historical image, and the historical image is the image in the sliding window before the current frame image, As shown in Figure 3, after the historical images are collected, one or more frames of historical images form a sliding window corresponding to the current frame image and are stored in the image library. Based on this, in this embodiment, after obtaining the current After framing the image, the historical images in the saved sliding window can be read in the image library. Among them, the feature points in the current frame image that match the feature points in the historical image are the feature points that correspond to the same spatial point in the map as the feature points in the historical image.

It should be noted that the historical images added to the sliding window in this embodiment are key frame images in the images collected by the image acquisition device. Key frames refer to image frames with representative significance, which can be understood as characters or The image frame where the key action in the movement or change of the object occurs, such as I frame, etc.

In one implementation, in this embodiment, when obtaining the feature points in the current frame image that match the feature points in the historical image in step 102, it can be achieved in the following manner:

First, extract the feature points in the current frame image. The feature points in the image refer to representative points in the image, which are another digital expression of image information. The feature points in this embodiment can be ORB (iented Fast and Rotated Brief) features in the current frame image, which are composed of key points and descriptors. The key points can be FAST corner points, which have scale and rotation invariance, and The descriptor can be a binary BRIEF descriptor. In specific implementation, this embodiment can use a preset feature extraction algorithm to extract feature points in the current frame image, thereby obtaining feature points in the current frame image that meet the extraction conditions. In order to further improve the accuracy, in this embodiment, the current frame image can be divided into multiple small squares, and then feature points are extracted for each small square, thereby increasing the uniformity of feature point extraction, so that the current frame The feature points of the image can be evenly distributed throughout the image field of view, and the extracted feature points at this time include the feature points in the current frame image;

Then, the feature points extracted from the current frame image are matched with the feature points extracted from the historical image. Among them, the feature points in the historical image are the feature points obtained by extracting feature points using the same feature extraction method as the current frame image.

Specifically, in this embodiment, a preset matching algorithm can be used to match the feature points in the current frame image with the feature points in the historical image, for example, ORB descriptor matching is performed on the feature points. In order to further improve the matching rate, in this embodiment, matching algorithms such as DBOW2 bag-of-word technology can be used to accelerate the matching of feature points in the current frame image and features in historical images, thereby matching features in the current frame image and historical images. Points correspond to feature points in the same space point, as shown in Figure 4. At this time, the feature points matched in the current frame image have matching feature points in the historical image.

Specifically, in this embodiment, the feature points in the current frame image can be point matched with the feature points extracted from the key frame image in the sliding window that is closest to the current frame image. If the feature points in the current frame image are consistent with the sliding window If the number of feature points matching the feature points extracted from the most recent key frame image of the current frame image does not meet the required number, then the feature points in the current frame image will be compared with the feature points in all key frame images in the sliding window. Feature points are matched until the required number is met.

Further, in order to improve the reliability of this embodiment, it is necessary to determine whether the number of matched feature points in the current frame image reaches the preset required number. If the number of matched feature points in the current frame image does not meet the requirement, number, then you can modify the matching algorithm, such as relaxing the matching conditions in the matching algorithm, and then use the matching algorithm after relaxing the matching conditions to match the feature points extracted from the current frame image with the feature points in the historical images in the sliding window. For example, the DBOW2 bag-of-words technology that relaxes the matching conditions is used to accelerate the matching of feature points in the current frame image that match the feature points in the historical image, thereby increasing the number of matched feature points in the current frame image, so that the matched features The number of points can meet the required number.

It should be noted that the historical images in the sliding window are key frame images. Therefore, when matching all the features in the current frame image, the feature points of the key frames in the sliding window are used, thus matching the current Feature points in the frame image that match key frame feature points in the sliding window.

Further, in this embodiment, after obtaining the feature points in the current frame image that match the feature points in the historical image in step 102, the reprojection error can be calculated based on the matched feature points in the current frame image, and based on The calculated reprojection error initially estimates the current pose of the mobile device. For example, in this embodiment, the depth values of the spatial points corresponding to the feature points in the historical image that match the feature points in the current frame image are obtained based on the depth images corresponding to the historical images, and then the three-dimensional coordinate values of these spatial points are obtained, and then used The initial pose transformation matrix converts the three-dimensional coordinate values into the current coordinate system corresponding to the current frame image, then projects the two-dimensional coordinate values of these spatial points in the current coordinate system, and then matches these coordinate values with the current frame image. The error is calculated based on the two-dimensional coordinate values of the corresponding feature points. From this, the pose estimation is performed based on the calculated reprojection error, thereby preliminarily estimating the current pose of the mobile device.

Step 103: Obtain the first pixel model of the matched feature points in each current frame image.

Among them, the first pixel model can also be called a feature point model. The first pixel model corresponds to multiple neighbor feature points, direction vectors and reprojection errors of the feature points matched in the current frame image. Further, in the current frame image The depth value of the neighbor feature point of the matched feature point is different from the depth value of the feature point in the corresponding current frame image. That is to say, in this embodiment, a separate establishment is established for each matched feature point in the current frame image. A first pixel model, and the first pixel model of each feature point matched in the current frame image is established based on multiple neighbor feature points that have different depth values from the feature points matched in the current frame image. These Neighbor feature points may have the same or different depth values. Based on this, in this embodiment, when obtaining the first pixel model of the matched feature points in the current frame image, it can be achieved in the following ways:

First, multiple neighbor feature points of the matched feature points in the current frame image are obtained. The neighbor feature points are distributed around the matched feature points in the current frame image, as shown in Figure 5;

Then, eliminate redundant neighbor feature points with the same depth value among these multiple neighbor feature points;

After that, the first pixel model of the matched feature points in the current frame image is established based on the remaining neighbor feature points.

Specifically, the first pixel model may contain feature data of multiple neighbor feature points, such as pixel values, coordinate values, etc. of neighbor feature points. It also includes: the direction vector of the first pixel model and the matching result in the current frame image. The reprojection error of the feature point, where the direction vector of the first pixel model is based on the three-dimensional coordinate value of the spatial center point corresponding to the neighbor feature point in the first pixel model and the spatial point corresponding to the matched feature point in the current frame image The three-dimensional coordinate values of Geometric errors in frame images are obtained. For example, based on the depth image corresponding to the historical image, the depth values of the spatial points corresponding to the feature points in the historical image that match the feature points matched in the current frame image are obtained, and then the three-dimensional coordinate values of these spatial points are obtained, and then the initial The pose transformation matrix converts the three-dimensional coordinate values into the current coordinate system corresponding to the current frame image, and then projects the two-dimensional coordinate values of these spatial points in the current coordinate system, and then matches these coordinate values with the coordinate values in the current frame image. The two-dimensional coordinate values of the corresponding feature points are used for error calculation.

Specifically, the first pixel model can be represented by {p' ₁ , p' ₂ , p' ₃ ...p' _N , n', r'}. Among them, {p' ₁ , p' ₂ , p' ₃ ...p' _N } are the neighbor feature points of the matched feature point p′ in the current frame image that are different in spatial depth from the feature point p′, and the direction vector Expressed by n′, specifically means, among them,/> is the three-dimensional coordinate value of the spatial center point corresponding to the neighbor feature point of feature point p′, (x, y, z) is the three-dimensional coordinate value of the spatial point corresponding to the matched feature point in the current frame image. The three-dimensional coordinate value can be passed The feature points in the current frame image and their corresponding depth values in the depth image are mapped into space to obtain. The reprojection error of the matched feature point p′ in the current frame image is represented by r`, and N is the number of neighbor feature points of the feature point p′, which can be a preset value, and can be determined based on empirical data.

It should be noted that since the matching feature points in the current frame image obtained in step 102 are feature points in the current frame image that match the feature points in the historical image in the sliding window, therefore, the first pixel model is based on The pixel model of the feature points obtained by filtering the feature points of the previous historical images of the current frame image against the original feature points of the current frame image. In addition, corresponding pixel models need to be established for other feature points in the current frame image other than the feature points that match the feature points in the historical image. The way to build pixel models for these other feature points can refer to the first pixel model. In the establishment method, the pixel models of these other feature points are mainly used to update the second pixel model corresponding to the feature points in the historical image that match the feature points of the current frame image. For details, see the following content. Therefore, the establishment of the first pixel model of the feature point matched in the current frame image is based on the premise that the feature point matches the feature point in the historical image. Therefore, in this embodiment, the matching feature point in the current frame image The establishment of the first pixel model of the feature points is based on historical images.

Step 104: Compare the first pixel model of the matched feature point in each current frame image with the corresponding second pixel model to obtain a model comparison result.

Among them, the model comparison result indicates whether the spatial point corresponding to the matched feature point in the current frame image belongs to a moving object; the second pixel model is the pixel model of the feature point in the historical image, and the second pixel model corresponds to the pixel model of the feature point in the historical image. The mean value of multiple neighbor feature points, direction vectors and reprojection errors of the feature point. Furthermore, the depth value of the neighbor feature point of the feature point in the historical image is different from the depth value of the feature point in the corresponding historical image.

It should be noted that in this embodiment, the method of establishing the second pixel model of the matched feature points in the historical image can refer to the method of establishing the first pixel model mentioned above, and will not be described in detail here. Among them, a second pixel model of matching feature points in the next frame image is established based on the previous frame image in a sliding window containing at least two frames of historical images, and the feature points of the first frame image in the sliding window can be pre-initialized through Suppose, thus, the pixel model of the feature points in the first frame of the image in the sliding window is established based on the initialized preset feature points. After the newly added images in the sliding window, the feature points extracted from these newly added images are used. The pixel model updates the second pixel model of the matched feature points in the previous frame image in the sliding window.

Of course, the content in the second pixel model is consistent with the content in the first pixel model. For example, the first pixel model contains feature data of multiple neighbor feature points of the feature point in the current frame image, and also includes There is the reprojection error of the direction vector of the first pixel model and the feature points in the historical image, and the second pixel model contains feature data of multiple neighbor feature points of the feature points in the historical image, and also contains the second pixel The mean value of the reprojection error between the model's direction vector and the feature points in the historical image.

Based on this, in this embodiment, the first pixel model of the matched feature points in each current frame image is separately matched with the second pixel model of the feature points in each historical image that matches the matched feature points in the current frame image. Comparison is performed, that is, one-to-one comparison, as shown in Figure 6. Thus, the model comparison results corresponding to the matched feature points in each current frame image are obtained.

Specifically, in this embodiment, the first pixel model of the matched feature points in each current frame image is compared with the corresponding second pixel model, which may be the first pixel model of the matched feature points in each current frame image. The first pixel model is compared with the corresponding second pixel model respectively on each of multiple neighbor points, direction vectors and reprojection errors, and then the corresponding features of the matched feature points in each current frame image are obtained. The comparison results of each item of multiple neighbor feature points, direction vectors and reprojection errors are then obtained. Based on these comparison results, the model comparison results corresponding to the matched feature points in each current frame image are obtained.

Step 105: Based on the model comparison results, filter out the target feature points in the current frame image.

Among them, the target feature points are the corresponding spatial points in the current frame image and do not belong to the feature points of the moving object.

Specifically, in this embodiment, the model comparison result represents whether the spatial point corresponding to the matched feature point in the current frame image belongs to a moving object, and the model comparison result represents whether the matched feature point in the current frame image is matched with it. When the difference between the feature points in the historical image exceeds a certain standard, it means that the matched feature point in the current frame image is a feature point that moves relative to the feature point in the matching historical image, then it can be determined The spatial point corresponding to the matched feature point in the current frame image belongs to a moving object. If the model comparison result indicates that the difference between the matched feature point in the current frame image and its matching feature point in the historical image does not exceed According to certain standards, it can be characterized that the matched feature points in the current frame image are static feature points relative to the feature points in the matching historical images, and then the spatial points corresponding to the matched feature points in the current frame image can be determined. Do not belong to moving objects. Based on this, based on the model comparison results, the corresponding spatial points in the current frame image are eliminated and the feature points belonging to moving objects are eliminated, thereby filtering out the corresponding spatial points that do not belong to the target feature points of moving objects, that is, static Feature points.

Step 106: Obtain the current pose of the mobile device based on the reprojection error corresponding to the target feature point in the current frame image.

Specifically, in this embodiment, the reprojection error can be minimized to obtain the pose transformation matrix corresponding to the minimum reprojection error of the target feature point, where the reprojection error is based on the historical image of the target feature point that matches it. The three-dimensional coordinate value of the spatial point corresponding to the feature point in and the two-dimensional coordinate value of the target feature point in the current frame image are obtained. After that, the current pose of the mobile device can be obtained according to the pose transformation matrix.

Furthermore, in this embodiment, based on the initially estimated current pose of the mobile device, the initially estimated current pose can be used as the optimization initial value, and the reprojection error corresponding to the target feature point in the current frame image can be performed. Minimize, and then obtain the corresponding pose transformation matrix when the reprojection error of the target feature point is minimum, and then obtain a more accurate current pose of the mobile device based on the pose transformation matrix.

It can be seen from the above solution that in the pose acquisition method provided in Embodiment 1 of the present application, after the current frame image is collected by the image acquisition device on the mobile device, the history contained in the current frame image and the previous sliding window is compared. The feature points that match the feature points in the image are obtained, and then after obtaining the pixel models corresponding to the matched feature points in the current frame image, the pixel model is combined with the features in the historical images included in the sliding window. The pixel models of the points are compared, and then based on the model comparison results, the corresponding spatial points in the current frame image can be screened out and the target feature points that do not belong to the moving object can be selected based on the reprojection error of these target feature points. Get the current pose of the mobile device. It can be seen that in this embodiment, the pixel model of the matching feature points in the front and rear images is used to eliminate the feature points belonging to the moving object, so as to realize the positioning of the mobile device based on the filtered feature points that do not belong to the moving object. , can avoid the interference of the feature points of moving objects on pose acquisition, thereby improving the positioning accuracy of mobile devices in dynamic scenes.

In order to further improve the positioning accuracy, when obtaining the current pose of the mobile device based on the reprojection error corresponding to the target feature point in the current frame image in step 106, the following steps can be implemented, as shown in Figure 7:

Step 701: Obtain the weight value corresponding to the target feature point according to the depth value of the spatial point corresponding to the target feature point.

Among them, the depth value of the spatial point corresponding to the target feature point can be obtained according to the depth value in the depth image corresponding to the current frame image where the target feature point is located. Based on this, the weight value is obtained based on the depth value. For example, the depth value of the spatial point corresponding to the i-th target feature point is represented by ω _i , where, λ is a scale constant, and _di is the depth value of the spatial point corresponding to the feature point matched by the i-th target feature point in the historical image in the depth image corresponding to the current frame image. This is because moving objects that interfere with positioning are usually located closer to the mobile device, and the position of moving objects farther away in a shorter period of time on the pixel plane is relatively small. Therefore, this implementation In the example, a weight value is configured for the target feature point based on the depth value of the spatial point corresponding to the target feature point, thereby improving the accuracy of subsequent pose acquisition.

Step 702: According to the weight value corresponding to the target feature point, obtain the pose transformation matrix corresponding to the minimum reprojection error of the target feature point.

Among them, the reprojection error is obtained based on the corresponding three-dimensional coordinate value of the target feature point in the historical image and the two-dimensional coordinate value of the target feature point in the current frame image. For example, in this embodiment, the depth values of the spatial points corresponding to the target feature points in the historical images are obtained based on the depth images corresponding to the historical images, and then the three-dimensional coordinate values of these spatial points are obtained, and then the initial pose transformation matrix is used to convert the three-dimensional coordinate values Convert to the current coordinate system corresponding to the current frame image, then project the two-dimensional coordinate values of these spatial points in the current coordinate system, and then compare these two-dimensional coordinate values with the two-dimensional coordinate values of the corresponding feature points matched in the current frame image. Error calculation is performed on the coordinate values to obtain the reprojection error. Based on this, in this embodiment, the corresponding reprojection error is weighted according to the weight value corresponding to the target feature point to obtain the optimal pose transformation matrix.

Step 703: Obtain the current pose of the mobile device according to the pose transformation matrix.

In this embodiment, a preset positioning algorithm can be used to calculate the current pose of the mobile device based on the pose transformation matrix.

Specifically, in this embodiment, the depth value of the feature points in the historical image that matches the target feature point can be first obtained based on the depth image corresponding to the historical image, and then the depth values of the feature points in the historical image that match each target feature point can be restored. The three-dimensional coordinate value of the feature point in space. Based on this, the three-dimensional coordinate value of the feature point in the historical image that matches each target feature point and the two-dimensional coordinate value of each target feature point in the current frame image can be composed. Three-dimensional-two-dimensional point pairs, that is, 3D-2D point pairs. Based on this, the reprojection error equation and the corresponding pose transformation matrix are constructed to represents, among them, F(·) is the Huber kernel function. Its function is that when a feature point mismatch occurs, the reprojection error will be very large. The kernel function can limit the growth of the reprojection error, and because the Huber kernel function is Smooth, convenient for derivation, and conducive to minimizing the solution of the reprojection error equation. π(·) is a projection transformation based on an image acquisition device such as a camera model. It can change the 3D coordinates in the camera coordinate system into the 2D pixel coordinates in the camera image. i is the serial number of the 3D-2D point pair. There are n pairs in total, u _i =[ _xi ,y _i ] ^T is the two-dimensional coordinates of the 2D point in the pixel plane, that is, the current frame image, in the 3D-2D point pair, P _i =[X _i ,Y _i ,Z _i ] ^T is the 3D-2D point Center the 3D point in space, that is, the three-dimensional coordinates of the corresponding spatial point in the historical image.

Based on this, when minimizing the function of the reprojection error, the most appropriate pose transformation matrix can be obtained. Specifically, in this embodiment, the EPNP algorithm can be used to perform nonlinear optimization on the reprojection error function, thereby obtaining the pose transformation matrix. Based on this, in this embodiment, using the initially estimated current pose of the mobile device as the initial value, the pose transformation matrix can be used to obtain a more accurate current pose of the mobile device.

Further, in order to improve the accuracy of the first pixel model, in this embodiment, it is necessary to select the neighbor features that best characterize the characteristics of the feature points in the current frame image from the neighbor feature points of the matched feature points in the current frame image. Based on this, in this embodiment, when obtaining the first pixel model of each matched feature point in the current frame image in step 103, this can be achieved through the following steps, as shown in Figure 8:

Step 801: Taking the matched feature point in each current frame image as the center, obtain multiple neighbor feature points of the matched feature point in the current frame image.

Among them, the depth values of the matched feature points in the current frame image and their corresponding neighbor feature points are different. That is to say, the depth values of the neighbor feature points of the matched feature points in the current frame image are different from the depth values of the neighbor feature points in the current frame image as the center. The depth values of the matched feature points are different, and the neighbor feature points of the matched feature points in the current frame image have depth values greater than the target depth and the distance between them and the matched feature points in the current frame image satisfies the distance sorting rule. Characteristically, the target depth is related to the depth average of the 24 neighborhood pixels of the matched feature point in the current frame image.

Taking the selection of N neighbor feature points as an example, the specific screening method is as follows:

First, taking the matched feature point in each current frame image as the center, obtain the pixel points within the 24 neighborhoods or 8 neighborhoods of the matched feature points in the current frame image, as shown in Figure 9;

Then, calculate the mean of the depth values of the pixels in the 24 neighborhoods or 8 neighborhoods of the matched feature point p′ in the current frame image, to means, for example, /> d _i is the depth value of the pixels in the 24-neighborhood; based on this, filter out the depth values greater than the target depth, such as/> pixels;

After that, the distance between the matched feature point p′ in the current frame image and each neighbor feature point is calculated, such as the Euclidean distance, and the depth value is greater than And the top N neighbor feature points ranked in ascending order of distance from the matched feature point p′ in the current frame image are used as neighbor feature points for establishing the first pixel model.

The above filtering scheme can exclude the situation where the feature point p′ matched in the current frame image belongs to the same rigid body, that is, the neighbor feature points of the same object are also added to the establishment of the first pixel model. This is because in the case of close depth , the pixel points are very likely to be in the same rigid body. Even if the motion changes, the relationship between them will not change much. Therefore, in this embodiment, the feature points eliminated in this embodiment may belong to the same rigid body as the matched feature points in the current frame image. Neighbor feature points can improve the accuracy of the first pixel model.

Step 802: Establish a first pixel model of the matched feature point in the current frame image based on at least a plurality of neighbor feature points of the matched feature point in the current frame image.

Specifically, in this embodiment, a first pixel model can be established based on the feature data of the multiple neighbor feature points. The first pixel model can include the feature data of each of the multiple neighbor feature points, and also includes the multiple neighbor features. The direction vector corresponding to the feature point and the reprojection error of the matched feature point in the current frame image.

Further, when comparing the first pixel model of the matched feature point in each current frame image with the corresponding second pixel model in step 104, the following steps can be implemented, as shown in Figure 10 :

Step 1001: Obtain the number of matching neighbor feature points in the first pixel model and its corresponding second pixel model.

Wherein, matching neighbor feature points means that the difference value between the corresponding neighbor feature points in the first pixel model and its corresponding second pixel model is less than or equal to the difference threshold. The first pixel model {p′ ₁ , p′ ₂ , p′ ₃ …p′ _N , n′, r′} of the matched feature point p′ in the current frame image and the matched feature point p in the historical image The second pixel model of For example, in this embodiment, the neighbor feature points in the first pixel model and the corresponding neighbor feature points in the second pixel model can be compared respectively to obtain one or more dimensions such as distance, pixel value, and depth value. difference values, when the difference value is less than the corresponding difference threshold, it is considered that the neighbor feature points in the first pixel model and the corresponding neighbor feature points in the second pixel model are similar, that is, they match, and if the difference If the value is greater than or equal to the corresponding difference threshold, then it is considered that the neighbor feature points in the first pixel model do not match the corresponding neighbor feature points in the second pixel model.

Based on this, in this embodiment, the number of matching neighbor feature points is obtained, that is, the number of neighbor feature points in the first pixel model whose difference value is less than or equal to the difference threshold in the corresponding second pixel model.

Step 1002: Obtain the module length of the difference between the direction vector in the first pixel model and its corresponding direction vector in the second pixel model.

Among them, when the moving object moves perpendicular to the camera plane, the similarity between the neighbor feature points in the two pixel models will be extremely high, that is, the difference value will be extremely low. Therefore, in order to further improve the accuracy , in this embodiment, the contrast condition of the direction vector is added. Based on this, in this embodiment, it is necessary to obtain the modulus length of the difference between the direction vectors of the two pixel models, represented by ||nn′|| ₂ , n is the th Direction vector in a two-pixel model.

Step 1003: Determine whether the reprojection error of the matched feature points in the current frame image in the first pixel model is less than or equal to the mean value of the reprojection error corresponding to the matched feature points in the historical image in the second pixel model to obtain critical result.

Among them, the mean value is the mean value of the cumulative reprojection error of the matched feature points in the historical image in the sliding window.

It should be noted that, in order to compare the possibility that the feature points are dynamic points through the accumulation of reprojection errors, the mean value in this embodiment is means, m is the number of historical images in the sliding window, /> is the corresponding reprojection error of the matched feature point p in the historical image in the i-th frame of the historical image in the sliding window. Based on this, in this embodiment, the reprojection error r _p′ of the matched feature points in the current frame image in the first pixel model is compared with the mean value/> A size judgment is performed to obtain a judgment result of whether the mean value is greater than the reprojection error of the matched feature points in the current frame image in the first pixel model.

Among them, the execution order between step 1002, step 1003 and step 1001 can be changed. For example, step 1002 can be executed first, and then other steps can be executed, or three steps can be executed at the same time. The different technical solutions obtained belong to the same inventive concept. All are within the protection scope of this application.

Step 1004: Obtain model comparison results based on the quantity, module length and judgment results.

Among them, when the number exceeds the preset value, the module length is less than the preset value, and the reprojection error is less than the mean value, the obtained model comparison result indicates that the spatial point corresponding to the matched feature point in the current frame image does not belong to motion. Object, i.e. static point.

That is to say, when at least the number exceeds a preset value such as 3 or 5, in order to further improve the accuracy, it is determined whether the modulus length of the direction vector difference is within a certain range, such as less than the preset value δ and whether the reprojection error is less than the mean to obtain model comparison results.

For example, the model comparison results are obtained based on whether the number exceeds a preset value, whether the module length of the direction vector difference is within a certain range, and whether the reprojection error is less than the mean:

If the number exceeds the preset value, the module length of the direction vector difference is within a certain range and the reprojection error is less than the corresponding mean value within the sliding window, then it can be determined that the spatial point corresponding to the matched feature point in the current frame image is static. Points are model comparison results that do not belong to moving objects;

If the number does not exceed the preset value, or the module length of the direction vector difference exceeds the preset range, or the reprojection error is greater than the corresponding mean value within the sliding window, then the corresponding feature points matched in the current frame image can be determined. The spatial points belong to dynamic points, that is, the model comparison results of moving objects.

That is to say, among the above three conditions: the number exceeds the preset value, the module length of the direction vector difference is within a certain range, that is, ||nn′|| ₂ <δ, and the reprojection error is less than the corresponding mean value within the sliding window, that is, Only when all three conditions are met, the spatial point corresponding to the matched feature point in the current frame image can be determined to be a static point. As long as one condition is not met, then the spatial point corresponding to the matched feature point in the current frame image can be determined. is a dynamic point.

In a specific implementation, each feature point in the historical image within the sliding window has a pixel model. In this embodiment, the pixel model of the feature point in the historical image that matches the feature point matched in the current frame image is recorded as the third The two-pixel model is used to distinguish the first pixel model of the feature point matched in the current frame image from the pixel model of the feature point that does not match the feature point.

Based on this, further, in order to increase the diversity of feature points in the pixel model of each feature point in the historical image in the sliding window, and ensure that the feature points in the pixel model of the old image in the sliding window where the historical image is located are not easily replaced, In this embodiment, the pixel model in the historical image can be updated, as shown in Figure 11:

Step 1101: Obtain an update request for the pixel model of the feature point in the historical image.

The update request can be automatically generated or generated by user operation, and the update request can include the identification of one or more feature points in the historical image that need to be updated.

Step 1102: Obtain the newly added image in the sliding window.

Among them, the newly added image can be the current frame image that has recently undergone pose positioning processing and is a key frame image. At this time, the current frame image has been added to the sliding window as a historical image and is recorded as a newly added image. It should be noted that when the current frame image is a key frame image, the current frame image will be added to the sliding window.

It should be noted that the newly added image is a key frame image that has been positioned and processed. When a new image is obtained and used as a new current frame image, the key frame image that has been positioned is added to the sliding window as a historical image. .

Step 1103: Extract feature points from the newly added image to obtain feature points in the newly added image.

Among them, various feature points can be extracted from the newly added image through feature extraction algorithms and other methods. Each feature point in the newly added image may have matching feature points in the historical image. For example, there are features in the historical image. The points match the descriptor with the feature points in the newly added image. It is also possible that each feature point in the newly added image does not have a matching feature point in the historical image.

Step 1104: Match the feature points in the newly added image with the feature points in the historical image in the sliding window to obtain the matching feature points in the newly added image that match the feature points in the historical image and the newly added Non-matching feature points in the image that do not match any feature points in the historical image.

Among them, the matching feature points refer to the feature points in the newly added image that match the feature points in the historical image, and the non-matching feature points refer to the feature points in the newly added image that do not match the feature points in the historical image. .

Specifically, in this embodiment, the feature points in the newly added image can be matched with the feature point ORB descriptors in the historical images in the sliding window to obtain matching feature points and non-matching feature points.

Step 1105: According to the matching feature points, update the pixel model of the feature points in the historical image that match the matching feature points.

Specifically, in this embodiment, a pixel model can be first established for the matching feature points. The pixel model of the matching feature points corresponds to multiple neighbor feature points, direction vectors and reprojection errors of the matching feature points. After that, in this embodiment, using The neighbor feature points in the pixel model of the matching feature point update the pixel model of the feature point in the historical image that matches the matching feature point.

For example, for the second pixel model of the feature point p in the historical image The matching feature point p ^F is found in the newly added image F within the sliding window. This feature point has a corresponding pixel model M(p ^F ). For/> Each point in has the same probability of being updated, and the update probability is 1/N; after that, the feature point with the smallest distance from p ^F , such as the smallest Euclidean distance, can be selected/> The second pixel model M(p) is updated.

Based on this, after the pixel feature points in the second pixel model are updated, the direction vector and the reprojection error in the second pixel model can be recalculated. The updated second pixel model can be used for model comparison, thereby obtaining more accurate model comparison results and improving the accuracy of pose acquisition.

Step 1106: Obtain the pixel model of the non-matching feature point, so that when the newly added image is used as the historical image in the sliding window, the pixel model of the non-matching feature point is used as the pixel model of the new feature point in the historical image.

That is to say, after the newly added image is added to the sliding window, the newly added image has been used as a historical image in the sliding window, so the pixel model of the non-matching feature points in the newly added image is directly used as the feature point in the historical image. pixel model. Based on this, when using the new current frame image to locate the current pose of the mobile device, the feature points extracted from the new current frame image are added to the sliding window with the feature points extracted from the historical images containing the new image. Matching is performed. After matching the feature points in the new current frame image that match the feature points in the historical image, the pixel model of the matched feature points in the new current frame image is matched with the features in the historical image. The pixel models of the points are compared, and then the target feature points in the new current frame image are screened out, that is, static points, and then the current pose of the mobile device is accurately estimated based on the reprojection errors corresponding to these static points.

It should be noted that the execution order of steps 1105 and 1106 is not limited to the order shown in the drawings. For example, step 1106 may be executed first, and then step 1105 may be executed, or steps 1105 and 1106 may be executed simultaneously, and steps 1105 and 1106 may be executed simultaneously. Different technical solutions formed by different execution orders of step 1106 belong to the same inventive concept and are all within the protection scope of this application.

In order to further improve the accuracy, in this embodiment, after filtering out the target feature points in the current frame image according to the model comparison result in step 105, in step 106 based on the target feature points in the current frame image Before obtaining the corresponding reprojection error and the current pose of the mobile device, the method in this embodiment may also include the following steps, as shown in Figure 12:

Step 107: From the target feature points in the current frame image, select the target feature points that satisfy the epipolar constraints with the feature points in the historical image, and then perform step 106.

That is to say, considering that there may be missed detections or no matching feature points were found in the previous processing, this application introduces the constraints of multi-view geometry for processing, and then proposes the feature points that do not satisfy the epipolar constraints among the target feature points, Only the target feature points that satisfy the epipolar constraints are retained, thereby improving the accuracy of subsequent pose acquisition.

Referring to Fig. 13, which is a schematic structural diagram of a posture acquisition device provided in Embodiment 2 of the present application. The device can be configured in an electronic device capable of processing images, such as a mobile device with an image acquisition device, such as a mobile robot, etc. , or it can be a device connected to the mobile device, such as a computer or server. The technical solution in this embodiment is used to improve the accuracy of obtaining the pose of the mobile device.

Specifically, the device in this embodiment may include the following units:

The image acquisition unit 1301 is used to obtain the current frame image, which is an image collected by the image acquisition device on the mobile device;

The feature point processing unit 1302 is used to obtain the matched feature points in the current frame image, where the matched feature points in the current frame image are historical images included in the sliding window corresponding to the current frame image. There are feature points with matching feature points in the sliding window; the sliding window contains multiple frames of historical images, and the historical images are key frame images before the current frame image;

The model building unit 1303 is configured to obtain a first pixel model of each feature point matched in the current frame image, where the first pixel model corresponds to a plurality of feature points matched in the current frame image. Neighbor feature points, and the first pixel model also has the direction vector of the first pixel model and the reprojection error of the matched feature points in the current frame image;

The model comparison unit 1304 is configured to compare the first pixel model of the matched feature point in each current frame image with the corresponding second pixel model to obtain a model comparison result. The result represents whether the spatial point corresponding to the matched feature point in the current frame image belongs to a moving object; wherein, the second pixel model is the corresponding feature point in the historical image and the matched feature point in the current frame image. A pixel model of the matched feature point, the second pixel model corresponds to a plurality of neighbor feature points of the matched feature point in the historical image, and the second pixel model also has the direction of the second pixel model The vector and the mean value of the reprojection error of the matched feature points in the historical image;

The feature point screening unit 1305 is configured to screen out target feature points in the current frame image according to the model comparison results. The target feature points are corresponding spatial points in the current frame image that do not belong to moving objects. Feature points;

The pose obtaining unit 1306 is configured to obtain the current pose of the mobile device according to the reprojection error corresponding to the target feature point in the current frame image.

It can be seen from the above solution that the pose acquisition device provided in Embodiment 1 of the present application, after the current frame image is collected by the image acquisition device on the mobile device, compares the current frame image with the historical images contained in the previous sliding window. The feature points that match the feature points in the current frame image are obtained, and then after obtaining the pixel models corresponding to multiple neighbor feature points of the feature points in the current frame image, the pixel model is compared with the historical images included in the sliding window. The pixel models of the feature points are compared, and then based on the model comparison results, the corresponding spatial points in the current frame image can be filtered out and the target feature points that do not belong to the moving object can be screened out. Based on this, the reprojection of these target feature points can be carried out. error to obtain the current pose of the mobile device. It can be seen that in this embodiment, the pixel model of the matching feature points in the front and rear images is used to eliminate the feature points belonging to the moving object, so as to realize the positioning of the mobile device based on the filtered feature points that do not belong to the moving object. , can avoid the interference of the feature points of moving objects on pose acquisition, thereby improving the positioning accuracy of mobile devices in dynamic scenes.

In one implementation, the pose obtaining unit 1306 is specifically configured to: obtain the weight value corresponding to the target feature point according to the depth value of the spatial point corresponding to the target feature point; value, obtain the pose transformation matrix corresponding to the minimum reprojection error of the target feature point, and the reprojection error is based on the corresponding three-dimensional coordinate value of the target feature point in the historical image and the location of the target feature point. The two-dimensional coordinate values in the current frame image are obtained; and the current posture of the mobile device is obtained according to the posture transformation matrix.

In one implementation, the model building unit 1303 is specifically configured to: take the matched feature points in each of the current frame images as the center, and obtain multiple neighbor features of the matched feature points in the current frame image. point; wherein, the feature point matched in the current frame image and its corresponding neighbor feature point have a depth value greater than the target depth and the distance between the feature point matched in the current frame image satisfies the distance sorting rule. Feature points, the target depth is related to the depth mean value of neighbor pixels of the matched feature points in the current frame image; at least based on multiple neighbor feature points of the matched feature points in the current frame image, a The first pixel model of the matched feature point in the current frame image, so that the first pixel model corresponds to a plurality of neighbor feature points of the matched feature point in the current frame image and also has the third The direction vector of a pixel model and the reprojection error of the matched feature points in the current frame image.

Wherein, the direction vector of the first pixel model is based on the three-dimensional coordinate value of the spatial center point corresponding to the neighbor feature point in the first pixel model and the three-dimensional coordinate value of the spatial point corresponding to the matched feature point in the current frame image. The coordinate values are obtained, and the reprojection error of the matched feature points in the current frame image is obtained based on the historical image.

Based on this, the model comparison unit 1304 is specifically configured to: obtain the number of matching neighbor feature points in the first pixel model and its corresponding second pixel model; obtain the direction vector in the first pixel model and its corresponding second pixel model. The module length of the difference between the corresponding direction vectors in the second pixel model; determine whether the reprojection error of the matched feature point in the current frame image in the first pixel model is less than or equal to that of the third pixel model. The mean value of the reprojection error corresponding to the matched feature points in the historical image in the two-pixel model is used to obtain the judgment result; the mean value is the cumulative reprojection of the matched feature points in the historical image in the sliding window The mean value of the error; according to the quantity, the module length and the judgment result, the model comparison result is obtained.

In an implementation manner, the device in this embodiment may also include the following units, as shown in Figure 14:

The model update unit 1307 is used to: obtain an update request for the pixel model of the feature points in the historical image; obtain a newly added image within the sliding window; perform feature point extraction on the newly added image to Obtain the feature points in the newly added image; match the feature points in the newly added image with the feature points in the historical images in the sliding window to obtain the corresponding features in the newly added image. matching feature points that match the feature points in the historical image and non-matching feature points in the newly added image that do not match the feature points in the historical image; based on the matching feature points, the historical image The pixel model of the feature point in the image that matches the matching feature point is updated; the pixel model of the non-matching feature point is obtained, so that when the newly added image is used as a historical image in the sliding window, The pixel model of the non-matching feature point is used as the pixel model of the new feature point in the historical image.

In one implementation, before obtaining the current pose of the mobile device according to the reprojection error corresponding to the target feature point in the current frame image, the feature point filtering unit 1305 is also configured to: in the current frame Among the target feature points in the image, target feature points that satisfy the epipolar constraint with the feature points in the historical image are screened out.

In one implementation, the feature point processing unit 1302 is specifically configured to: extract feature points in the current frame image; combine the feature points extracted in the current frame image with the feature points extracted in the historical image. Feature points are matched to obtain feature points in the current frame image that match feature points in the historical image.

In one implementation, the feature point processing unit 1302 is further configured to: perform reprojection error calculation based on the matched feature points in the current frame image, and initially estimate the current pose of the mobile device based on the calculated reprojection error.

It should be noted that the specific implementation of each unit in this embodiment can refer to the corresponding content in the foregoing text and will not be described in detail here.

Referring to Figure 15, a schematic structural diagram of a mobile device is provided in Embodiment 3 of the present application. The mobile device can be an electronic device capable of processing images, such as a mobile device with an image acquisition device, such as a mobile robot, etc., or it can It is a device connected to a mobile device, such as a computer or server. The technical solution in this embodiment is used to improve the accuracy of obtaining the pose of the mobile device.

Specifically, the mobile device in this embodiment at least includes the following structures:

Memory 1501, used to store application programs and data generated by running the application programs;

Processor 1502, used to execute the application program to implement:

Obtain a current frame image, which is an image collected by an image acquisition device on the mobile device;

Obtain the matched feature points in the current frame image, wherein the matched feature points in the current frame image are those with matching feature points in the historical images included in the sliding window corresponding to the current frame image. Feature points; the sliding window contains multiple frames of historical images, and the historical images are key frame images before the current frame image;

A first pixel model of each matched feature point in the current frame image is obtained respectively, the first pixel model corresponds to a plurality of neighbor feature points of the matched feature point in the current frame image, and the The first pixel model also has a direction vector of the first pixel model and a reprojection error of the matched feature points in the current frame image;

The first pixel model of the matched feature point in each of the current frame images is compared with the corresponding second pixel model to obtain a model comparison result, and the model comparison result characterizes the current frame image. Whether the spatial point corresponding to the matched feature point belongs to a moving object; wherein, the second pixel model is a pixel model of the feature point in the historical image that matches the feature point matched in the current frame image, The second pixel model corresponds to a plurality of neighbor feature points of the matched feature points in the historical image, and the second pixel model also has a direction vector of the second pixel model and a matching feature point in the historical image. The mean value of the reprojection error of the feature points;

According to the model comparison result, target feature points in the current frame image are screened out, and the target feature points are feature points whose corresponding spatial points in the current frame image do not belong to moving objects;

According to the reprojection error corresponding to the target feature point in the current frame image, the current pose of the mobile device is obtained.

It can be seen from the above technical solutions that a mobile device according to Embodiment 3 of the present application, after the current frame image is collected by the image acquisition device on the mobile device, compares the current frame image with the historical images contained in the previous sliding window. The feature points that match the feature points in the current frame image are obtained, and then after obtaining the pixel models corresponding to multiple neighbor feature points of the feature points in the current frame image, the pixel model is compared with the historical images included in the sliding window. The pixel models of the feature points are compared, and then based on the model comparison results, the corresponding spatial points in the current frame image can be filtered out and the target feature points that do not belong to the moving object can be screened out. Based on this, the reprojection of these target feature points can be carried out. error to obtain the current pose of the mobile device. It can be seen that in this embodiment, the pixel model of the matching feature points in the front and rear images is used to eliminate the feature points belonging to the moving object, so as to realize the positioning of the mobile device based on the filtered feature points that do not belong to the moving object. , can avoid the interference of the feature points of moving objects on pose acquisition, thereby improving the positioning accuracy of mobile devices in dynamic scenes.

Wherein, when the electronic device is a mobile device, the electronic device also includes an image collection device, such as a camera.

It should be noted that the specific implementation of the processor in this embodiment can refer to the corresponding content in the foregoing text and will not be described in detail here.

Taking the positioning of robots as an example, the following is a detailed example of how to implement SLAM using the technical solution of this application:

First of all, the inventor of this application found the following defects in the process of using SLAM for robot positioning:

In the visual SLAM framework used, it is necessary to assume that the environment is a static environment, or a quasi-static scene in which static features occupy the majority of the environment. Whether it is a visual SLAM system based on the direct method or the feature point method, it is difficult to deal with the interference of moving objects, and during the SLAM operation process, if the solution to the camera pose completely relies on minimizing the energy function or random sampling, With the linear RANSAC method, when static feature points do not occupy the majority in the entire image, or the proportion is low, it is often difficult to obtain the correct camera motion model in a limited number of iterative calculations;

The inventor of this application continued to test the positioning solution based on the above defects, and also found that: for the SLAM algorithm based on the inertial measurement unit IMU (Inertial Measurement Unit) to solve the dynamic environment, this method does not essentially solve the dynamic environment. The problem is that it only increases the accuracy of feature point matching by providing a motion prior to the system, but it is still possible for a large number of mismatches to occur in scenes with large environmental similarities; and adding additional sensors will increase the complexity of the positioning algorithm, increasing Sensor hardware cost;

Since dynamic objects are not necessarily moving objects, if predefined dynamic objects are blindly excluded, it will easily cause the information in the image to be greatly reduced. However, using the current semantic dynamic SLAM is to identify dynamic objects and eliminate them. This This will make its applicability even worse in scenarios where feature points are sparse, and the introduction of a deep learning network will require very high computing power of the system. On a hardware platform that does not include a graphics processor GPU (Graphics Processing Unit), this It is difficult for the algorithm to achieve real-time performance. If a GPU is added, the hardware cost will increase, which will greatly limit the product application. On the other hand, since the target detection algorithm of deep learning requires a large amount of data for pre-training, this will also limit the exploration of unknown environments by robots based on the visual SLAM positioning algorithm to a certain extent;

The inventor of this application also found that: for an algorithm based on motion segmentation, it can simultaneously estimate the motion of the camera and rigid dynamic objects in the scene. This method does not rely on the semantic information of the image, by clustering points with the same motion into A motion model performs motion estimation to segment different moving rigid bodies in dynamic scenes. However, this method has high requirements for scene conditions. When there are too few measurable 3D point depths in the scene, the error in camera and dynamic object motion estimation will be large, and due to the inaccuracy of clustering segmentation, it will cause certain problems. A cluster contains multiple kinematic rigid bodies.

Therefore, in order to solve the problem of reduced accuracy of traditional visual SLAM positioning algorithms in dynamic environments, the inventor of this application proposed a method based on feature point modeling and multi-view geometry to eliminate dynamic areas and abnormal feature points in the scene. method. This method takes visual SLAM as the main body and establishes a model for feature points in a sliding window. By comparing the model with the current frame, it can effectively eliminate the feature points corresponding to the dynamic objects in the scene on the image, and realize the camera in a dynamic environment. Robust estimation of pose. details as follows:

Combined with the technical framework of this application shown in Figure 16, this application can be divided into five parts: calculating a rough pose transformation matrix based on minimizing the reprojection error, dynamic area judgment, feature point modeling, and epipolar constraint judgment. Outer points and precise pose estimation. These five parts are introduced in detail below:

1) Calculate pose estimation based on minimizing reprojection error

The pose estimation flow chart is shown in Figure 17. This application adopts a coarse-to-fine approach to pose estimation. The pose estimation in this step is a rough pose estimation, and the core is to calculate the motion of the 3D-2D point pair by constructing a reprojection error. The steps and principles of this part of the algorithm are as follows:

(1) Extract feature points in the current frame image. Feature points are representative points in the image and are another digital expression of image information. In the invention, we extract ORB features from images, which consist of key points and descriptors. The key points are called "Oriented FAST", which is an improved FAST corner point with scale and rotation invariance. The descriptor uses a binary BRIEF descriptor. During the feature extraction process, the feature points may be concentrated in one area due to the rich texture in one part of the image, causing the pose estimation to tend to local motion. In order to improve the accuracy of the system, we increase the uniformity of feature extraction: divide the image into S*S small squares, and extract feature points for each square so that the image features are distributed throughout the entire image field of view as much as possible.

(2) Match the feature points extracted from the current frame with the corresponding feature points in the previous key frame. Here, DBOW2 bag-of-word technology is used to match the feature points extracted from the current frame with the feature points in the previous key frame. Accelerate matching is performed. After the matching is completed, it is judged whether the number of matching feature points reaches the required number. If it is reached, proceed directly to the next step to construct the reprojection error and estimate the camera pose. If the requirements are not met, the matching conditions are relaxed, and the DBOW2 bag-of-words technology is used again to accelerate the matching of the feature points extracted from the current frame and the local map points constructed from the previous frame.

(3) Use the depth image data input by the sensor of the previous key frame to obtain the depth value of the feature point of the previous key frame and restore the 3D coordinates of the feature point. The previous 3D points and the current frame 2D pixels form a 3D-2D point pair, and the reprojection error equation is constructed as follows:

Among them, ω _i is the predefined weight, and its calculation formula is: λ is a scale constant, and _di is the depth value of the 3D point in the i-th feature point pair. The addition of this weight is a novel angle. It is based on two considerations: generally in indoor scenes, moving targets are located closer to the robot; the displacement of pixel planes caused by the movement of distant moving 3D points in a short period of time is negligible. F(·) is the Huber kernel function. Its function is that when mismatching of feature points occurs, the reprojection error will be very large. The kernel function can limit the growth of the reprojection error, and because the Huber kernel function is smooth, it is convenient Derivative is helpful for solving the reprojection error equation. π(·) is a projection transformation based on the camera model, which can change the 3D coordinates in the camera coordinate system into the 2D pixel coordinates in the camera image. u _i =[u _i ,vi _] ^T is the coordinate of the 2D point in the 3D-2D point pair in the pixel plane, P _i =[X _i ,Y _i ,Z _i ] ^T is the 3D point in the 3D-2D point pair in coordinates in space. Finally, the EPNP algorithm is used to perform nonlinear optimization to obtain a preliminary estimate of the current pose of the robot.

2) Feature point modeling

This step is the core part of this application. It mainly establishes a staged model for feature points, that is, the pixel model mentioned above. Because the model will be replaced and updated within a sliding window, it is called staged. The flow chart is shown in Figure 18 shown. The purpose of establishing the model is to judge whether the feature points are dynamic by comparing the feature points of the current frame with the matching feature point model, and by comparing the differences between the feature points of the current frame and the feature point model.

The establishment and initialization of the model are as follows: First, for each feature point p, that is, the feature point in the historical image, establish its pixel model M(p), that is, the second pixel model, where, take Among them,/> are the depth-inconsistent neighbor feature points of feature point p in space, and N is the number of selected feature points. The selection of these neighbor feature points is not randomly selected. First, it is necessary to calculate the mean depth value of the pixels in the 24 neighborhoods of the feature point p/> Among them,/> Then calculate the Euclidean distance between the feature point p and other feature points, and set the depth value greater than/> The top N feature points ranked in ascending order by their Euclidean distance from the feature point p are regarded as depth-inconsistent neighbor feature points.

The purpose of this step is to exclude feature points that belong to the same rigid body as the feature point p from being added to the model. When the depth is close, the pixel points are most likely to be in the same rigid body. Even if the motion changes, the relationship between them Not much will change. n is the direction vector of the model, which is calculated by calculating the position of the spatial center point of the 3D map point corresponding to all neighbor feature points in the model. Then it is obtained by subtracting the position (x, y, z) of the corresponding 3D map point of the feature point p, that is/> r _p is the cumulative sum of reprojection errors,/> It is 0 during initialization, and m is the number of frames in the sliding window. The model in this step is established through previous frames, that is, frames within a sliding window, and the first frame is taken for initialization, and subsequent frames are used to update the model.

3) Dynamic feature point judgment and model update

Before judging the feature points of the current frame, it is also necessary to establish a pixel feature point model, that is, the first pixel model, for the valid feature points of the current frame (that is, the feature points with matching feature points in the historical image). In the above steps, first find the feature point p', that is, the N depth-inconsistent neighbor feature points of the feature point in the current frame image, then calculate the direction vector n' of the model, and finally add the reprojection error r _p calculated in the first step. _' , the inventor of this application can set three judgment conditions to judge whether a certain feature point of the current frame is static. If the following three conditions are met at the same time, the feature point is a static point:

(1) Using the result of feature point matching in step 1), find the feature point that matches the feature point p' of the current frame in the feature point pixel model, that is, the second pixel model (it can also be called a feature point model), Convert the neighbor feature points in the feature point model to Match with the neighbor feature points {p' ₁ , p' ₂ , p' ₃ ...p' _N } of the feature point p' in the previous frame. This condition is satisfied if the match is successful with more than the preset N _th feature points.

(2) Considering that dynamic objects move perpendicular to the camera plane, the depth-inconsistent neighbor feature points in the feature point pixel model will have extremely high similarity with the N depth-inconsistent neighbor feature points of the current frame feature point p'. Further, Add a direction vector comparison condition, that is, if the module length of the difference between the direction vector n in the feature point model and the direction vector n' in the feature point p' model of the current frame is within a certain range, the requirement that the feature point is static is met: | |nn′|| ₂ <δ.

(3) This condition can be regarded as an additional constraint, mainly to compare the possibility of feature points being dynamic points through the accumulation of reprojection errors. The reprojection error calculation here uses the result of the first step of calculation. The formula is expressed as m is the number of frames in the sliding window. What it means here is to find the mean value of the cumulative reprojection error of the feature point model midpoint p in the sliding window. If the reprojection error of the feature point p′ of the current frame is less than the feature point model midpoint p in By using the mean value of the reprojection error in the sliding window, we can be more confident that the feature point p′ is a static point, which satisfies this determination condition.

Further, as the robot acquires the image, the robot processes the positioned current frame image as a historical image, such as moving it to a sliding window, and collects a new current frame image at the same time. Based on this, when there is an update to the historical image in the sliding window, the model of the static feature point p in the historical image in the sliding window, that is, the second pixel model, can be updated. A random update strategy is adopted in this application. The main purpose is to ensure that the model pixels generated by old frames in the sliding window are not easily replaced, and to increase the diversity of pixel feature points in the model:

First, for the feature point model of feature point p in the sliding window If a matching feature point p ^F is found among the feature points of the current frame, and the feature point is judged to be a static feature point, and the feature point model M(p ^F ) established by the feature point, based on this, for the sliding window model of feature points Each point in , is given the same probability of updating, that is to say, every point has the same probability of being updated, and the update probability is/>

Next, as to which feature point model in the model M(p ^F ) is used to update, the update probability can be first defined, that is, the model M(p ^F ) only has the possibility of updating the model of the feature point p with probability eta. If the model M(p ^F ) can update the model of the feature point p, select the pixel feature point with the smallest Euclidean distance from p ^F Update model M(p). Similarly, when the pixel feature points in the model M(p) are updated, the direction vector n of the model can be recalculated and the reprojection error r _p can be updated. For the feature points of the current frame for which no matching point is found in the sliding window, this application considers adding them to the feature point model according to the initialization step to facilitate the next feature point matching and model comparison.

4) Epipolar constraints determine external points

After using the feature point model to perform dynamic point removal (filtering), considering that there may be missed detection or no matching feature points were found in the feature point model in the previous step, this application handles it by introducing the constraints of multi-view geometry. . This application is based on the fact that in static scenes, dynamic features will violate the standard constraints in multi-view geometry, as shown in Figure 19.

where x ₁ , x ₂ and x′ ₂ are three points on the pixel coordinates. If the 3D point P ₁ is a static point, then x ₁ and x ₂ are a pair of matching points, then they will satisfy the equation x ₂ Fx ₁ = 0, F is the basic matrix, which can be obtained through the camera motion rotation matrix R and translation vector t. l ₁ and l ₂ are called epipolar lines, so this geometric constraint is also called epipolar line constraint. If 3D point P ₁ is a dynamic point, that is, at the next moment, it moves to point P ₂ , then the projection of P ₂ on the camera plane becomes x′ _2. It is obvious that x ₁ and x′ ₂ do not satisfy the epipolar constraints, That is, x ₂ Fx ₁ ≠0, and is much greater than 0. This application uses this multi-view constraint to further eliminate missed dynamic 3D points.

5) Accurate pose estimation

This step is to locally optimize the feature points of the current frame that survived the previous steps, that is, the target feature points, and the static feature points in the local map. The optimization method is still by calculating the minimum value of the reprojection error function, so that the position of the current frame The pose estimation is optimal. The form of the reprojection error function is Among them, _Pi is the 3D point in the local map matched by the static feature point in the current frame, and u _i is the coordinate of the static feature point in the current frame on the pixel plane.

It can be seen that the inventor of this application proposed a concept of feature point modeling when dealing with positioning problems in dynamic environments, and eliminated dynamics by comparing feature points with the model. Moreover, when calculating the reprojection error function, the concept of pre-weighting is introduced, and the error value is calculated using the idea that far dynamic points will not cause large displacements in the pixel plane in consecutive frames and static points are more likely to be distant backgrounds. Pre-weighted processing.

Furthermore, this application proposes a method of modeling feature points with inconsistent depth. The idea is to use the surrounding environment information to model the feature points instead of modeling the information around the pixels. By combining the model with the feature points that need to be judged, Three conditional judgments can be made to effectively distinguish dynamic and static feature points. Moreover, in this application, a sliding window method is used to judge multiple consecutive frames, and feature points are judged through accumulated information, which avoids the shortcoming of inconspicuous motion between two short frames and effectively improves the static and dynamic differentiation of feature points. accuracy.

It can be seen that this application is different from the existing way of processing dynamic environments. It adopts the method of feature point modeling, using the information of feature points in time and space to establish a pixel model. In space, the depth-inconsistent neighbor feature points are used To represent pixels in time, a random update strategy is used to increase the amount of past information contained in the model. And three conditions are set to judge the feature points. Only the feature points that meet the three conditions at the same time can be judged as static points, and then these feature points are used to accurately estimate the pose. Furthermore, this application is a way of using the surrounding environment information of pixels. It is different from the pre-training and motion segmentation methods using deep learning. It does not require a prior understanding of the scene environment, nor does it need to conduct analysis on different moving rigid bodies in the scene. Segmentation reduces the need for hardware levels and is a highly practical SLAM algorithm for handling dynamic environments.

It should be noted that in terms of sensors, the method of filtering static feature points in this application is not only applicable to RGB-D cameras, but is also applicable to monocular and binocular cameras, except that RGB-D cameras can directly The sensor can obtain pixel depth information, and no additional algorithm is needed for calculation. Moreover, the pixel modeling method used in this application is based on the characteristic that the surrounding pixel feature points of dynamic feature points will also change. The feature points are modeled. If other solutions can be found to distinguish dynamic and static feature points as the The similarities and differences in time changes are still within the protection scope of this application. In addition, this application is based on a visual SLAM algorithm, but it is essentially based on environmental feature point sampling. However, it is also applicable to SLAM algorithms using laser sensors. You only need to find other feature points that model laser points. That's it, and then set the static and dynamic feature point judgment conditions based on the characteristics of the sampling points acquired by the laser sensor, distinguish dynamic and static feature points, and use the static points to estimate the robot's pose.

Each embodiment in this specification is described in a progressive manner. Each embodiment focuses on its differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple. For relevant details, please refer to the description in the method section.

Those skilled in the art may further realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. In order to clearly illustrate the possible functions of hardware and software, Interchangeability, in the above description, the composition and steps of each example have been generally described according to functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.

The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly in hardware, in software modules executed by a processor, or in a combination of both. Software modules may be located in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or anywhere in the field of technology. any other known form of storage media.

The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be practiced in other embodiments without departing from the spirit or scope of the application. Therefore, the present application is not to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. A pose acquisition method, characterized in that it includes: Obtain a current frame image, which is an image collected by an image acquisition device on the mobile device; Obtain the matched feature points in the current frame image, wherein the matched feature points in the current frame image are those with matching feature points in the historical images included in the sliding window corresponding to the current frame image. Feature points; the sliding window contains multiple frames of historical images, and the historical images are key frame images before the current frame image; A first pixel model of each matched feature point in the current frame image is obtained respectively, the first pixel model corresponds to a plurality of neighbor feature points of the matched feature point in the current frame image, and the The first pixel model also has a direction vector of the first pixel model and a reprojection error of the matched feature points in the current frame image. The first pixel model is to remove the depth values of the plurality of neighbor feature points. After obtaining the same redundant neighbor feature points, the remaining neighbor feature points are established based on the matched feature points in the current frame image; The first pixel model of the matched feature point in each of the current frame images is compared with the corresponding second pixel model to obtain a model comparison result, and the model comparison result characterizes the current frame image. Whether the spatial point corresponding to the matched feature point belongs to a moving object; wherein, the second pixel model is a pixel model of the feature point in the historical image that matches the feature point matched in the current frame image, The second pixel model corresponds to a plurality of neighbor feature points of the matched feature points in the historical image, and the second pixel model also has a direction vector of the second pixel model and a matching feature point in the historical image. The mean value of the reprojection error of the feature points, and the comparison result is that the first pixel model of the matched feature points in each current frame image and the corresponding second pixel model are in multiple neighbor points, direction vectors and Each item in the reprojection error is compared separately, and then the comparison results of multiple neighbor feature points, direction vectors and each item in the reprojection error corresponding to the matched feature points in each current frame image are obtained; According to the model comparison result, target feature points in the current frame image are screened out, and the target feature points are feature points whose corresponding spatial points in the current frame image do not belong to moving objects; According to the reprojection error corresponding to the target feature point in the current frame image, the current pose of the mobile device is obtained. 2. The method of claim 1, wherein obtaining the current pose of the mobile device according to the reprojection error corresponding to the target feature point in the current frame image includes: Obtain the weight value corresponding to the target feature point according to the depth value of the spatial point corresponding to the target feature point; According to the weight value corresponding to the target feature point, the pose transformation matrix corresponding to the minimum reprojection error of the target feature point is obtained. The reprojection error is based on the three-dimensional corresponding three-dimensional image of the target feature point in the historical image. The coordinate value and the two-dimensional coordinate value of the target feature point in the current frame image are obtained; According to the pose transformation matrix, the current pose of the mobile device is obtained. 3. The method according to claim 1, characterized in that, respectively obtaining the first pixel model of the matched feature points in each of the current frame images includes: Using the matched feature points in each current frame image as the center, obtain multiple neighbor feature points of the matched feature points in the current frame image; Wherein, the depth value of the matched feature point in the current frame image is different from that of its corresponding neighbor feature point, and the depth value of the neighbor feature point of the matched feature point in the current frame image is greater than the target depth and is different from the target depth. The distance between the matched feature points in the current frame image satisfies the feature points of the distance sorting rule, and the target depth is related to the depth mean of the neighborhood pixels of the matched feature points in the current frame image; Establish a first pixel model of the matched feature point in the current frame image based on at least a plurality of neighbor feature points of the matched feature point in the current frame image, so that the first pixel model corresponds to the The multiple neighbor feature points of the matched feature point in the current frame image also have the direction vector of the first pixel model and the reprojection error of the matched feature point in the current frame image. 4. The method of claim 3, wherein the direction vector of the first pixel model is based on the three-dimensional coordinate value of the spatial center point corresponding to the neighbor feature point in the first pixel model and the current frame. The three-dimensional coordinate value of the spatial point corresponding to the matched feature point in the image is obtained, and the reprojection error of the matched feature point in the current frame image is obtained based on the historical image. 5. The method according to claim 3 or 4, characterized in that the first pixel model of the matched feature points in each of the current frame images is compared with the corresponding second pixel model to obtain Model comparison results include: Obtain the number of matching neighbor feature points in the first pixel model and its corresponding second pixel model; Obtain the module length of the difference between the direction vector in the first pixel model and its corresponding direction vector in the second pixel model; Determine whether the reprojection error of the matched feature points in the current frame image in the first pixel model is less than or equal to the reprojection error corresponding to the matched feature points in the historical image in the second pixel model The mean value is the mean value of the cumulative reprojection error of the matched feature points in the historical image in the sliding window to obtain the judgment result; According to the quantity, the module length and the judgment result, a model comparison result is obtained. 6. The method of claim 5, further comprising: Obtain an update request for a pixel model of a feature point in the historical image; Obtain newly added images within the sliding window; Perform feature point extraction on the newly added image to obtain feature points in the newly added image; Match the feature points in the newly added image with the feature points in the historical images in the sliding window to obtain matching features in the newly added image that match the feature points in the historical images points and non-matching feature points in the newly added image that do not match the feature points in the historical image; According to the matching feature point, update the pixel model of the feature point in the historical image that matches the matching feature point; Obtain the pixel model of the non-matching feature point, so that when the newly added image is used as a historical image in the sliding window, the pixel model of the non-matching feature point is used as a new feature point in the historical image. pixel model. 7. The method according to claim 1, characterized in that, after filtering out the target feature points in the current frame image according to the model comparison result, according to the target feature points in the current frame image Corresponding reprojection error, before obtaining the current pose of the mobile device, the method further includes: Among the target feature points in the current frame image, target feature points that satisfy the epipolar constraint with the feature points in the historical image are screened out. 8. The method of claim 1, wherein obtaining the matched feature points in the current frame image includes: Extract feature points in the current frame image; Match the feature points extracted from the current frame image with the feature points extracted from the historical image to obtain feature points in the current frame image that match the feature points in the historical image. 9. A posture acquisition device, characterized in that it includes: An image acquisition unit, used to obtain a current frame image, where the current frame image is an image collected by an image acquisition device on a mobile device; A feature point processing unit, configured to obtain the matched feature points in the current frame image, where the matched feature points in the current frame image are historical images included in the sliding window corresponding to the current frame image. There are feature points that match the feature points; the sliding window contains multiple frames of historical images, and the historical images are key frame images before the current frame image; A model building unit configured to obtain a first pixel model of each feature point matched in the current frame image, where the first pixel model corresponds to multiple neighbors of the feature point matched in the current frame image. feature points, and the first pixel model also has a direction vector of the first pixel model and a reprojection error of the matched feature points in the current frame image. The first pixel model is to remove the plurality of feature points. After the redundant neighbor feature points with the same depth value among the neighbor feature points are established, the remaining neighbor feature points are established based on the matched feature points in the current frame image; A model comparison unit, configured to compare the first pixel model of the matched feature points in each of the current frame images with the corresponding second pixel model to obtain a model comparison result. The model comparison The result represents whether the spatial point corresponding to the matched feature point in the current frame image belongs to a moving object; wherein, the second pixel model matches the feature point matched in the current frame image in the historical image. A pixel model of the feature point, the second pixel model corresponds to a plurality of neighbor feature points of the matched feature point in the historical image, and the second pixel model also has a direction vector of the second pixel model and the mean value of the reprojection error of the matched feature points in the historical image, and the comparison result is that the first pixel model of the matched feature point in each current frame image is compared with the corresponding second pixel model at multiple times. Neighbor points, direction vectors and reprojection errors are compared separately, and then multiple neighbor feature points, direction vectors and reprojection errors corresponding to the matched feature points in each current frame image are obtained. The comparison result of one item; A feature point screening unit, configured to screen out target feature points in the current frame image based on the model comparison results, where the target feature points are features of corresponding spatial points in the current frame image that do not belong to moving objects. point; A pose obtaining unit is configured to obtain the current pose of the mobile device based on the reprojection error corresponding to the target feature point in the current frame image. 10. A mobile device, characterized in that it includes: Memory for storing application programs and data generated by the operation of said application programs; A processor for executing said application to achieve: Obtain a current frame image, which is an image collected by an image acquisition device on the mobile device; Obtain the matched feature points in the current frame image, wherein the matched feature points in the current frame image are those with matching feature points in the historical images included in the sliding window corresponding to the current frame image. Feature points; the sliding window contains multiple frames of historical images, and the historical images are key frame images before the current frame image; A first pixel model of each matched feature point in the current frame image is obtained respectively, the first pixel model corresponds to a plurality of neighbor feature points of the matched feature point in the current frame image, and the The first pixel model also has a direction vector of the first pixel model and a reprojection error of the matched feature points in the current frame image. The first pixel model is to remove the depth values of the plurality of neighbor feature points. After obtaining the same redundant neighbor feature points, the remaining neighbor feature points are established based on the matched feature points in the current frame image; The first pixel model of the matched feature point in each of the current frame images is compared with the corresponding second pixel model to obtain a model comparison result, and the model comparison result characterizes the current frame image. Whether the spatial point corresponding to the matched feature point belongs to a moving object; wherein, the second pixel model is a pixel model of the feature point in the historical image that matches the feature point matched in the current frame image, The second pixel model corresponds to a plurality of neighbor feature points of the matched feature points in the historical image, and the second pixel model also has a direction vector of the second pixel model and a matching feature point in the historical image. The mean value of the reprojection error of the feature points, and the comparison result is that the first pixel model of the matched feature points in each current frame image and the corresponding second pixel model are in multiple neighbor points, direction vectors and Each item in the reprojection error is compared separately, and then the comparison results of multiple neighbor feature points, direction vectors and each item in the reprojection error corresponding to the matched feature points in each current frame image are obtained; According to the model comparison results, target feature points in the current frame image are screened out, and the target feature points are feature points whose corresponding spatial points in the current frame image do not belong to moving objects; according to the current frame The reprojection error corresponding to the target feature point in the image is used to obtain the current pose of the mobile device.