CN111179309B

CN111179309B - Tracking method and device

Info

Publication number: CN111179309B
Application number: CN201911319416.9A
Authority: CN
Inventors: 周毅; 高江涛; 陈建冲; 杨旭; 孙炼杰
Original assignee: Lenovo Beijing Ltd
Current assignee: Lenovo Beijing Ltd
Priority date: 2019-12-19
Filing date: 2019-12-19
Publication date: 2024-05-24
Anticipated expiration: 2039-12-19
Also published as: CN111179309A

Abstract

The application provides a tracking method and equipment, the method comprises the steps of obtaining characteristic points of a current frame image of a target object, updating the characteristic points of the current frame image according to initial information of the target object when the number of the characteristic points of the current frame image is smaller than a number threshold, wherein the initial information comprises initial pose information of an acquisition device and initial three-dimensional characteristic points corresponding to the characteristic points of the initial frame image when the initial frame image is acquired, and continuously tracking the target object according to the updated characteristic points of the current frame image.

Description

Tracking method and device

Technical Field

The application relates to the technical field of digital image processing, and relates to a tracking method and tracking equipment.

Background

With rapid development of multimedia technology and continuous improvement of computer performance, dynamic image processing technology is increasingly favored by people, and achieves great achievements, and is widely applied to the fields of traffic management, military target tracking, biomedicine and the like.

In the identification and tracking of a moving target, the original practice is to extract and match feature points of each shot frame of image, and the defects of large calculation amount and low processing speed exist. In order to avoid the defects, the existing solution is to perform recognition and tracking based on an optical flow method, specifically to extract characteristic points of a moving object, calculate pose based on a characteristic point extraction and matching method, and realize recognition and tracking of the moving object by reducing the characteristic points capable of recognizing and tracking to be less and less due to the movement of the object and returning to re-extract the characteristic points after the characteristic points are reduced to be unable to calculate the pose.

The prior proposal has the defects that: in the existing recognition and tracking based on the optical flow method, feature points only gradually decrease until no pose is calculated, and in the process of feature point reduction, the accuracy of calculated pose is lower and lower. Especially when the camera moves from one azimuth to another azimuth, the content of two continuous frame images is completely different, the characteristic points of recognition tracking are completely lost, the pose cannot be calculated any more, so that the recognition tracking is failed, or the calculated pose is wrong, so that the recognition tracking is wrong.

Disclosure of Invention

Accordingly, embodiments of the present application provide a tracking method and apparatus for solving the problems in the prior art.

The technical scheme of the embodiment of the application is realized as follows:

In a first aspect, an embodiment of the present application provides a tracking method, including:

acquiring characteristic points of a current frame image of a target object;

When the number of the characteristic points of the current frame image is smaller than a number threshold value, the characteristic points of the current frame image are updated according to the initial information of the target object, wherein the initial information comprises initial pose information of an acquisition device and initial three-dimensional characteristic points corresponding to the characteristic points of the initial frame image when the initial frame image is acquired;

And continuing to track the target object according to the characteristic points updated by the current frame image.

In a second aspect, an embodiment of the present application provides a tracking apparatus, including:

The acquisition module is used for acquiring the characteristic points of the current frame image of the target object;

The updating module is used for updating the characteristic points of the current frame image according to the initial information of the target object when the number of the characteristic points of the current frame image is smaller than a number threshold value, wherein the initial information comprises initial pose information of an acquisition device and initial three-dimensional characteristic points corresponding to the characteristic points of the initial frame image when the initial frame image is acquired;

and the tracking module is used for continuously tracking the target object according to the characteristic points updated by the current frame image.

In a third aspect, an embodiment of the present application provides a tracking apparatus, including:

A memory for storing executable instructions;

And the processor is used for realizing the method provided by the embodiment of the application when executing the executable instructions stored in the memory.

In a fourth aspect, an embodiment of the present application provides a storage medium storing executable instructions for implementing a method provided by an embodiment of the present application when the executable instructions cause a processor to execute.

In the tracking method provided by the embodiment of the application, a tracking device acquires the characteristic points of the current frame image of a target object, when the number of the characteristic points of the current frame image is smaller than a number threshold value, the characteristic points of the current frame image are updated according to initial information of the target object, wherein the initial information comprises initial pose information of the acquisition device and initial three-dimensional characteristic points corresponding to the characteristic points of the initial frame image when the initial frame image is acquired, and finally, the target object is continuously tracked according to the updated characteristic points of the current frame image; in this way, in the process of tracking the target object, when the feature points of the current frame image are reduced to the point that the current pose cannot be accurately calculated, the reduced feature points of the current frame image can be supplemented according to the initial information, so that the number of the feature points of the current frame image is more than a number of threshold values, the accuracy of the current pose calculated according to the feature points of the current frame image is ensured, and the success rate of tracking the target object is improved.

Drawings

Fig. 1A is a schematic diagram of a network architecture of a tracking method according to an embodiment of the present application;

fig. 1B is a schematic diagram of another network architecture of the tracking method according to the embodiment of the present application;

fig. 2 is a schematic diagram of a composition structure of a tracking device according to an embodiment of the present application;

FIG. 3 is a schematic flow chart of an implementation of the tracking method according to the embodiment of the present application;

FIG. 4 is a schematic flow chart of another implementation of the tracking method according to the embodiment of the present application;

FIG. 5 is a schematic flow chart of another implementation of the tracking method according to the embodiment of the present application;

fig. 6 is a schematic flow chart of still another implementation of the tracking method according to the embodiment of the present application.

Detailed Description

The present application will be further described in detail with reference to the accompanying drawings, for the purpose of making the objects, technical solutions and advantages of the present application more apparent, and the described embodiments should not be construed as limiting the present application, and all other embodiments obtained by those skilled in the art without making any inventive effort are within the scope of the present application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is to be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments and can be combined with one another without conflict.

In the following description, the terms "first", "second", "third" and the like are merely used to distinguish similar objects and do not represent a specific ordering of the objects, it being understood that the "first", "second", "third" may be interchanged with a specific order or sequence, as permitted, to enable embodiments of the application described herein to be practiced otherwise than as illustrated or described herein.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the application only and is not intended to be limiting of the application.

Before describing embodiments of the present application in further detail, the terms and terminology involved in the embodiments of the present application will be described, and the terms and terminology involved in the embodiments of the present application will be used in the following explanation.

1) Optical flow is due to movement of the foreground objects themselves in the scene, movement of the camera, or movement of both together.

2) Principle of optical flow method for tracking moving object: first, a velocity vector (optical flow, including magnitude and direction) is assigned to each pixel in the image, thus forming an optical flow field. If no moving object exists in the image, the optical flow field is continuous and uniform; if a moving object exists in the image, the optical flow of the moving object is different from the optical flow of the image, and the optical flow field is not continuous and uniform, so that the moving object and the position thereof can be detected, and the tracking of the moving object is realized.

3) The internal parameters may also be referred to as internal parameters. The internal parameters refer to parameters related to the characteristics of the acquisition device itself, such as the focal length, pixel size, etc. of the acquisition device. In particular, the internal parameters of the acquisition device may include: 1/dx, 1/dy, u0, v0 and f, wherein dx and dy respectively represent how many length units a pixel in the x direction and the y direction respectively occupies, namely, the size of an actual physical value represented by a pixel, and dx and dy are keys for realizing conversion of an image physical coordinate system and a pixel coordinate system. u0, v0 denote the number of horizontal and vertical pixels of the phase difference between the center pixel coordinates of the image and the image origin pixel coordinates, f being the focal length. In some embodiments, the internal parameter information may also include distortion parameters, which in turn further include radial distortion coefficients and tangential distortion coefficients. Radial distortion occurs during the conversion of the camera coordinate system to the physical coordinate system of the image. Tangential distortion occurs during camera fabrication due to the fact that the plane of the photoreceptor element is not parallel to the lens.

4) The external parameters may also be referred to as external parameters. The external parameters may refer to pose information of the acquisition device, and the external parameters can determine the position and orientation of the acquisition device in a certain three-dimensional space. The external parameters may include the position of the acquisition device, the rotation angle, etc. In particular, the external parameters of the acquisition device may comprise rotational parameter information of the three axes and translational parameter information of the three axes. The size of the matrix R of the rotation parameter information of the three axes is 3*3, and the translation parameters T (Tx, ty, tz) of the three axes, and the matrix of 3×4 formed by R, T are the external parameters of the acquisition device.

5) The point cloud, the point data set of the product appearance surface obtained by a measuring instrument in reverse engineering is also called as point cloud, the number of points obtained by using a three-dimensional coordinate measuring machine is usually small, the distance between the points is also large, and the point cloud is called as sparse point cloud; and the point cloud obtained by using a three-dimensional laser scanner or a photographic scanner, for example, has larger and denser point number, and is called dense point cloud. And the point cloud data is recorded in the form of points, and each point comprises three-dimensional coordinates.

For a better understanding of the embodiments of the present application, a tracking method and existing drawbacks in the related art will be described first.

In the related art, one method for identifying and tracking a three-dimensional object is to shoot each frame of image of a moving object, extract characteristic points for each frame of image to determine the moving object, and the extracted characteristic points have large data quantity, low identification speed and high requirements on processing equipment, so that the tracking cost is increased.

In the related art, another method for identifying and tracking the three-dimensional object is to calculate the initial pose based on the feature point extraction and matching method, then track the feature point based on the optical flow method, and the feature point extraction is not required for each frame of image, so that the processing speed is improved. However, when the method is adopted, due to continuous motion of a moving object, the feature points which can be identified and tracked in the tracking process are fewer and fewer. When the feature points are reduced to the pose which cannot be calculated, the feature points need to be extracted again, so that the identification and tracking of the moving object are realized. Therefore, the method still needs to extract the feature points for many times, and when the feature points are reduced to the position and pose which cannot be calculated, the feature points are extracted again, so that the recognition tracking is possibly failed, or the recognition tracking is wrong due to the error of the position and pose which is calculated again.

Based on the above problems, in the embodiment of the application, when the feature points are reduced to the pose which cannot be accurately calculated, the reduced feature points can be supplemented according to the initial information, so that the number of the feature points is always more than the number threshold value, the accuracy of the pose calculated according to the feature points is ensured, and the success rate of tracking the target object is improved.

An exemplary application of the apparatus implementing the embodiment of the present application is described below, and the apparatus provided in the embodiment of the present application may be implemented as a terminal device. In the following, an exemplary application covering a terminal device when the apparatus is implemented as a terminal device will be described.

Referring to fig. 1A, fig. 1A is a schematic diagram of a network architecture of a tracking method according to an embodiment of the present application, where the network architecture includes at least a terminal 101, a server 102, a network 103, and a tracked target object 104, as shown in fig. 1A. The terminal 101 may be a mobile terminal with wireless communication capability, such as a mobile phone (mobile phone), a tablet computer, a notebook computer, or AR glasses. The terminal 101 comprises at least acquisition means by which a user can acquire an image of the tracked target object 104 in the terminal 101. The terminal 101 is exemplarily shown in the form of AR glasses in fig. 1A. The server 102 may be one server, or may be a server cluster including a plurality of servers, a cloud computing center, or the like, which is not limited herein. The terminal 101 establishes a communication connection with the server 102 through a network 103, where the network 103 may be a wide area network or a local area network, or a combination of both, and uses a wireless link to implement data transmission.

In the network architecture, the terminal 101 may collect a current frame image of the target object 104, send the current frame image of the target object 104 to the server 102, further obtain feature points of the current frame image by the server 102, compare the number of the obtained feature points of the current frame image with a preset number threshold, update the feature points of the current frame image based on initial information of the target object 104 stored in the terminal when the number of the feature points of the current frame image is smaller than the number threshold, obtain updated feature points of the current frame image, then send the updated feature points of the obtained current frame image to the terminal 101, and keep tracking the target object 104 by the terminal 101 based on the updated feature points of the current frame image.

Fig. 1B is a schematic diagram of another network architecture of the tracking method according to the embodiment of the present application, as shown in fig. 1B, in the network architecture, the method includes: a terminal 111 and a tracked target object 114. The terminal 111 comprises at least acquisition means by which a user can acquire images of the tracked target object 114 in the terminal 111. The terminal 111 is also shown in fig. 1B in the form of AR glasses by way of example. The terminal 111 acquires the current frame image of the target object 114, acquires the feature points of the current frame image, compares the number of the feature points of the acquired current frame image with a preset number threshold, and updates the feature points of the current frame image based on initial information of the target object 114 stored in the terminal when the number of the feature points of the current frame image is smaller than the number threshold to obtain updated feature points of the current frame image, and continuously tracks the target object 114 based on the updated feature points of the current frame image.

In the network architecture shown in fig. 1B, the network architecture shown in fig. 1A is generally used because the requirements for the computing efficiency, the storage space, and the like of the terminal 111 are high.

The apparatus provided in the embodiments of the present application may be implemented in hardware or a combination of hardware and software, and various exemplary implementations of the apparatus provided in the embodiments of the present application are described below.

Other exemplary configurations of the tracking device 200 are contemplated in accordance with the exemplary configuration of the tracking device 200 illustrated in fig. 2, and thus the configurations described herein should not be considered limiting, as, for example, some of the components described below may be omitted or components not described below may be added to accommodate the particular needs of certain applications.

The tracking device 200 shown in fig. 2 includes: at least one processor 210, a memory 240, at least one network interface 220, and a user interface 230. Each of the components in the tracking device 200 are coupled together by a bus system 250. It is understood that the bus system 250 is used to enable connected communications between these components. The bus system 250 includes a power bus, a control bus, and a status signal bus in addition to the data bus. But for clarity of illustration the various buses are labeled as bus system 250 in fig. 2.

The user interface 230 may include a display, keyboard, mouse, touch pad, touch screen, and the like.

The memory 240 may be volatile memory or nonvolatile memory, and may include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read Only Memory (ROM). The volatile memory may be random access memory (RAM, random Access Memory). The memory 240 described in embodiments of the present application is intended to comprise any suitable type of memory.

The memory 240 in embodiments of the present application is capable of storing data to support the operation of the tracking device 200. Examples of such data include: any computer programs for operating on the tracking device 200, such as an operating system and application programs. The operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks. The application may comprise various applications.

As an example of implementation of the method provided by the embodiment of the present application by software, the method provided by the embodiment of the present application may be directly embodied as a combination of software modules executed by the processor 210, the software modules may be located in a storage medium, the storage medium is located in the memory 240, and the processor 210 reads executable instructions included in the software modules in the memory 240, and the method provided by the embodiment of the present application is completed by combining necessary hardware (including, for example, the processor 210 and other components connected to the bus 250).

By way of example, the Processor 210 may be an integrated circuit chip having signal processing capabilities such as a general purpose Processor, such as a microprocessor or any conventional Processor, a digital signal Processor (DSP, digital Signal Processor), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like.

Embodiments of a tracking method, a tracking device, and an apparatus are described below with reference to application scenario diagrams shown in fig. 1A and 1B.

The present embodiment provides a tracking method, which is applied to a tracking apparatus, and the tracking apparatus may be the server 102 as shown in fig. 1A or may be the terminal 111 as shown in fig. 1B.

Fig. 3 is a schematic flow chart of an implementation of a tracking method according to an embodiment of the present application, as shown in fig. 3, the method includes the following steps:

step S301, obtaining feature points of a current frame image of a target object.

After the tracking device determines the tracked target object, firstly acquiring an initial frame image acquired by the acquisition device, extracting characteristic points of the initial frame image, further determining the initial pose of the acquisition device when the acquisition device acquires the initial frame image according to the extracted characteristic points, and then tracking the extracted characteristic points based on an optical flow method.

The method comprises the steps of taking an initial frame image as a first frame image of a target object acquired during tracking, taking characteristic points of the frame image as a reference, acquiring characteristic points of a current frame image by utilizing characteristic points of a previous frame image based on an optical flow tracking principle, taking the current frame image as a new previous frame image, continuously acquiring characteristic points of a new current frame image by utilizing characteristic points of the new previous frame image, and circularly acquiring characteristic points of a subsequent frame image.

In the actual implementation process, if the acquisition device of the current terminal moves from one azimuth to another azimuth, the acquired content difference between the current frame image and the previous frame image is large, and even the content is completely different. At this time, when the feature points of the next frame image are acquired according to the feature points of the previous frame image, the acquired feature points are rarely or even completely lost, so that tracking cannot be continued; or the acquired characteristic points are wrong, so that tracking errors are caused. To solve this problem, in the present embodiment, before step S302, it is determined whether the tracking is successful. Specifically, whether the current pose can be calculated is determined according to the obtained feature points of the current frame image, if the current pose can be calculated, tracking is determined to be successful, at this time, whether the number of the feature points of the current frame image is less than a number threshold is further determined, and when the number of the feature points of the current frame image is less than the number threshold, step S302 is performed. If the current pose cannot be calculated, the tracking failure is indicated, and at this time, the initial frame image of the target object needs to be acquired again and tracked.

Step S302, when the number of feature points of the current frame image is smaller than a number threshold, updating the feature points of the current frame image according to the initial information of the target object.

If it is judged whether the tracking is successful before step S302, when the number of the feature points of the current frame image is smaller than the number threshold, it indicates that the current pose cannot be accurately calculated according to the feature points of the current frame image, that is, the accuracy of the current pose calculated according to the feature points of the current frame image is low. If it is not determined whether the tracking is successful before step S302, when the number of feature points of the current frame image is less than the number threshold, it indicates that the current pose cannot be calculated according to the feature points of the current frame image, or the current pose can be calculated according to the feature points of the current frame image, but the accuracy of the calculated current pose is low. At this time, it is necessary to acquire initial information of the target object, and perform an update operation on feature points of the current frame image according to the initial information of the target object. Here, the initial information includes initial pose information of the acquisition device at the time of acquiring the initial frame image and initial three-dimensional feature points corresponding to feature points of the initial frame image.

In some embodiments, the process of acquiring the initial information of the target object may be implemented by:

Step S31, obtaining feature points of the initial frame image.

In this embodiment, after the tracking device receives the initial frame image acquired by the acquisition device, the feature points of the initial frame image are extracted. The algorithm used in extracting features is an existing extraction algorithm, for example, a feature extraction algorithm such as Scale-invariant feature transform (SIFT, scale-INVARIANT FEATURE TRANSFORM), acceleration feature algorithm with robust features (SURF, speed Up Robust Features), FAST corner detection and BRIEF features (ORB, oriented FAST and Rotated BRIEF) can be used. In some embodiments, the feature points of the initial frame image may also be artificially marked.

Each extracted feature point has two-dimensional spatial information that characterizes the position of each feature point in the initial frame image. In practical applications, the two-dimensional spatial information may be coordinate information of each feature point in the initial frame image, for example, the two-dimensional spatial information of one extracted feature point may be (20, 30), and the two-dimensional spatial information may represent that the feature point is a pixel point of the 20 th row and 30 th column.

And step S32, determining initial pose information of the acquisition device and initial three-dimensional characteristic points corresponding to the characteristic points of the initial frame image when the initial frame image is acquired according to the characteristic points of the initial frame image and training data.

Here, the training data includes two-dimensional feature points extracted from training images acquired at different angles of the target object and three-dimensional feature points corresponding to the two-dimensional feature points.

In this embodiment, the feature points have two-dimensional spatial information and three-dimensional spatial information, wherein the two-dimensional spatial information of the feature points represents positions of the feature points in the frame image, and the three-dimensional information of the feature points represents three-dimensional coordinates of the feature points in practice. For clarity of description, when a feature point in a frame image is specified, it is referred to as a two-dimensional feature point, and when a feature point in a space is specified, it is referred to as a three-dimensional feature point.

The tracking device is pre-stored with training data of a tracked target object, wherein the training data are acquired by the tracking device according to a plurality of pre-acquired training images of the target object at different angles. In the implementation process, firstly, training images of a plurality of different angles of a target object acquired by an acquisition device are acquired, and then, a three-dimensional space model of the target object is established according to depth information of the training images of the plurality of different angles. And extracting the features of each training image to obtain two-dimensional feature points of each training image, and combining the built three-dimensional space model to obtain three-dimensional feature points corresponding to each two-dimensional feature point, thereby obtaining training data. The training data includes two-dimensional feature points and three-dimensional feature points corresponding to the two-dimensional feature points.

After the tracking device acquires the feature points of the initial frame image, the initial two-dimensional feature points corresponding to the feature points are matched in the training data, and then the corresponding initial three-dimensional feature points are determined according to the initial two-dimensional feature points. And the tracking device determines initial pose information when the acquisition device acquires the initial frame image by utilizing a PNP (PESPECTIVE-N-Point) algorithm according to the two-dimensional characteristic points and the three-dimensional characteristic points of the initial frame image and the internal parameters of the acquisition device.

After the initial information of the target object is obtained according to the steps S31 and S32, the current three-dimensional feature point of the current frame image is determined according to the initial three-dimensional feature point, the current three-dimensional feature point is mapped to the current frame image to obtain a mapping point, and finally the feature point of the current frame image is updated according to the mapping point, so that the reduced feature point is supplemented, the number of the feature point of the current frame is more than a threshold value, the accuracy of pose calculated according to the feature point of the current frame image is ensured, and the success rate of tracking the target object is improved.

Step S303, continuing to track the target object according to the feature points updated by the current frame image.

After the feature points of the current frame image are supplemented, tracking of the target object based on an optical flow method is continued until tracking is finished.

In some embodiments, the step S301 "acquiring the feature point of the current frame image of the target object" may be implemented by the following steps:

In step S3011, feature points of the previous frame image of the target object are acquired.

According to the tracking principle of the optical flow method, the characteristic points of the second frame image are obtained through the characteristic points of the initial frame image, then the characteristic points of the third frame image are obtained according to the characteristic points of the second frame image, and the characteristic points of the current frame image are obtained through circulation. Based on this, when the feature point of the current frame image of the target object is to be acquired, the feature point of the previous frame image of the target object is acquired first.

Step S3012, determining a corresponding point of the feature point of the previous frame image in the current frame image.

When step S3012 is implemented, firstly, optical flow changes between the previous frame image and the current frame image are determined according to the optical flow changes between the previous frame image and the current frame image, and then, corresponding points of feature points of the previous frame image in the current frame image are determined according to the optical flow changes between the previous frame image and the current frame image, so that feature points of the current frame image are obtained.

And step S3013, determining the corresponding points as characteristic points of the current frame image.

According to the tracking method provided by the embodiment, the optical flow method is adopted for tracking, so that the characteristic points of the current frame image are obtained on the basis of the characteristic points of the previous frame image of the target object, the characteristic points of the current frame image can be obtained without extracting the characteristic points of the current frame image, and the processing speed of the current frame image in the tracking process is improved.

In some embodiments, the matching can be directly performed according to the only feature points in the current frame image and the three-dimensional feature points in the initial information, so as to update the feature points of the current frame image, but because the feature points of the current frame image are less than the number threshold, when the matching is performed by adopting the fewer feature points and the three-dimensional feature points in the initial information, the accuracy of successful matching is lower, the accuracy of the updated feature points of the current frame image is affected, the accuracy of the current pose calculated according to the updated feature points of the current frame image is reduced, and the success rate of tracking the target object is further affected.

In this embodiment, the feature point of the previous frame image may be updated first, and then the feature point of the current frame image may be obtained again according to the updated feature point of the previous frame image. When the method is adopted to obtain the feature point of the current frame image, the step S3013 "the corresponding point is determined as the feature point of the current frame image" may be implemented by the following steps:

Step S3013a, acquiring first pose information of the acquisition device when acquiring a previous frame of image.

In step S3013a, when implementing, the tracking device determines, firstly, a three-dimensional feature point corresponding to a two-dimensional feature point of the previous frame image according to the feature point of the previous frame image and the training data, and then determines, according to the two-dimensional feature point of the previous frame image, the three-dimensional feature point and an internal parameter of the acquisition device, first pose information of the acquisition device when acquiring the previous frame image.

And step S3013b, updating the characteristic points of the previous frame of image according to the first pose information, the initial pose information and the initial three-dimensional characteristic points.

In the step S3013b, when implementing, the tracking device determines a transformation model from the initial pose information to the first pose information, transforms the initial three-dimensional feature point into a first three-dimensional feature point according to the transformation model, maps the first three-dimensional feature point to the previous frame image to obtain a mapping point, and finally updates the feature point of the previous frame image according to the mapping point.

Firstly, screening the mapping points according to the size of the previous frame image and/or the first pose information to obtain mapping points matched with the previous frame image; and then determining the mapping point matched with the previous frame image as the characteristic point updated by the previous frame image.

And according to the difference of the sizes of the previous frame image and the initial frame image, or according to the difference of the first pose information corresponding to the previous frame image and the initial pose information corresponding to the initial frame image, or according to the difference of the sizes of the previous frame image and the initial frame image, and the difference of the first pose information corresponding to the previous frame image and the initial pose information corresponding to the initial frame image, mapping the first three-dimensional feature point to the previous frame image, and then removing mapping points which do not accord with the size and/or the pose information of the previous frame image to obtain mapping points matched with the previous frame image so as to ensure the accuracy of the extracted feature point of the previous frame image.

And step S3013c, updating the characteristic points of the current frame image according to the characteristic points updated by the previous frame image and the first pose information.

In the process of tracking the target object, if the feature points of the current frame image are reduced to less than the number threshold, the current pose of the acquisition device cannot be calculated according to the feature points of the current frame image, or the current pose can be calculated according to the feature points of the current frame image, but the calculation accuracy is low. In this embodiment, the feature points of the previous frame image are updated according to the three-dimensional feature points in the initial information, so that the matching accuracy can be improved, then the feature points of the current frame are obtained again according to the updated feature points of the previous frame image, the accuracy of the feature points of the updated current frame image can be ensured, the accuracy of the current pose calculated according to the updated feature points of the current frame image is improved, and the success rate of tracking the target object is improved.

On the basis of the embodiment shown in fig. 3, the embodiment of the present application further provides a tracking method, and fig. 4 is a schematic flow chart of another implementation of the tracking method provided by the embodiment of the present application, as shown in fig. 4, where the method includes:

in step S401, feature points of a current frame image of a target object are acquired.

In this embodiment, the descriptions of the corresponding parts in step S401, step S403, and step S404 are specifically referred to in step S301 to step S303 in the embodiment shown in fig. 3, and are not repeated in this embodiment.

Step S402, determining whether the number of feature points of the current frame image is less than a number threshold.

When the number of the feature points of the current frame image is smaller than the number threshold, it indicates that the current pose cannot be accurately calculated according to the feature points of the current frame image, and the feature points of the current frame image need to be updated, and the step S403 is performed. When the number of the feature points of the current frame image is greater than or equal to the number threshold, it indicates that the accuracy of the current pose calculated according to the feature points of the current frame image is higher, and the step S405 is performed without updating the feature points.

Step S403, updating the feature points of the current frame image according to the initial information of the target object.

Here, the initial information includes initial pose information of the acquisition device at the time of acquiring the initial frame image and initial three-dimensional feature points corresponding to feature points of the initial frame image.

And step S404, continuing to track the target object according to the characteristic points updated by the current frame image.

When the number of the characteristic points of the current frame image is greater than or equal to the number threshold, determining second pose information of the acquisition device when the current frame image is acquired according to the characteristic points of the current frame image, and determining the current position of the target object by combining the initial pose information and the coordinates of the target object in the current frame image, and executing step S405 and step S406.

Step S405, determining second pose information of the acquisition device and coordinates of the target object in the current frame image when the current frame image is acquired according to the feature points of the current frame image.

When the feature points of the current frame image are not updated, that is, the number of the feature points of the current frame image is greater than or equal to the number threshold, or when the feature points of the current frame image are updated feature points of the current frame image, on the basis of the embodiment, the second pose information corresponding to the current frame image is accurately calculated according to the feature points of the current frame image.

In the actual implementation process, two-dimensional feature points corresponding to the training data and three-dimensional feature points corresponding to the two-dimensional feature points are determined according to the feature points of the current frame image, and then second pose information of the acquisition device when the current frame image is acquired is determined according to the two-dimensional feature points, the three-dimensional feature points and internal parameters of the acquisition device.

And determining the coordinates of the target object in the current frame image according to the two-dimensional space information of all the characteristic points of the current frame image, namely the pixel positions of all the characteristic points in the current frame image, wherein the coordinates are plane coordinates.

Step S406, determining the position of the target object according to the initial pose information, the second pose information, and the coordinates of the target object in the current frame image.

And determining the current position and orientation of the acquisition device in world coordinates according to the second pose information when the acquisition device acquires the current frame image and the initial pose information when the acquisition device acquires the initial frame image, which are obtained in the step S405. And determining the current position of the target object according to the known current position and orientation of the acquisition device and the acquired coordinates of the target object in the current frame image.

In some embodiments, as shown in fig. 4, after step S406, the method further comprises:

step S407, it is determined whether the end tracking condition is reached.

When the end tracking condition is reached, returning to execute step S405; when the end tracking condition is not reached, the current frame image is taken as the previous frame image, and the process returns to step S401.

Step S408, stopping tracking, and transmitting the position of the target object.

In this embodiment, when the end tracking condition is not reached, the tracking device always tracks the target object based on the optical flow method. In the tracking process, once the tracking condition is finished, the tracking is stopped, and the current position of the target object when the tracking is stopped is sent to the terminal.

In a possible implementation, the end tracking condition may be: reaching a preset tracking time length, and/or receiving an operation instruction sent by the terminal and used for indicating the end of tracking, and/or determining that the target object reaches a preset position, and the like. For example, when the duration from the start of tracking to the current time reaches the preset tracking duration, the tracking is not continued, and the tracking device determines that the end tracking condition is reached. For another example, when the duration from the start of tracking to the current time reaches the preset tracking duration or when the target object reaches the preset position, the tracking is not continued, and the tracking device determines that the end tracking condition is reached.

Based on the foregoing embodiments, the embodiment of the present application further provides a tracking method, which is applied to the network architecture shown in fig. 1A, and fig. 5 is a schematic flowchart of still another implementation of the tracking method provided by the embodiment of the present application, as shown in fig. 5, where the method includes:

In step S501, the terminal acquires training images of the target object at a plurality of different angles.

In the static state of the target object, the position and the angle of the terminal are changed, the image of the target object at a plurality of different angles is acquired by utilizing the acquisition device of the terminal, and the image is uploaded to a server to be used as a training image for training, so that the characteristic point data of the moving target object at a plurality of different angles is obtained.

In step S502, the terminal sends training images of a plurality of different angles to the server.

In step S503, the server extracts two-dimensional feature points of the training images of the plurality of different angles and three-dimensional feature points corresponding to the two-dimensional feature points, respectively.

In the step S503, feature extraction is performed on the training images of each angle to obtain two-dimensional feature points of all the training images, and then the two-dimensional position information of the two-dimensional feature points is combined with the depth information of the feature points and the internal parameters of the acquisition device to obtain three-dimensional feature points corresponding to the two-dimensional feature points.

Each extracted two-dimensional feature point has two-dimensional space information, and the two-dimensional space information can represent the position of each two-dimensional feature point in the training image. In practical applications, the two-dimensional spatial information of the two-dimensional feature points may be coordinate information of the feature points in the training image. The three-dimensional spatial information of the three-dimensional feature points may be world coordinates of the feature points in reality.

In step S504, the server determines the two-dimensional feature point and the three-dimensional feature point corresponding to the two-dimensional feature point as training data.

Since the same feature point of the target object may exist in a plurality of training images, the two-dimensional feature point corresponding to the feature point in the training images is actually the same point. Accordingly, the same feature point of the target object is fused with a plurality of two-dimensional feature points in different training images, so that two-dimensional feature points of training data are obtained, then three-dimensional feature points corresponding to the two-dimensional feature points are determined according to the two-dimensional feature points, and the two-dimensional feature points and the three-dimensional feature points corresponding to the two-dimensional feature points are determined to be training data.

In step S505, the terminal acquires an initial frame image of the target object.

After the training data are obtained, the terminal acquires an initial frame image of the target object through the acquisition device of the terminal, and starts to track the target object.

In step S506, the terminal sends the initial frame image to the server.

In step S507, the server acquires feature points of the initial frame image according to the initial frame image.

The server adopts the existing feature extraction algorithm to extract the feature points of the initial frame image, and the feature points of the initial frame image are obtained. For example, feature extraction algorithms such as Scale-invariant feature transforms (SIFT, scale-INVARIANT FEATURE TRANSFORM), acceleration feature algorithms with robust features (SURF, speed Up Robust Features), FAST corner detection and BRIEF features (ORB, oriented FAST and Rotated BRIEF) may be utilized. In some embodiments, the feature points of the initial frame image may also be artificially marked.

Step S508, the server determines initial pose information of the acquisition device and initial three-dimensional feature points corresponding to the feature points of the initial frame image when the initial frame image is acquired according to the feature points of the initial frame image and the training data.

After the tracking device acquires the feature points of the initial frame image, the initial two-dimensional feature points corresponding to the feature points are matched in the training data, and then the corresponding initial three-dimensional feature points are determined according to the initial two-dimensional feature points. And the tracking device determines initial pose information when the acquisition device acquires the initial frame image according to the two-dimensional characteristic points and the three-dimensional characteristic points of the initial frame image and the internal parameters of the acquisition device.

After the terminal sends the initial frame image to the server, the terminal continues to collect the subsequent frame image of the target object and sends the subsequent frame image to the server. The server starts tracking the target object in the received frame image based on the optical flow method.

In step S509, the server acquires feature points of the previous frame image of the target object.

The server acquires the characteristic points of the second frame image according to the characteristic points of the initial frame image, acquires the characteristic points of the third frame image according to the characteristic points of the second frame image, and circularly acquires the characteristic points of the previous frame image.

In step S510, the terminal acquires a current frame image of the target object.

In step S511, the terminal sends the current frame image to the server.

In step S512, the server determines the corresponding point of the feature point of the previous frame image in the current frame image.

The server determines the optical flow change between the previous frame image and the current frame image according to the previous frame image and the current frame image, and then determines the corresponding point of the characteristic point of the previous frame image in the current frame image according to the optical flow change of the previous frame image and the current frame image, so as to obtain the characteristic point of the current frame image.

In step S513, the server determines the corresponding point as a feature point of the current frame image.

In step S514, the server determines whether the number of feature points of the current frame image is less than a number threshold.

When the number of feature points of the current frame image is smaller than the number threshold, the process advances to step S515. When the number of feature points of the current frame image is greater than or equal to the number threshold, the process advances to step S523.

In step S515, the server determines three-dimensional feature points of the previous frame corresponding to the feature points of the previous frame according to the feature points of the previous frame and the training data.

In step S516, the server determines, according to the feature points of the previous frame of image, the three-dimensional feature points of the previous frame of image and the internal parameters of the acquisition device, the first pose information of the acquisition device when the previous frame of image is acquired.

In step S517, the server determines a transformation model from the initial pose information to the first pose information.

In step S518, the server transforms the initial three-dimensional feature point into a first three-dimensional feature point according to the transformation model.

Here, the first three-dimensional feature point is a three-dimensional feature point corresponding to the previous frame image.

Since the tracked target object is unchanged, the three-dimensional model of the target object is unchanged. The acquisition device is corresponding to the transformation relation from the initial pose information to the first pose information and the transformation relation from the initial three-dimensional feature point of the initial frame image to the first three-dimensional feature point of the previous frame image, so that the initial three-dimensional feature point can be transformed into the first three-dimensional feature point according to the transformation model.

In step S519, the server maps the first three-dimensional feature point to the previous frame image, to obtain a mapped point.

In step S520, the server performs a filtering operation on the mapping points according to the size of the previous frame image and/or the first pose information, so as to obtain mapping points matched with the previous frame image.

After the first three-dimensional feature point is mapped to the previous frame of image, the mapping points which do not accord with the size and/or pose information of the previous frame of image are removed, and the mapping points matched with the previous frame of image are obtained, so that the accuracy of the feature points of the previous frame of image is ensured.

In step S521, the server determines the mapping point matched with the previous frame image as the feature point after the previous frame image is updated.

The mapping points after the screening operation are used as the characteristic points after the previous frame of image is updated, so that the characteristic points lost by the previous frame of image are supplemented.

In step S522, the server continues to track the target object according to the feature points updated by the current frame image.

When the number of the characteristic points of the current frame image is smaller than the number threshold value, the characteristic points of the previous frame image are updated, the characteristic points of the current frame image are obtained again according to the updated characteristic points of the previous frame image, the characteristic points of the current frame image are supplemented, the number of the characteristic points of the current frame image is enabled to be larger than the number threshold value, accuracy of the current pose calculated according to the characteristic points of the current frame image is ensured, and the success rate of tracking the target object is improved.

In step S523, the server determines, according to the feature points of the current frame image, second pose information of the acquisition device and coordinates of the target object in the current frame image when the current frame image is acquired.

In step S524, the server determines the position of the target object according to the initial pose information, the second pose information, and the coordinates of the target object in the current frame image.

The number of the characteristic points of the current frame image is larger than or equal to a number threshold value, and the second pose information corresponding to the current frame image can be accurately calculated according to the characteristic points of the current frame image, so that the current position of the target object can be determined, and real-time tracking is realized.

In step S525, the server determines whether the end tracking condition is reached.

When the tracking condition is reached, the process advances to step S526. When the tracking condition is not reached, the current frame image is taken as the previous frame image, and the process returns to the step S510 again.

In step S526, the server sends an instruction to stop tracking to the terminal.

In step S527, the server sends the location of the target object to the terminal.

In step S528, the terminal stops tracking.

In this embodiment, when the end tracking condition is not reached, the server always tracks the target object based on the optical flow method. In the tracking process, once the tracking condition is finished, the tracking is stopped, and an instruction for stopping the tracking and the current position of the target object when the tracking is stopped are sent to the terminal. And the terminal stops tracking according to the received instruction.

According to the tracking method provided by the embodiment, in the process of tracking the target object, when the characteristic points of the current frame image are reduced to the point that the current pose cannot be accurately calculated, the characteristic points of the previous frame image can be updated according to the three-dimensional characteristic points in the initial information, and then the characteristic points of the current frame are obtained again according to the updated characteristic points of the previous frame image, so that the reduced characteristic points of the current frame image are supplemented, the number of the characteristic points of the current frame image is more than a threshold value, the accuracy of the characteristic points of the updated current frame image is ensured, the accuracy of the current pose calculated according to the characteristic points of the updated current frame image is improved, and the success rate of tracking the target object is further improved.

Based on the foregoing embodiments, the embodiment of the present application further provides a tracking method, which is applied to the network architecture shown in fig. 1A. In the tracking method provided by the embodiment of the application, firstly, the server acquires training data of a target object according to training images of a plurality of different angles acquired by the acquisition device of the terminal. After tracking is started, an initial frame image is acquired, feature points in the initial frame image are extracted, each frame image in the tracking process is continuously acquired, feature points of a current frame image are acquired according to feature points of a previous frame image, whether the pose can be calculated or not is judged according to the acquired feature points of the current frame image, if the current pose cannot be calculated according to the feature points of the current frame image, tracking failure is determined, and if tracking failure is not, tracking is restarted. If the current pose can be calculated according to the characteristic points of the current frame image, the tracking success is determined, whether the characteristic points of the current frame image are less than the number threshold value is further judged, and if the characteristic points of the current frame image are less than the number threshold value, the tracking is continued after the characteristic points of the current frame image are supplemented.

Fig. 6 is a schematic flow chart of still another implementation of the tracking method according to the embodiment of the present application, as shown in fig. 6, where the method includes:

in step S601, pose is calculated based on the feature point extraction and matching method.

After the server acquires the initial frame image acquired by the acquisition device of the terminal, extracting characteristic points in the initial frame image, and calculating the initial pose of the acquisition device.

Since the training image of the target object is an image matched with the initial frame image, the two-dimensional feature points matched with the feature points of the initial frame image in the training data can be correspondingly used as the three-dimensional feature points of the initial frame image. After the two-dimensional characteristic points, the three-dimensional characteristic points and the internal parameters of the acquisition device, which are matched with the characteristic points of the initial frame image, are known, the initial pose of the acquisition device when the acquisition device acquires the initial frame image is determined by utilizing a PNP algorithm, wherein the initial pose can comprise the position, the rotation direction and the like of the acquisition device when the initial frame image is acquired.

Step S602, tracking the feature points based on the optical flow method.

In practical implementation, each frame of image in the tracking process is acquired, the characteristic point of the current frame of image is acquired according to the characteristic point of the previous frame of image, the current frame of image is taken as the previous frame of image, and the next frame of image is continuously acquired.

Step S603, determining whether the tracking is successful.

If the acquisition device of the current terminal moves from one azimuth to another azimuth, the acquired current frame image and the acquired previous frame image have larger content difference, even completely different content. At this time, when the feature points of the next frame image are acquired according to the feature points of the previous frame image, the acquired feature points are rarely or even completely lost, so that tracking cannot be continued; or the acquired characteristic points are wrong, so that tracking errors are caused. To solve this problem, in the present embodiment, before step S604, it is determined whether the tracking is successful. Specifically, whether the current pose can be calculated is determined according to the feature points of the acquired current frame image, if the current pose can be calculated, the tracking is determined to be successful, and the step S604 is performed. If the current pose cannot be calculated, it indicates that tracking fails, at this time, the initial frame image of the target object needs to be acquired again, and tracking is performed, that is, step S601 is executed again.

In step S604, it is determined whether the number of feature points is less than a set number threshold.

When the optical flow method is adopted for tracking, the acquired characteristic points are fewer and fewer due to the movement of the target. When the feature points of the current frame image are less than the number threshold, although the current pose can be calculated, the accuracy of the calculated current pose cannot be ensured. In the present embodiment, when the feature point of the current frame image is reduced to the number threshold, the process advances to step S605. If not less, the process returns to step S602.

Step S605 supplements the feature points.

When the number of the feature points of the current frame image is smaller than the number threshold, the feature points of the current frame image are updated according to the initial information of the target object, and then the target object is continuously tracked according to the updated feature points of the current frame image, namely, the step S602 is executed in a return mode.

Continuing with the description below of an exemplary architecture of the tracking device 700 implemented as a software module provided by embodiments of the present application, in some embodiments, as shown in fig. 2, the software modules stored in the tracking device 700 of the memory 140 may include: a first acquisition module 701, an update module 702, and a tracking module 703, wherein:

A first obtaining module 701, configured to obtain feature points of a current frame image of a target object;

the updating module 702 is configured to update the feature points of the current frame image according to initial information of the target object when the number of the feature points of the current frame image is less than a number threshold, where the initial information includes initial pose information of an acquisition device when the initial frame image is acquired and initial three-dimensional feature points corresponding to the feature points of the initial frame image;

and the tracking module 703 is used for continuously tracking the target object according to the feature points updated by the current frame image.

In other embodiments, the first obtaining module 701 further includes:

The first acquisition sub-module is used for acquiring the characteristic points of the previous frame of image of the target object;

The first determining submodule is used for determining corresponding points of the characteristic points of the previous frame image in the current frame image;

And the second determining submodule is used for determining the corresponding points as characteristic points of the current frame image.

In other embodiments, the update module 702 further comprises:

the second acquisition sub-module is used for acquiring first pose information of the acquisition device when acquiring the previous frame of image;

The first updating sub-module is used for updating the characteristic points of the previous frame of image according to the first pose information, the initial pose information and the initial three-dimensional characteristic points;

And the second updating sub-module is used for updating the characteristic points of the current frame image according to the characteristic points updated by the previous frame image and the first pose information.

In other embodiments, the apparatus further comprises:

The second acquisition module is used for acquiring the characteristic points of the initial frame image;

The first determining module is used for determining initial pose information of the acquisition device and initial three-dimensional characteristic points corresponding to the characteristic points of the initial frame image when the initial frame image is acquired according to the characteristic points of the initial frame image and training data. The training data comprises two-dimensional feature points extracted according to training images acquired by the target object at different angles and three-dimensional feature points corresponding to the two-dimensional feature points.

In other embodiments, the second acquisition sub-module further includes:

a first determining unit, configured to determine a three-dimensional feature point of a previous frame corresponding to a feature point of the previous frame according to the feature point of the previous frame and the training data;

and the second determining unit is used for determining first pose information of the acquisition device when the last frame of image is acquired according to the characteristic points of the last frame of image, the three-dimensional characteristic points of the last frame of image and the internal parameters of the acquisition device.

In other embodiments, the first update sub-module further comprises:

A third determining unit configured to determine a transformation model from the initial pose information to the first pose information;

the first transformation unit is used for transforming the initial three-dimensional characteristic points into first three-dimensional characteristic points according to the transformation model;

The first mapping unit is used for mapping the first three-dimensional feature points to the previous frame image to obtain mapping points;

and the first updating unit is used for updating the characteristic points of the previous frame of image according to the mapping points.

In other embodiments, the first updating unit further comprises:

The first screening subunit is used for carrying out screening operation on the mapping points according to the size of the previous frame image and/or the first pose information to obtain mapping points matched with the previous frame image;

And the first determination subunit is used for determining the mapping point matched with the previous frame image as the characteristic point after the previous frame image is updated.

In other embodiments, the apparatus further comprises:

The second determining module is used for determining second pose information of the acquisition device and coordinates of the target object in the current frame image when the current frame image is acquired according to the characteristic points of the current frame image when the number of the characteristic points of the current frame image is larger than or equal to a number threshold;

And the third determining module is used for determining the position of the target object according to the initial pose information, the second pose information and the coordinates of the target object in the current frame image.

In other embodiments, the apparatus further comprises:

a fourth determining module, configured to determine whether an end tracking condition is reached;

And the first sending module is used for stopping tracking and sending the position of the target object when the tracking ending condition is determined to be reached.

It should be noted here that: the description of the tracking device embodiment items above, similar to the method description above, has the same advantageous effects as the method embodiment. For technical details not disclosed in the embodiments of the tracking device of the present application, those skilled in the art will understand with reference to the description of the embodiments of the method of the present application.

Embodiments of the present application provide a storage medium having stored therein executable instructions which, when executed by a processor, cause the processor to perform a method provided by embodiments of the present application, for example, as shown in fig. 3, 4 and 5.

In some embodiments, the storage medium may be FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; but may be a variety of devices including one or any combination of the above memories.

In some embodiments, the executable instructions may be in the form of programs, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and they may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

As an example, executable instructions may, but need not, correspond to files in a file system, may be stored as part of a file that holds other programs or data, such as in one or more scripts in a hypertext markup language (HTML, hyper Text Markup Language) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code).

As an example, executable instructions may be deployed to be executed on one computing device or on multiple computing devices located at one site or distributed across multiple sites and interconnected by a communication network.

The foregoing is merely exemplary embodiments of the present application and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A tracking method, comprising:

acquiring characteristic points of a current frame image of a target object;

When the number of the characteristic points of the current frame image is smaller than a number threshold, the characteristic points of the current frame image are updated according to the initial information of the target object, wherein the initial information comprises initial pose information of an acquisition device when the initial frame image is acquired and initial three-dimensional characteristic points corresponding to the characteristic points of the initial frame image, and the number threshold is a minimum value capable of accurately calculating the current pose information of the acquisition device;

continuously tracking the target object according to the characteristic points updated by the current frame image;

the updating the feature point of the current frame image according to the initial information of the target object comprises the following steps:

Acquiring first pose information of the acquisition device when acquiring a previous frame of image;

Determining a transformation model from the initial pose information to the first pose information;

transforming the initial three-dimensional feature points into first three-dimensional feature points according to the transformation model;

Mapping the first three-dimensional feature points to the previous frame of image to obtain mapping points;

Updating the feature points of the previous frame of image according to the mapping points;

And updating the characteristic points of the current frame image according to the characteristic points updated by the previous frame image and the first pose information.

2. The method according to claim 1, wherein the acquiring the feature point of the current frame image of the target object includes:

acquiring characteristic points of a previous frame of image of a target object;

Determining corresponding points of the characteristic points of the previous frame image in the current frame image;

And determining the corresponding points as characteristic points of the current frame image.

3. The method of claim 1, further comprising:

acquiring characteristic points of an initial frame image;

And determining initial pose information of an acquisition device and initial three-dimensional feature points corresponding to the feature points of the initial frame image when the initial frame image is acquired according to the feature points of the initial frame image and training data, wherein the training data comprises two-dimensional feature points extracted from training images acquired at different angles according to the target object and three-dimensional feature points corresponding to the two-dimensional feature points.

4. A method according to claim 3, said acquiring first pose information of the acquisition device when acquiring a previous frame of image, comprising:

Determining a three-dimensional characteristic point of a previous frame corresponding to the characteristic point of the previous frame according to the characteristic point of the previous frame and the training data;

And determining first pose information of the acquisition device when the previous frame of image is acquired according to the characteristic points of the previous frame of image, the three-dimensional characteristic points of the previous frame of image and the internal parameters of the acquisition device.

5. The method of claim 1, the updating feature points of the previous frame image according to the mapping points, comprising:

According to the size of the previous frame image and/or the first pose information, screening the mapping points to obtain mapping points matched with the previous frame image;

And determining the mapping point matched with the previous frame of image as the characteristic point after updating of the previous frame of image.

6. The method of claim 1, further comprising:

when the number of the characteristic points of the current frame image is larger than or equal to a number threshold value, determining second pose information of the acquisition device and coordinates of the target object in the current frame image when the current frame image is acquired according to the characteristic points of the current frame image;

And determining the position of the target object according to the initial pose information, the second pose information and the coordinates of the target object in the current frame image.

7. The method of claim 6, further comprising:

determining whether an end tracking condition is reached;

and stopping tracking when the tracking ending condition is determined to be reached, and sending the position of the target object.

8. A tracking device, comprising:

A memory for storing executable instructions;

a processor for implementing the method of any one of claims 1 to 7 when executing executable instructions stored in said memory.