CN114399528B - Three-dimensional space moving target tracking method and related device based on two-dimensional image - Google Patents
Three-dimensional space moving target tracking method and related device based on two-dimensional image Download PDFInfo
- Publication number
- CN114399528B CN114399528B CN202111433206.XA CN202111433206A CN114399528B CN 114399528 B CN114399528 B CN 114399528B CN 202111433206 A CN202111433206 A CN 202111433206A CN 114399528 B CN114399528 B CN 114399528B
- Authority
- CN
- China
- Prior art keywords
- camera
- coordinate system
- target
- axis
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 230000003287 optical effect Effects 0.000 claims abstract description 31
- 239000011159 matrix material Substances 0.000 claims description 13
- 230000006870 function Effects 0.000 claims description 10
- 230000033001 locomotion Effects 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000012546 transfer Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 14
- 238000005259 measurement Methods 0.000 description 12
- 238000012545 processing Methods 0.000 description 6
- 238000000342 Monte Carlo simulation Methods 0.000 description 5
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000009545 invasion Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
- G06T7/251—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a three-dimensional space moving target tracking method and a related device based on a two-dimensional image. The method comprises the following steps: the camera acquires a current image of a target at the current moment; determining a first position of a target in a pixel coordinate system; determining an observation position of a target under a camera coordinate system according to the first position, the pixel size, the deflection angle of the optical axis of the camera, the height and the focal length of the camera; the origin of the camera coordinate system is the optical center of the camera, the Z axis of the camera coordinate system is arranged along the optical axis of the camera, the X axis and the Y axis of the camera coordinate system are respectively parallel to the X axis and the Y axis of the image coordinate system, and the image coordinate system is positioned on the focal plane of the camera; updating each parameter in the volume Kalman filter by using the estimated position and the observed position, and obtaining the current estimated position of the target under a camera coordinate system; the last estimated position is the estimated position obtained by carrying out track prediction on the target at the last moment. By the method, the accuracy of target tracking can be improved.
Description
Technical Field
The application relates to the field of target tracking, in particular to a three-dimensional space moving target tracking method based on two-dimensional images and a related device.
Background
The use of the sensor to achieve the target positioning is one of the important uses of the sensor, and is also a key link for providing the on-line measurement information of the sensor for target tracking. Currently, based on different sensor types, target positioning can be roughly classified into radar-based, sonar-based, infrared-based, visible-light-image-based and the like. The target positioning has great significance in the real world, plays an important role in detecting the invasion and the movement direction of an enemy target and detecting the occurrence place of an event, and provides powerful guarantee for later correct decision and corresponding measures. The sensor is used for positioning the target, and the position of the target is obtained mainly by using some collected known information about the target and processing the information to a certain extent. The target positioning application range based on the common RGB image is the most wide in consideration of factors such as comprehensive cost, complexity, transmission convenience and the like.
The object tracking is essentially to continuously estimate the characteristics of a moving object through a single sensor or a plurality of sensors, and generally can be used for tracking estimated moving object characteristics including quantity, position, acceleration, speed and other state quantities related to a moving process, and further including the shape, size and other expansion forms of the object.
In summary, the tracking estimation of the moving object is a process of performing fusion estimation on the characteristics of the moving object by processing and converting the on-line measurement information of the sensor and the prior information of the object through the object tracking technology. Target tracking now plays an important role in the fields of missile guidance, urban traffic, intelligent monitoring, virtual reality, medical diagnosis and the like. Among the technical methods for tracking targets, the filtering method is one of the most important research contents, and under the condition that a moving target state equation and an observation equation are known, continuous estimation of the target state can be realized by using the filtering method, a Kalman filter is the optimal method for realizing tracking purposes for a linear Gaussian system, and estimation by means of a nonlinear filtering algorithm, such as an extended Kalman filter and a volume Kalman filter, is needed for the most common nonlinear and non-Gaussian motion system in the real world.
At present, the equipment and the operation method for tracking and positioning the moving target are increasingly complex, the realization cost and the complexity are inevitably greatly improved while the positioning and tracking precision is improved, and the development of a simple moving target positioning and tracking strategy based on a common optical camera is an important approach which is more close to the actual target tracking requirement.
Disclosure of Invention
The application mainly provides a three-dimensional space moving target tracking method and a related device based on a two-dimensional image, which solve the problem of low tracking accuracy of the three-dimensional space moving target based on the two-dimensional image in the prior art.
To solve the above technical problem, a first aspect of the present application provides a three-dimensional space moving object tracking method based on a two-dimensional image, including: acquiring a current image of a target at a current moment by using a camera; determining a first position of the target under a pixel coordinate system where the current image is located; determining an observation position of the target under a camera coordinate system where the camera is located according to the first position, the pixel size under the pixel coordinate system, the deflection angle of the optical axis of the camera, the height of the camera and the focal length; the origin of the camera coordinate system is the optical center of the camera, the Z axis of the camera coordinate system is arranged along the optical axis of the camera, the X axis and the Y axis of the camera coordinate system are respectively parallel to the X axis and the Y axis of the image coordinate system, and the image coordinate system is positioned on the focal plane of the camera; updating each parameter in the volume Kalman filter by using the last estimated position and the observed position, and obtaining the current estimated position of the target under the camera coordinate system; the last estimated position is an estimated position obtained by carrying out track prediction on the target at the last moment.
In order to solve the above technical problem, a second aspect of the present application provides a target tracking device, including a processor and a memory coupled to each other; the memory stores a computer program, and the processor is configured to execute the computer program to implement the three-dimensional space moving object tracking method based on the two-dimensional image provided in the first aspect.
To solve the above-mentioned technical problem, a third aspect of the present application provides a computer-readable storage medium storing program data which, when executed by a processor, implements the three-dimensional space moving object tracking method based on a two-dimensional image provided in the first aspect.
The beneficial effects of the application are as follows: different from the prior art, the camera acquires the current image of the target at the current moment; determining a first position of a target in a pixel coordinate system; determining an observation position of a target under a camera coordinate system according to the first position, the pixel size, the deflection angle of the optical axis of the camera, the height and the focal length of the camera; the origin of the camera coordinate system is the optical center of the camera, the Z axis of the camera coordinate system is arranged along the optical axis of the camera, the X axis and the Y axis of the camera coordinate system are respectively parallel to the X axis and the Y axis of the image coordinate system, and the image coordinate system is positioned on the focal plane of the camera; updating each parameter in the volume Kalman filter by using the estimated position and the observed position, and obtaining the current estimated position of the target under a camera coordinate system; the last estimated position is the estimated position obtained by carrying out track prediction on the target at the last moment. According to the method, the observation position of the target under the camera coordinate system can be obtained rapidly and accurately according to the first position of the target under the image coordinate system, the camera height, the focal length and the deflection angle of the optical axis, and the volume Kalman filter is small in calculated amount, high in estimation accuracy and not prone to divergence, so that the position estimation of the moving target is further realized by the volume Kalman filter based on the high-accuracy observation position, and the accuracy of target tracking is effectively improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic block diagram of a process of one embodiment of a two-dimensional image-based three-dimensional moving object tracking method of the present application;
FIG. 2 is a schematic diagram showing the positional relationship between the camera coordinate system O cXcYcZc and the image coordinate system O ixiyi according to the present application;
FIG. 3 is a schematic block diagram illustrating a flowchart of an embodiment of step S13 of the present application;
FIG. 4 is a schematic diagram of the positional relationship between the object and the camera coordinate system, and the world coordinate system;
FIG. 5 is a schematic block diagram illustrating a flowchart of an embodiment of step S14 of the present application;
FIG. 6 is a partial tracking trace of a target using a volume Kalman filter and an extended Kalman filter in accordance with the present application;
FIG. 7 is a graph of the root mean square error of the trajectory predictions of the volume Kalman filter and the extended Kalman filter of the present application;
FIG. 8 is a graph of the Euclidean distance predicted using a volume Kalman filter and an extended Kalman filter trajectory under the Monte Carlo method of the present application;
FIG. 9 is a graph of the predicted trajectory Manhattan distance using a volume Kalman filter and an extended Kalman filter under the Monte Carlo method of the present application;
FIG. 10 is a block diagram illustrating the construction of an embodiment of the object tracking device of the present application;
FIG. 11 is a block diagram illustrating the structure of one embodiment of a computer-readable storage medium of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly understand that the embodiments described herein may be combined with other embodiments.
Referring to fig. 1, fig. 1 is a schematic block flow diagram of an embodiment of a three-dimensional moving object tracking method based on two-dimensional images according to the present application. It should be noted that, if there are substantially the same results, the present embodiment is not limited to the flow sequence shown in fig. 1. The embodiment comprises the following steps:
step S11: and acquiring a current image of the target at the current moment by using the camera.
This step is based on the camera capturing a current image of the target at the current time, and the camera may be any instrument that can be used to capture the target, such as an industrial camera, a miniature camera, a monitoring camera, etc.
The target described herein is usually a moving target, and the method may acquire images of the target at preset intervals according to a preset manner, and the image acquired at the current time is the current image.
Step S12: and determining a first position of the target under the pixel coordinate system of the current image.
The first position, i.e. the coordinates of the object in the pixel coordinate system, may be denoted (u, v).
Step S13: and determining the observation position of the target under the camera coordinate system where the camera is positioned according to the first position, the pixel size under the pixel coordinate system, the deflection angle of the optical axis of the camera, the height of the camera and the focal length.
Referring to fig. 2, fig. 2 is a schematic diagram illustrating a positional relationship between a camera coordinate system O cXcYcZc and an image coordinate system O ixiyi according to the present application. The origin O c of the camera coordinate system is the optical center of the camera, the Z axis of the camera coordinate system is disposed along the optical axis of the camera, the X axis and the Y axis of the camera coordinate system are parallel to the X axis and the Y axis of the image coordinate system, respectively, and the image coordinate system is located on the focal plane of the camera.
Wherein, P (X c,Yc,Zc) is the position of the target under the camera coordinate system, P (X, Z) is the projection position of the target on the X cOcZc plane, P (Y, Z) is the projection of the target on the Y cOcZc plane, P (X i,yi) is the position of the target under the image coordinate system, beta is the pitch angle of the target, and alpha is the azimuth angle of the target.
Referring to fig. 3, fig. 3 is a schematic block diagram illustrating a flow of an embodiment of step S13 of the present application. It should be noted that, if there are substantially the same results, the embodiment is not limited to the flow sequence shown in fig. 3. The embodiment comprises the following steps:
step S131: and determining the pitch angle and the azimuth angle of the target according to the first position, the pixel size under the pixel coordinate system, the height of the camera and the focal length.
The azimuth angle is a vector formed by the optical center of the camera and the target, the projection on the XOZ plane of the camera under the camera coordinate system where the camera is located forms an included angle with the optical axis of the camera, and the pitch angle is a vector formed by the optical center and the target, and the projection on the YOZ plane of the camera coordinate system forms an included angle with the optical axis of the camera.
Referring to fig. 4, fig. 4 is a schematic diagram showing the positional relationship between the object and the camera coordinate system, and the world coordinate system. Wherein, O cXcYcZc is a camera coordinate system, O wXwYwZw is a world coordinate system, the origin of the camera coordinate system is coincident with the origin of the world coordinate system, the plane of the camera coordinate system Y cOcZc is coincident with the plane of the world coordinate system Y wOwZw, the Z axis of the world coordinate system is arranged along the horizontal direction, and the Y axis of the world coordinate system is arranged along the vertical direction. A represents the position of the mass center of the target, B is the projection of the target on the Z c axis of the camera coordinate system, and C is the projection of the target on the Y c axis of the camera coordinate system.
In fig. 4, β is a pitch angle, θ is a deflection angle of an optical axis of the camera, h is a height of the camera, and an azimuth angle is not shown.
Specifically, the azimuth and pitch angles are determined according to the following two equations:
Where α is azimuth angle, β is pitch angle, (u 0,v0) is the position of the image coordinate system in the pixel coordinate system, d represents the pixel size, f represents the focal length of the camera, and (u, v) is the first position of the target in the pixel coordinate system.
Step S132: and determining the Y-axis coordinate of the target in the camera coordinate system through a trigonometric function formula according to the deflection angle, the height of the camera and the pitch angle of the target.
Specifically, the Y-axis coordinates of the target in the camera coordinate system are determined according to the following equation:
Wherein Y c is the Y-axis coordinate of the target in the camera coordinate system.
Step S133: and determining the Z-axis coordinate of the target under the camera coordinate system according to the Y-axis coordinate and the pitch angle.
Specifically, the Z-axis coordinate of the target in the camera coordinate system is determined according to the following equation:
wherein Y c is the Z-axis coordinate of the target in the camera coordinate system.
Step S134: and determining the X-axis coordinate of the target under the camera coordinate system according to the Z-axis coordinate and the azimuth angle of the target.
Specifically, the X-axis coordinates of the target in the camera coordinate system are determined according to the following equation:
Xc＝Zctanα
Wherein X c is the X-axis coordinate of the target in the camera coordinate system.
According to the method, the observation position (X c,Yc,Zc) of the target under the camera coordinate system can be obtained.
Step S14: and updating each parameter in the volume Kalman filter by using the estimated position and the observed position, and obtaining the current estimated position of the target under the camera coordinate system.
The state transfer equation and the observation equation of the volume kalman filter algorithm are as follows:
Wherein f (X K-1,UK-1) is a state function, h (X K) is a measurement function, X K and Z K are respectively a state variable and an observation variable at the moment K, U K-1 is an external input at the moment K-1, omega K-1,νK is Gaussian white noise with zero additive mean value, process noise at the moment K-1 and measurement noise at the moment K are represented, and covariance of the process noise at the moment K and the measurement noise at the moment K is Q K,RK.
In particular, for nonlinear systems, the posterior probability density function of nonlinear systems at time K is knownAnd then, the execution steps of parameter updating for the iteration of the volume Kalman filter are as follows:
1. time update
1) Decomposing an estimation error covariance matrix:
PK|K＝SK|K(SK|K)T。
2) Calculating volume points:
3) Calculating a state transfer function propagation volume point:
wherein, X i,K|K is a group of the formula, All are volume points. m is the number of volume points and should be guaranteed to be 2 times the nonlinear system state vector dimension n when using the third order spherical radial criterion. Xi i is the basic volume point set,/>[1] Is a set of points in n-dimensional space, wherein/>[1] i Is the ith column in the point set [1 ].
4) Calculating a state prediction value:
5) Calculating a prediction covariance matrix:
2. Measurement update
6) Decomposing the prediction covariance matrix:
PK+1|K＝SK+1|K(SK+1|K)T。
7) Calculating volume points:
8) Calculating the propagation volume point of the measurement function:
Zi,K+1|K＝hK+1(Xi,K+1|K)，i＝1，2，…m。
9) Calculating a measurement predicted value:
10 Calculating innovation:
Wherein: z K+1 is a measurement at time K+1.
11 Calculating an innovation covariance matrix:
12 Calculating a cross covariance matrix:
13 Calculating the volume kalman filter gain:
KK+1＝PXY,K+1|K(PYY,K+1|K)-1。
14 Calculating a state estimation value at the moment K+1):
15 Calculating an estimation error covariance matrix):
Taking K as the previous time and K+1 as the current time as an example, the above steps iteratively update parameters in the volume Kalman filter.
Wherein, the estimated value of the state at the time of K+1 calculated in the step 14), that is, the output value of the volume Kalman filter, corresponds to the estimated position at the current time of the present application.
Specifically, referring to fig. 5, fig. 5 is a schematic block diagram illustrating a flow chart of an embodiment of step S14 of the present application. It should be noted that, if there are substantially the same results, the embodiment is not limited to the flow sequence shown in fig. 5. The embodiment comprises the following steps:
step S141: and according to the last estimated position, calculating at least one first propagation volume point by using a state transfer equation, and taking the average value of the propagation volume points as a predicted position.
The last estimated position is an estimated position obtained by carrying out track prediction on the target at the last moment, namely, the output of the volume Kalman filter at the last moment.
Optionally, the first propagation volume point is calculated using a state transition equation represented by:
Wherein, The first propagation volume point is represented by k+1, K is the current time, X i,K|K is the volume point in the time update, i=1, 2, … m, m is the number of volume points, and T is the time interval between the current time and the previous time. Omega K is gaussian white noise with an average value of 0 and G is the noise driving matrix.
Wherein, gaussian white noise omega meets normal distribution with mean value of 0 and variance of Q; and, in addition, the method comprises the steps of,
Wherein,Ζ is the amplitude and frequency, respectively, of the sinusoidal motion of the object.
Finally, obtaining the predicted position according to the formula in the iterative updating step 4)
Step S142: and calculating at least one second propagation volume point by using an observation equation according to the predicted position, and taking the average value of the propagation volume points as an observation predicted value.
Decomposing the prediction covariance matrix according to the sequence of the iterative updating steps 6) to 8), calculating to obtain volume points in measurement updating, calculating to obtain a second propagation volume point Z i,K+1|K by using an observation equation, i=1, 2, … m, and finally calculating to obtain an observation predicted value according to the formula in the step 9)
Wherein the observation equation may be expressed in the form of:
wherein, (X X,YY,ZZ,VX,VY,VZ) is a vector representation of the volume point in the measurement update, and each parameter corresponds to the coordinate of the target along the X C,YC,ZC axis and the speed along the X C,YC,ZC direction of the coordinate axis under the camera coordinate system.
Step S143: and calculating information based on the observed position and the observed predicted value, and updating the gain of the volume Kalman filter.
Wherein the update of the innovation corresponds to the iterative update step 10), equationWherein Z K+1 is the current observation position of the target under the camera coordinate system,/>I.e. the observed predicted value, and the result e K+1 is the updated value of the innovation.
The gain update method corresponds to the iterative update step 13).
Step S144: and calculating according to the gain, the innovation and the predicted position to obtain the current estimated position.
Referring to the update step 14 described above),I.e. the current estimated position. The updating step iterates according to the sequence, and after the last output result (namely the current estimated position) and the current observation position of the target are obtained, the updating step iterates continuously, and finally the estimated position of the target at each moment can be obtained, so that the target tracking from the two-dimensional image to the three-dimensional space is realized.
The iterative manner of the other parameters of the volume kalman filter corresponds to the above steps 1) to 15), and is not repeated here.
The current estimated position obtained by the method is the current coordinate of the target under the camera coordinate system, and the real position of the target under the world coordinate can be calculated according to the conversion relation between the camera coordinate system and the world coordinate system.
The real position of the target in the world coordinate system can be calculated according to the following formula:
Wherein Xw, yw and Zw are the X-axis coordinate, Y-axis coordinate and Z-axis coordinate of the real position corresponding to the world coordinate system, xc, yc and Zc are the X-axis coordinate, Y-axis coordinate and Z-axis coordinate of the current estimated position corresponding to the camera coordinate system, respectively, and θ is the deflection angle of the camera optical axis.
Optionally, after the actual position of the target at the current time is obtained in the above steps, the next time is the current time, and the steps of the above embodiments are continued to obtain multiple actual positions of the target, and interpolation fitting is performed on the multiple positions, so that a motion curve of the target in the three-dimensional space can be obtained, where the motion curve can represent motion track information of the target in the three-dimensional space.
Compared with the prior art, the method and the device have the advantages that the image of the target obtained by shooting by using the camera only can directly obtain the observation position of the target under the camera coordinate system according to the position of the target in the pixel coordinate system where the current image is, the pixel size, the focal length and the height of the camera and the deflection angle of the optical axis of the camera, then the estimated position of the target under the camera coordinate system can be obtained by iterating by using the volume Kalman filter, the position of the target in the world coordinate system at the moment can be obtained by coordinate conversion, the position can be used for describing the actual position of the target at the moment, and the accuracy is high. This method is implemented using only a monocular camera in combination with the algorithm described in the above embodiments, without requiring multiple sensors to locate the target, and eliminating the way in which target tracking location relies on multiple sensors.
In one embodiment, the Kalman filter is initialized according to the following manner, wherein X 0 is the initial state quantity of the target, R is the measurement noise covariance matrix, P 0 is the initial value of the state covariance matrix, A is the state transition matrix, and H is the observation matrix:
X0＝[-1850 200 5000 4 1 2]T；
And setting a corresponding volume point set of the 6-dimensional nonlinear system as follows, wherein the weight of each volume point is as follows
When the volume Kalman filter is used for estimating the target motion trail, the sampling time is set to be 0.2s, the total running time is 200s, a Monte Carlo method is added, and the Monte Carlo times are set to be 100. Under the same condition, the target track is tracked by using an extended Kalman filter, and simulation results are shown in fig. 6-9.
FIG. 6 is a graph showing the local tracking of a target using a volume Kalman filter and an extended Kalman filter in accordance with the present application; FIG. 7 is a graph of the present application utilizing a volume Kalman filter and an extended Kalman filter trajectory prediction root mean square error; FIG. 8 is a graph of the Euclidean distance predicted using a volume Kalman filter and an extended Kalman filter trajectory under the Monte Carlo method of the present application; FIG. 9 is a graph of the predicted trajectory Manhattan distance using a volume Kalman filter and an extended Kalman filter under the Monte Carlo method of the present application.
After different simulation experiments are set and experimental results are obtained, we can conclude that the target tracking model and the volume Kalman filter provided by the application can effectively realize track prediction and tracking of a moving target in a three-dimensional space, the error is smaller than that of the extended Kalman filter, and meanwhile, the experimental results also verify the accuracy of the method about a positioning model.
Referring to fig. 10, fig. 10 is a schematic block diagram of an embodiment of the object tracking device of the present application. The object tracking device 200 includes a processor 210 and a memory 220 coupled to each other, and the memory 220 stores a computer program, and the processor 210 is configured to execute the computer program to implement the two-dimensional image-based three-dimensional moving object tracking method according to the above embodiments.
For the description of each step of the processing execution, please refer to the description of each step of the embodiment of the three-dimensional space moving object tracking method based on the two-dimensional image of the present application, and the description is omitted herein.
The memory 220 may be used to store program data and modules, and the processor 210 performs various functional applications and data processing by executing the program data and modules stored in the memory 220. The memory 220 may mainly include a storage program area that may store an operating system, application programs required for at least one function (such as an image recognition function, a coordinate conversion function, etc.), and a storage data area; the storage data area may store data created according to the use of the object tracking device 200 (such as image data, coordinate position information, camera parameter information, object trajectories, etc.), and the like. In addition, memory 220 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 220 may also include a memory controller to provide the processor 210 with access to the memory 220.
In the embodiments of the present application, the disclosed method and apparatus may be implemented in other manners. For example, the embodiments of the object tracking device 200 described above are merely illustrative, e.g., the division of the modules or units is merely a logical functional division, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on this understanding, the technical solution of the application, or the part contributing to the prior art or all or part of the technical solution, may be embodied in the form of a software product, which is stored in a storage medium.
Referring to fig. 11, fig. 11 is a schematic block diagram illustrating the structure of an embodiment of a computer readable storage medium 300 according to the present application, where the computer readable storage medium 300 stores program data 310, and the program data 310 when executed implements the steps of the embodiments of the three-dimensional moving object tracking method based on two-dimensional images as described above.
For the description of each step of the processing execution, please refer to the description of each step of the embodiment of the three-dimensional space moving object tracking method based on the two-dimensional image of the present application, and the description is omitted herein.
The computer readable storage medium 300 may be a usb disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, etc. various media that can store program codes.
The foregoing description is only illustrative of the present application and is not intended to limit the scope of the application, and all equivalent structures or equivalent processes or direct or indirect application in other related technical fields are included in the scope of the present application.
Claims (7)
1. A method for tracking a three-dimensional moving object based on a two-dimensional image, the method comprising:
Acquiring a current image of a target at a current moment by using a camera;
Determining a first position of the target under a pixel coordinate system where the current image is located;
Determining an observation position of the target under a camera coordinate system where the camera is located according to the first position, the pixel size under the pixel coordinate system, the deflection angle of the optical axis of the camera, the height of the camera and the focal length; the origin of the camera coordinate system is the optical center of the camera, the Z axis of the camera coordinate system is arranged along the optical axis of the camera, the X axis and the Y axis of the camera coordinate system are respectively parallel to the X axis and the Y axis of the image coordinate system, and the image coordinate system is positioned on the focal plane of the camera;
according to the last estimated position, calculating at least one first propagation volume point by using a state transfer equation, and taking the average value of the propagation volume points as a predicted position;
Calculating at least one second propagation volume point by using an observation equation according to the predicted position, and taking the average value of the propagation volume points as an observation predicted value;
Calculating innovation based on the observed position and the observed predicted value, and updating the gain of the volume Kalman filter;
calculating according to the gain, the news and the predicted position to obtain a current estimated position; the last estimated position is an estimated position obtained by carrying out track prediction on the target at the last moment;
wherein the first propagation volume point is calculated using the following equation:
Wherein, Represents a first propagation volume point, K+1 represents the current moment, K is the last moment,/>Is a volume point in a time update, wherein/>M represents the number of volume points, T is the time interval between the last moment and the current moment,/>Is Gaussian white noise with the average value of 0, and G is a noise driving matrix;
The Gaussian white noise Satisfies normal distribution with a mean value of 0 and a variance of Q; and, in addition, the method comprises the steps of,
，/>
Wherein,、/>Respectively controlling the amplitude and the frequency of the target in the sinusoidal motion process.
2. The method of claim 1, wherein determining the observed position of the target in the camera coordinate system in which the camera is located based on the first position, the pixel size in the pixel coordinate system, the angle of deflection of the optical axis of the camera, the height of the camera, and the focal length comprises:
Determining a pitch angle and an azimuth angle of the target according to the first position, the pixel size in the pixel coordinate system, the height of the camera and the focal length; the azimuth angle is a vector formed by an optical center of the camera and the target, the projection of the camera on an XOZ plane under a camera coordinate system where the camera is located is an included angle with an optical axis of the camera, and the pitch angle is a vector formed by the optical center and the target, and the projection of the camera on a YOZ plane under the camera coordinate system is an included angle with the optical axis of the camera;
Determining Y-axis coordinates of the target in the camera coordinate system through a trigonometric function formula according to the deflection angle, the height of the camera and the pitch angle of the target;
determining a Z-axis coordinate of the target under the camera coordinate system according to the Y-axis coordinate and the pitch angle;
And determining the X-axis coordinate of the target under the camera coordinate system according to the Z-axis coordinate and the azimuth angle of the target.
3. The method of claim 2, wherein determining the pitch and azimuth of the target based on the first location, the pixel size in the pixel coordinate system, the height of the camera, and the focal length comprises:
determining the azimuth angle according to:
And determining the pitch angle according to the formula:
Wherein, For the azimuth angle,/>For the pitch angle,/>For the position of the image coordinate system in the pixel coordinate system, d represents the pixel size, f represents the focal length of the camera,/>Is a first position of the target in the pixel coordinate system.
4. The method according to claim 1, wherein the method further comprises:
Calculating the real position of the target in a world coordinate system according to the current estimated position of the target in the camera coordinate system; wherein the origin of the world coordinate system coincides with the origin of the camera coordinate system, and the Y-O-Z plane of the world coordinate system is coplanar with the Y-O-Z plane of the camera coordinate system.
5. The method of claim 4, wherein a Z-axis of the world coordinate system is disposed in a horizontal direction and a Y-axis of the world coordinate system is disposed in a vertical direction;
the calculating the real position of the target in the world coordinate system according to the current estimated position of the target in the camera coordinate system comprises the following steps:
the true position is calculated according to the following formula:
Wherein Xw, yw and Zw are respectively the X-axis coordinate, Y-axis coordinate and Z-axis coordinate of the real position corresponding to the world coordinate system, xc, yc and Zc are respectively the X-axis coordinate, Y-axis coordinate and Z-axis coordinate of the current estimated position corresponding to the camera coordinate system, Is the angle of deflection of the optical axis of the camera.
6. An object tracking device comprising a processor and a memory coupled to each other; the memory has stored therein a computer program, the processor being adapted to execute the computer program to carry out the steps of the method according to any of claims 1-5.
7. A computer readable storage medium, characterized in that the computer readable storage medium stores program data, which when executed by a processor, implements the steps of the method according to any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111433206.XA CN114399528B (en) | 2021-11-29 | 2021-11-29 | Three-dimensional space moving target tracking method and related device based on two-dimensional image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111433206.XA CN114399528B (en) | 2021-11-29 | 2021-11-29 | Three-dimensional space moving target tracking method and related device based on two-dimensional image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114399528A CN114399528A (en) | 2022-04-26 |
| CN114399528B true CN114399528B (en) | 2024-06-07 |
Family
ID=81225700
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111433206.XA Active CN114399528B (en) | 2021-11-29 | 2021-11-29 | Three-dimensional space moving target tracking method and related device based on two-dimensional image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114399528B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115511955B (en) * | 2022-11-18 | 2023-03-10 | 西湖大学 | Distributed ground-to-air cooperative unknown unmanned aerial vehicle state estimation method and application thereof |
| CN116091546B (en) * | 2023-01-12 | 2024-04-19 | 北京航天飞行控制中心 | Observation construction method under push-broom mode of optical camera |
| CN117032068B (en) * | 2023-07-24 | 2024-02-27 | 苏州福斯特万电子科技有限公司 | Control method, device and equipment of dispensing machine and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108038868A (en) * | 2017-10-17 | 2018-05-15 | 国网河南省电力公司郑州供电公司 | Across the visual field method for tracking target of substation's complex environment based on three-dimensional digital model |
| CN109902543A (en) * | 2017-12-11 | 2019-06-18 | 北京京东尚科信息技术有限公司 | Target trajectory estimation method, device and Target Tracking System |
| CN111563919A (en) * | 2020-04-03 | 2020-08-21 | 深圳市优必选科技股份有限公司 | Target tracking method and device, computer readable storage medium and robot |
| CN112037260A (en) * | 2020-08-31 | 2020-12-04 | 深圳市道通智能航空技术有限公司 | Position estimation method and device for tracking target and unmanned aerial vehicle |
| CN112363167A (en) * | 2020-11-02 | 2021-02-12 | 重庆邮电大学 | Extended target tracking method based on fusion of millimeter wave radar and monocular camera |
| CN112465877A (en) * | 2020-12-09 | 2021-03-09 | 北京航空航天大学 | Kalman filtering visual tracking stabilization method based on motion state estimation |
| CN113269098A (en) * | 2021-05-27 | 2021-08-17 | 中国人民解放军军事科学院国防科技创新研究院 | Multi-target tracking positioning and motion state estimation method based on unmanned aerial vehicle |
-
2021
- 2021-11-29 CN CN202111433206.XA patent/CN114399528B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108038868A (en) * | 2017-10-17 | 2018-05-15 | 国网河南省电力公司郑州供电公司 | Across the visual field method for tracking target of substation's complex environment based on three-dimensional digital model |
| CN109902543A (en) * | 2017-12-11 | 2019-06-18 | 北京京东尚科信息技术有限公司 | Target trajectory estimation method, device and Target Tracking System |
| CN111563919A (en) * | 2020-04-03 | 2020-08-21 | 深圳市优必选科技股份有限公司 | Target tracking method and device, computer readable storage medium and robot |
| CN112037260A (en) * | 2020-08-31 | 2020-12-04 | 深圳市道通智能航空技术有限公司 | Position estimation method and device for tracking target and unmanned aerial vehicle |
| CN112363167A (en) * | 2020-11-02 | 2021-02-12 | 重庆邮电大学 | Extended target tracking method based on fusion of millimeter wave radar and monocular camera |
| CN112465877A (en) * | 2020-12-09 | 2021-03-09 | 北京航空航天大学 | Kalman filtering visual tracking stabilization method based on motion state estimation |
| CN113269098A (en) * | 2021-05-27 | 2021-08-17 | 中国人民解放军军事科学院国防科技创新研究院 | Multi-target tracking positioning and motion state estimation method based on unmanned aerial vehicle |
Non-Patent Citations (1)
| Title |
|---|
| A Moving Target Detection and Localization Strategy Based on Optical Flow and Pin-hole Imaging Methods Using Monocular Vision;Shun Wang et al.;《IEEE》;20210831;147-152 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114399528A (en) | 2022-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114399528B (en) | Three-dimensional space moving target tracking method and related device based on two-dimensional image | |
| CN110880189B (en) | Combined calibration method and combined calibration device thereof and electronic equipment | |
| US10109104B2 (en) | Generation of 3D models of an environment | |
| Davison | Real-time simultaneous localisation and mapping with a single camera | |
| Janabi-Sharifi et al. | A kalman-filter-based method for pose estimation in visual servoing | |
| JP4079690B2 (en) | Object tracking apparatus and method | |
| JP5012615B2 (en) | Information processing apparatus, image processing method, and computer program | |
| CN110501036A (en) | The calibration inspection method and device of sensor parameters | |
| CN110827321B (en) | Multi-camera collaborative active target tracking method based on three-dimensional information | |
| KR101793975B1 (en) | Method and apparatus of camera tracking for streaming images from depth camera | |
| CN107391446A (en) | Irregular shape based on random matrix extends target shape and method for estimating state more | |
| CN112067007B (en) | Map generation method, computer storage medium, and electronic device | |
| Dinc et al. | Mirage: an O (n) time analytical solution to 3D camera pose estimation with multi-camera support | |
| EP3098682B1 (en) | Moving object controller, program, and integrated circuit | |
| CN116774589A (en) | Visual servo target tracking control method for robust nonlinear model predictive control | |
| Hou et al. | Octree-based approach for real-time 3d indoor mapping using rgb-d video data | |
| Ababsa et al. | Hybrid three-dimensional camera pose estimation using particle filter sensor fusion | |
| Jurado et al. | Inertial and imaging sensor fusion for image-aided navigation with affine distortion prediction | |
| CN112578363B (en) | Laser radar motion track obtaining method and device and medium | |
| Ming et al. | A real-time monocular visual SLAM based on the bundle adjustment with adaptive robust kernel | |
| Ashikuzzaman et al. | Fast and robust localization of surgical array using Kalman filter | |
| CN117495900B (en) | Multi-target visual tracking method based on camera motion trend estimation | |
| Singhirunnusorn et al. | Single‐camera pose estimation using mirage | |
| Gong | Kalman Filter-based Signal Processing for Robot Target Tracking | |
| Xiao et al. | Combining monocular camera and 2D lidar for target tracking using deep convolution neural network based detection and tracking algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |