CN115098609A

CN115098609A - Multi-sensor combined space-time deviation calibration and multi-target association fusion method and device

Info

Publication number: CN115098609A
Application number: CN202210695174.9A
Authority: CN
Inventors: 周共健; 卜石哲
Original assignee: Harbin Institute of Technology
Current assignee: Harbin Institute of Technology
Priority date: 2022-06-20
Filing date: 2022-06-20
Publication date: 2022-09-23

Abstract

The embodiment of the invention relates to the technical field of data processing, in particular to a multi-sensor combined space-time deviation calibration and multi-target association fusion method and device. The method comprises the following steps: determining an extended dimension state estimation sampling point set and a weight set corresponding to each track in the track list based on the track list at the previous moment, and calculating the association cost of each pair of track-observation data combination by combining the observation data list at the current moment to associate the tracks and the observation data to obtain an association result list, an unassociated track list and an unassociated observation data list; respectively updating the flight paths in the unassociated flight path list and the associated result list to obtain an unassociated updated flight path list and an associated flight path list at the current moment; and performing space-time deviation feedback type fusion processing on the associated track list to obtain a fused associated track list so as to obtain a track list at the current moment. The invention can realize effective multi-sensor multi-target tracking.

Description

Multi-sensor combined space-time deviation calibration and multi-target association fusion method and device

Technical Field

The embodiment of the invention relates to the technical field of data processing, in particular to a multi-sensor combined space-time deviation calibration and multi-target association fusion method and device.

Background

The multi-sensor data fusion method can enhance the overall performance of the system, and thus has been widely used in many fields. Deviation compensation and data association are two important preconditions for obtaining accurate data fusion results, and the two are highly coupled. The deviation compensation depends on a multi-sensor observation data set which is generated by data association and originates from the same target, and the data association can not correctly solve the problem of uncertainty of an observation source after the deviation compensation. Therefore, in order to obtain accurate multi-sensor data fusion results, both need to be processed simultaneously.

In the related art, most of the existing multi-sensor data fusion methods are iterative optimization processing of deviation compensation and data association. The processing mode takes longer time and does not meet the real-time processing requirement of an actual system. In the prior art, only the conditions of space deviation and observation source uncertainty of the sensor are considered, and the influence of time deviation on the fusion performance is not considered. Therefore, under the condition that space-time deviation and observation source uncertainty exist simultaneously, the accuracy of a data fusion result generated by the existing related technology is not high, and effective multi-sensor multi-target tracking is difficult to realize.

Therefore, a new method for multi-sensor space-time offset calibration and correlation fusion is needed.

Disclosure of Invention

In order to solve the problem that the fusion result of the original multi-sensor data fusion method is low in precision, the embodiment of the invention provides a multi-sensor combined space-time deviation calibration and multi-target association fusion method and device.

In a first aspect, an embodiment of the present invention provides a multi-sensor joint space-time offset calibration and multi-target association fusion method, including:

determining a dimension expansion state estimation sampling point set and a weight set corresponding to each track in a track list based on the track list at the previous moment; the flight path list comprises dimension expansion state estimation and dimension expansion state estimation covariance of a plurality of flight paths corresponding to a plurality of targets; the dimension expansion state estimation comprises a basic state estimation and a space-time deviation estimation;

calculating the associated cost of each pair of track-observation data combinations based on the extended dimension state estimation sampling point set and the weight set corresponding to each track at the previous moment and the observation data list at the current moment; the observation data list comprises a plurality of observation data measured by the sensor on the target or the clutter;

associating the flight path with the observation data based on the association cost and the one-to-one constraint relation between the flight path and the observation data to obtain an association result list, an unassociated flight path list and an unassociated observation data list;

respectively updating the unassociated flight path list and the flight path in the association result list to obtain an unassociated updated flight path list at the current moment and an associated flight path list at the current moment;

performing space-time deviation feedback type fusion processing on the associated track list to obtain a fused associated track list;

merging the unassociated updated track list and the fused associated track list, and managing the tracks in the merged list to obtain a survival track list at the current moment;

and carrying out non-deflection conversion on each observation data in the unassociated observation data list to construct the dimension expansion state estimation and the dimension expansion state estimation covariance corresponding to the new track to obtain a new track list, and merging the new track list and the survival track list to obtain a track list at the current moment.

Preferably, the calculating the associated cost of each pair of track-observation data combinations based on the set of extended-dimension state estimation sampling points and the set of weights corresponding to each track at the previous time and the observation data list at the current time includes:

for each pair of track-observation data combinations, performing:

determining an observation function of the sensor according to the flight path and the observation data in the pair of flight path-observation data combinations and the sensor corresponding to the measurement observation data list;

according to the observation function, the dimension-expansion state estimation sampling point set, the weight set and the observation data in the pair of track-observation data combinations, the observation prediction error and the observation prediction covariance of the pair of track-observation data combinations are determined;

calculating the mahalanobis distance of the pair of track-observation data combinations according to the observation prediction error and the observation prediction covariance of the pair of track-observation data combinations;

and calculating the association cost of the pair of track-observation data combinations according to the Mahalanobis distance.

Preferably, the observation function is determined according to the following formula:

where p is the track number, s is the sensor number, X _p Is the extended dimension state estimate of the track, k being the time, (x) _s,p (k),y _s,p (k) For the true position of the track at the time of the observation, (x) _s ,y _s ) For the position of sensor s, Δ r _s (k) And Δ θ _s (k) Is the distance deviation and angle deviation, x, of the sensor s _p (k) And y _p (k) Respectively the position of the flight path p in the x and y directions,

and

the speed of the track p in the x and y directions, respectively, Δ t _s,l (k) Is the time offset of the sensor S relative to the reference sensor l, where S ═ 1.., S and S ≠ l, S being the number of sensors;

the correlation cost is determined according to the following formula:

in the formula (I), the compound is shown in the specification,

is the associated cost, i, of the pair of track-observation data combinations _s Numbering the observed data of sensor s, u (i) _s ) Is a binary variable when i _s When 0, u (i) _s ) When i is equal to 0 _s When not equal to 0, u (i) _s )＝1，

Is the detection probability, psi, of the sensor s _s Is the area of the observation region of the sensor s;

is the mahalanobis distance of the pair of track-observation data combinations,

is the observed predicted covariance of the pair of track-observed data combinations.

Preferably, the associating the track and the observation data based on the association cost and the one-to-one constraint relationship between the track and the observation data to obtain an association result list, an unassociated track list and an unassociated observation data list includes:

establishing a two-dimensional distribution model of track-observation data association according to the association cost of each pair of track-observation data combination and the one-to-one constraint relation between the track and the observation data;

and determining an associated result list, an unassociated track list and an unassociated observation data list by adopting a generalized auction algorithm.

Preferably, the updating the tracks in the unassociated track list and the association result list respectively to obtain the unassociated updated track list at the current time and the associated track list at the current time includes:

calculating the dimension-expanding state prediction covariance of each track in the unassociated track list, and taking the dimension-expanding state prediction covariance of each track as the dimension-expanding state estimation covariance and the dimension-expanding state estimation covariance of each track at the current moment to obtain an unassociated updated track list at the current moment; the unassociated updated track list comprises the dimension expansion state estimation and the dimension expansion state estimation covariance of each track at the current moment;

filtering each observation data in the association result list by adopting an unscented Kalman filtering method to obtain the dimension expansion state estimation and the dimension expansion state estimation covariance of each flight path associated with each observation data at the current moment so as to obtain an associated flight path list at the current moment; the associated track list includes a dimension extended state estimate and a dimension extended state estimate covariance for a current time of each track.

Preferably, the performing feedback type fusion processing of space-time deviation on the associated track list to obtain a fused associated track list includes:

aiming at each track with the updating times larger than a first set value in the associated track list, executing the following steps:

extracting the space-time deviation estimation and the space-time deviation estimation covariance of each flight path with the updating times larger than a first set value at the current moment so as to calculate the space-time deviation estimation after the flight paths are fused and the space-time deviation estimation covariance after the flight paths are fused;

replacing the space-time deviation estimation originally in the dimension-extended state estimation at the current moment with the space-time deviation estimation after the flight path is fused to obtain the dimension-extended state estimation after the flight path is fused;

calculating the covariance between the space-time deviation estimation after the flight path fusion and the basic state estimation of the flight path by adopting a non-track transformation method to obtain the covariance of the dimension-expanding state estimation after the flight path fusion;

obtaining the fused track; the fused track comprises fused dimension-expanding state estimation and fused dimension-expanding state estimation covariance;

and replacing the corresponding track in the associated track list by each fused track with the updating times of which is more than the first set value to obtain a fused associated track list.

Preferably, the managing the tracks in the merged list to obtain the surviving track list at the current time includes:

determining the updating times of each track in the merged list, and if the updating times are equal to a second set value and the updating times which are not associated with the observation data are greater than the specified times, removing the track from the merged list; the specified times are less than the second set value;

if the updating times are larger than a second set value and the latest set times are not associated with the observation data, the flight path is cancelled from the merged list;

and obtaining a survival track list at the current moment.

In a second aspect, an embodiment of the present invention further provides a multi-sensor joint space-time offset calibration and multi-target association fusion apparatus, including:

the determining unit is used for determining a dimension expansion state estimation sampling point set and a weight set corresponding to each flight path in the flight path list based on the flight path list at the previous moment; the flight path list comprises dimension expansion state estimation and dimension expansion state estimation covariance of a plurality of flight paths corresponding to a plurality of targets; the dimension expansion state estimation comprises a basic state estimation and a space-time deviation estimation;

the calculation unit is used for calculating the associated cost of each pair of track-observation data combination based on the dimension-extended state estimation sampling point set and the weight set corresponding to each track at the previous moment and the observation data list at the current moment; the observation data list comprises a plurality of observation data measured by the sensor on the target or the clutter;

the association unit is used for associating the flight path with the observation data based on the association cost and the one-to-one constraint relation between the flight path and the observation data to obtain an association result list, an unassociated flight path list and an unassociated observation data list;

the updating unit is used for respectively updating the unassociated track list and the tracks in the association result list to obtain an unassociated updated track list at the current moment and an associated track list at the current moment;

the fusion unit is used for performing space-time deviation feedback type fusion processing on the associated track list to obtain a fused associated track list;

the management unit is used for merging the unassociated updated track list and the fused associated track list, managing the tracks in the merged list and obtaining a survival track list at the current moment;

and the generating unit is used for carrying out deflection-free conversion on each observation data in the unassociated observation data list so as to construct the dimension expansion state estimation and the dimension expansion state estimation covariance corresponding to the new track to obtain a new track list, and combining the new track list and the survival track list to obtain the track list at the current moment.

In a third aspect, an embodiment of the present invention further provides an electronic device, which includes a memory and a processor, where the memory stores a computer program, and the processor executes the computer program to implement the method according to any embodiment of this specification.

In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed in a computer, the computer program causes the computer to execute the method described in any embodiment of the present specification.

The embodiment of the invention provides a multi-sensor combined space-time deviation calibration and multi-target association fusion method and device, which comprises the steps of firstly determining a dimension expansion state estimation sampling point set and a weight set corresponding to each flight path based on dimension expansion state estimation of each flight path in a flight path list at the previous moment, wherein the dimension expansion state estimation comprises space-time deviation estimation and target basic state estimation; then, combining an observation data list reported by a sensor at the current moment, calculating the association cost of each pair of track-observation data combination, associating the track with the observation data according to the association cost to generate an association result list, an unassociated track list and an unassociated observation data list, updating the tracks in the association result list and the unassociated track list, then, performing space-time deviation feedback type fusion processing on the updated associated track list to further improve the space-time deviation compensation effect, and finally, generating a new track by managing the track and using each observation data in the unassociated observation data list to obtain the track list at the current moment. According to the scheme, the space-time deviation is expanded to the basic state vector of the target, a state space model under the space-time deviation condition is established, the joint data association and deviation calibration problem can be converted into the classic data association and filtering problem under the unified Bayesian framework based on the model, iterative optimization processing between deviation calibration and data association is not needed, the compensation performance of the space-time deviation is further improved through feedback type fusion processing, and effective multi-sensor multi-target tracking is further achieved under the condition that the space-time deviation exists.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.

Fig. 1 is a flowchart of a multi-sensor joint space-time offset calibration and multi-target association fusion method according to an embodiment of the present invention;

FIG. 2 is a graph comparing time offset estimates (RMSEs) for three methods provided by an embodiment of the present invention;

FIG. 3 is a graph comparing the distance deviation estimates RMSEs for three methods provided by an embodiment of the present invention;

FIG. 4 is a graph comparing results of angle deviation estimation RMSEs for three methods provided by an embodiment of the present invention;

FIG. 5 is a graph comparing results of RMSEs for target position estimation in three ways provided by an embodiment of the present invention;

FIG. 6 is a graph comparing results of RMSEs for target speed estimates for three methods provided by one embodiment of the present invention;

FIG. 7 is a graph of the comparison of the probability of correct association for three methods provided by an embodiment of the present invention;

FIG. 8 is a diagram illustrating a hardware architecture of an electronic device according to an embodiment of the present invention;

fig. 9 is a structural diagram of a multi-sensor joint space-time offset calibration and multi-target association fusion apparatus according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer and more complete, the technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention, and based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative efforts belong to the scope of the present invention.

As described above, in the related art, when the problem that uncertainty of an observation source and sensor deviation exist simultaneously is solved, iterative optimization processing of data association and deviation estimation is generally performed, but the problems that an algorithm is long in time consumption and not in line with real-time processing requirements of an actual system are faced, and the problem of time deviation existing in multiple sensors is not considered; when multi-sensor observation data containing space-time deviation and observation source uncertainty are processed by the related technology, the problem that the filtering fusion precision is seriously reduced can be faced, and effective multi-sensor multi-target tracking is difficult to realize.

In order to solve the technical problem, the inventor can consider that the space-time deviation is expanded into a basic state vector of a target, a state space model under the condition that the space-time deviation exists is established, and based on the state space model, the combined data association and deviation compensation problem can be converted into a data association and filtering problem under a unified Bayes framework without iterative optimization processing between data association and deviation estimation; the inventor associates the observation data reported by each sensor with the track sequentially, namely, estimates a sampling point set and a weight set based on the dimension expansion state corresponding to each track at the previous moment and an observation data list reported by the sensor at the current moment, calculates the association cost of each track at the previous moment and each observation data in the observation data list at the current moment, adopts a generalized auction algorithm to complete data association, generates an association result list, an unassociated track list and an unassociated observation data list, updates the tracks in the association result list and the unassociated track list, then performs feedback type fusion processing of space-time deviation on the updated associated track list to further improve the effect of space-time deviation compensation, and finally generates a new track by managing the track and using each observation data in the unassociated observation data list, to obtain the track list at the current moment. Therefore, the scheme can complete accurate multi-sensor combined space-time deviation calibration and multi-target association fusion, and can realize effective multi-sensor multi-target tracking under the condition of space-time deviation.

Specific implementations of the above concepts are described below.

Referring to fig. 1, an embodiment of the present invention provides a multi-sensor joint space-time offset calibration and multi-target association fusion method, where the method includes:

step 100: determining a dimension expansion state estimation sampling point set and a weight set corresponding to each track in a track list based on the track list at the previous moment; the flight path list comprises dimension expansion state estimation and dimension expansion state estimation covariance of a plurality of flight paths corresponding to a plurality of targets; the dimension expansion state estimation comprises basic state estimation and space-time deviation estimation;

step 102: calculating the associated cost of each pair of track-observation data combinations based on the extended dimension state estimation sampling point set and the weight set corresponding to each track at the previous moment and the observation data list at the current moment; the observation data list comprises a plurality of observation data measured by the sensor on the target or the clutter;

step 104: associating the flight path with the observation data based on the association cost and the one-to-one constraint relation between the flight path and the observation data to obtain an association result list, an unassociated flight path list and an unassociated observation data list;

step 106: respectively updating the unassociated flight tracks in the unassociated flight track list and the flight tracks in the association result list to obtain an unassociated updated flight track list at the current moment and an associated flight track list at the current moment;

step 108: performing space-time deviation feedback type fusion processing on the associated track list to obtain a fused associated track list;

step 110: merging the unassociated updated track list and the fused associated track list, and managing the tracks in the merged list to obtain a survival track list at the current moment;

step 112: and carrying out deflection-free conversion on each observation data in the unassociated observation data list to construct the dimension expansion state estimation and the dimension expansion state estimation covariance corresponding to the new track to obtain a new track list, and merging the new track list and the survival track list to obtain the track list at the current moment.

In the embodiment of the invention, firstly, a dimension expansion state estimation sampling point set and a weight set corresponding to each track are determined based on the dimension expansion state estimation of each track in a track list at the previous moment, wherein the dimension expansion state estimation comprises space-time deviation estimation and target basic state estimation; then, combining an observation data list reported by a sensor at the current moment, calculating the association cost of each pair of track-observation data combination, associating the track with the observation data according to the association cost to generate an association result list, an unassociated track list and an unassociated observation data list, updating the tracks in the association result list and the unassociated track list, then, performing space-time deviation feedback type fusion processing on the updated associated track list to further improve the space-time deviation compensation effect, and finally, generating a new track by managing the track and using each observation data in the unassociated observation data list to obtain the track list at the current moment. According to the scheme, the space-time deviation is expanded to the basic state vector of the target, a state space model under the space-time deviation condition is established, the joint data association and deviation calibration problem can be converted into the classic data association and filtering problem under the unified Bayesian framework based on the model, iterative optimization processing between deviation calibration and data association is not needed, the compensation performance of the space-time deviation is further improved through feedback type fusion processing, and effective multi-sensor multi-target tracking is further achieved under the condition that the space-time deviation exists.

The manner in which the various steps shown in fig. 1 are performed is described below.

With respect to step 100:

it should be noted that, in the scheme, the step 100-.

For example, the flight path list based on the k-1 th time

Calculating a set delta of extended dimension state estimation sampling points corresponding to each flight path p by adopting a traceless transformation method _p (k-1| k-1) and corresponding set of weights W _p The concrete formula is as follows:

where p is the track number and n _t (k-1) is the number of tracks at time k-1,

and

is the ith expanded dimension state estimation sampling point and its corresponding weight, where i is 0, …,2m _x ；m _x Is the dimension of the expanded state estimate vector, and κ is the dimension used to determine the expanded state estimate at time k-1

The scale parameter of the distribution state of the surrounding sampling points and satisfies (m) _x +κ)≠0；

Is composed of

The ith row or the ith column of (a),

and P _p (k-1| k-1) is the extended dimensional state vector X of the flight path p at the k-1 th time, respectively _p (k-1) corresponding dimension expansion state estimates and dimension expansion state estimate covariance.

In the embodiment of the invention, a plurality of sensors are used for tracking a plurality of targets in a region, each sensor provides distance and angle observation data of the plurality of targets in a polar coordinate system, however, the observation data of each sensor has fixed distance deviation and angle deviation and is influenced by time required by asynchronous clocks, data transmission and/or signal processing, and fixed unknown time delay exists between a time stamp of the observation data and a real observation moment, so that a difference value of a real observation interval and a time stamp interval of the observation data from different sensors is used as a time deviation, and the time deviation and the space deviation are simultaneously expanded to a basic state vector of the targets to be constructed into an expanded state vector.

Specifically, the dimension-extended state vector X of the flight path p at the kth time _p (k) The concrete form of (A) is as follows:

wherein the content of the first and second substances,

in the formula, X _p (k) Is the basic state vector, x, of the flight path p _p (k) And y _p (k) Respectively the position of the flight path p in the x and y directions,

and

respectively the speed of the flight path p in the x and y directions,

is the space-time offset vector of S sensors, B (k) is the spatial offset vector of S sensors, b _s (k)＝[Δr _s (k),Δθ _s (k)]Is the spatial deviation of the sensor S1, wherein ar _s (k) And Δ θ _s (k) Is the distance and angle deviations of sensor S, psi (k) is the time deviation vector of S-1 sensors with respect to reference sensor l, where Δ t _s,l (k) Is the time offset of the sensor S relative to the sensor l, where S ≠ 1.

It should be noted that, in the embodiment of the present invention, the unscented transformation method is adopted to obtain the extended-dimension state estimation sampling point set and the weight set corresponding to each flight path, and other methods may also be adopted, so that the method is not particularly limited.

With respect to step 102:

it should be noted that, in the embodiment of the present invention, a plurality of sensors are used to track a plurality of targets in an area, and sampling periods of the sensors are different, so that an observation data list obtained by measuring the plurality of targets by each sensor is a list reported by one of the sensors at intervals, and each observation data list reported by each sensor corresponds to a time.

In some embodiments, step 102 may include steps S1-S4 as follows:

step S1, for each pair of track-observation data combinations, performing: determining an observation function of the sensor according to the flight path and the observation data in the pair of flight path-observation data combinations and the sensor corresponding to the measurement observation data list;

step S2, according to the observation function, the extended dimension state estimation sampling point set, the weight set and the observation data in the pair of track-observation data combination, the observation prediction error and the observation prediction covariance of the pair of track-observation data combination are determined;

step S3, calculating the Mahalanobis distance of the pair of track-observation data combinations according to the observation prediction error and the observation prediction covariance of the pair of track-observation data combinations;

and step S4, calculating the associated cost of the pair of track-observation data combination according to the Mahalanobis distance.

In step S1, in order to align the observation data with the basic state of the target by using the time offset, it is necessary to express the observation data as a function of the basic state and the space-time offset, that is, an observation function.

The observation function is determined according to the following formula:

where p is the track number, s is the sensor number, X _p Is the extended dimension state estimate of the track, k being the time, (x) _s,p (k),y _s,p (k) Is the true position of the track corresponding to the time of observation data, (x) _s ,y _s ) For the position of sensor s, Δ r _s (k) And Δ θ _s (k) Is the distance deviation and angle deviation, x, of the sensor s _p (k) And y _p (k) Respectively the position of the flight path p in the x and y directions,

and

speed of the flight path p in the x and y directions, Δ t, respectively _s,l (k) Is the time offset of the sensor S relative to the reference sensor l, where S ≠ 1.

In step S2, the observation prediction error and the observation prediction covariance of the track-observation data combination are determined by the following equations:

wherein the content of the first and second substances,

in the formula (I), the compound is shown in the specification,

is a track-observation data combination (p, i) _s ) The error of the observed prediction of (a),

is a track-observation data combination (p, i) _s ) The observation data list may be expressed as

i _s Is the observed data number of sensor s, n _s (k) Is the amount of observed data that sensor s reports at the current time,

is the ith report from sensor s _s The number of the observed data is the same as the number of the observed data,

is an observed prediction of the flight path p,

is the ith observation prediction sampling point of the flight path p

And observation prediction

The difference value of (a) to (b),

is the observed noise covariance, σ, of the sensor s _r And σ _θ Respectively, distance and angle observation noise standard deviations;

is the ith extended-dimension-state prediction sampling point, h, of the flight path p _s Of sensors sObservation function, F (k-1) is the extended dimension state transition matrix, F (k-1) is the transition matrix corresponding to the base state, 0 _4,(3S-1) Is a zero matrix with dimension 4 · (3S-1), 0 _(3S-1),4 Is a zero matrix with dimension (3S-1). 4, I _3S-1 Is an identity matrix with dimension 3. S-1; Δ T (k-1) is a time stamp of observation data provided by sensor s at time k

And time stamp of observation data provided by sensor l at time k-1

The difference between them.

The pair of track-observation data combinations (p, i) in equations (7) to (8) can be expressed by equations (9) to (14) _s ) Is calculated from the observed prediction error and the observed prediction covariance.

In step S3, the mahalanobis distance of the pair of track-observation data combinations is calculated by the following formula:

in the formula (I), the compound is shown in the specification,

is a track-observation data combination (p, i) _s ) The distance between the two adjacent channels of the channel,

is a track-observation data combination (p, i) _s ) The observation of (2) predicts covariance.

In step S4, the associated cost is determined according to the following formula:

in the formula (I), the compound is shown in the specification,

is the associated cost, i, of the pair of track-observation data combinations _s Numbering the observations of sensor s, u (i) _s ) Is a binary variable when i _s When 0, u (i) _s ) When i is equal to 0 _s When not equal to 0, u (i) _s )＝1，

is the mahalanobis distance of the pair of track-observation data combinations,

With respect to step 104:

in some embodiments, step 104 may include the steps of:

In this step, the two-dimensional distribution model of the track-observation data association is established as follows:

wherein, the first and the second end of the pipe are connected with each other,

is a binary variable when (p, i) _s ) In the correlation result of the track-observation data,

otherwise

Association result list of track-observation data obtained by adopting generalized auction algorithm

Unassociated track list

And list of unassociated observed data

Can be represented as:

in the formula (I), the compound is shown in the specification,

is the number of track-observation combinations in the correlation result list,

is the serial number of the track-observation data combination in the association result list, (m (a), n (a)) shows that the serial numbers of the track and the observation data in the a-th association combination are m (a) and n (a), respectively, and m (-) isA slave

To {1,.. n _t (k-1) }, n (-) is a slave

To {1,.. n _s (k) The one-way mapping of indicates that the track in the a-th association combination may be a track list T _k-1 Of the first associative combination may be the observation list z _s (k) While the track-observation data in each associated combination is numbered differently than in the other combinations.

In addition, unassociated track lists

Is a total track list T _k-1 With difference sets of correlated tracks, analogous to lists of unassociated observed data

Is the total observed data list z _s (k) And a difference set of correlated observed data.

For step 106:

in some embodiments, step 106 may include steps B1-B2 as follows:

step B1, calculating the dimension-expanding state prediction covariance and the dimension-expanding state prediction covariance of each track in the unassociated track list, and taking the dimension-expanding state prediction covariance and the dimension-expanding state prediction covariance of each track as the dimension-expanding state estimation covariance and the dimension-expanding state estimation covariance of each track at the current moment to obtain an unassociated updated track list at the current moment; the unassociated updated track list comprises the dimension expansion state estimation and the dimension expansion state estimation covariance of each track at the current moment;

step B2, filtering each observation data in the association result list by adopting an unscented Kalman filtering method to obtain the dimension expansion state estimation and the dimension expansion state estimation covariance of each flight path associated with each observation data at the current moment so as to obtain an associated flight path list at the current moment; the associated track list includes the extended dimension state estimate and the extended dimension state estimate covariance for the current time for each track.

In step B1, the unassociated track list may be calculated according to the following formula

The dimension-extended state prediction and the dimension-extended state prediction covariance of each track:

P _p (k|k-1)＝F(k-1)·P _p (k-1|k-1)·F(k-1)′+Q _p (k-1) (22)

wherein the content of the first and second substances,

Q _p (k-1)＝Γ(k-1)Q _p (k-1)Γ(k-1)′ (23)

in the formula (I), the compound is shown in the specification,

is a dimension-extended state prediction of the flight path P, P _p (k | k-1) is the extended-dimension state prediction covariance of the flight path p,

Q _p (k-1) is the dimension-extended process noise covariance of track p, Γ (k-1) is the dimension-extended process noise gain matrix,

is the process noise covariance, v, of the underlying state corresponding to the target _x And v _y Is the process noise standard deviation in the x and y directions, respectively, and Γ (k-1) is the process noise gain matrix corresponding to the fundamental state of the track, 0 _(3S-1),2 Is a zero matrix with dimension (3S-1) · 2, and F (k-1) is an extended dimension state transition matrix.

Using the dimension-expanding state prediction and the dimension-expanding state prediction covariance of each track as the dimension-expanding state estimation and the dimension-expanding state estimation covariance of each track at the current moment, namely, commanding

P _p (k|k)＝P _p (k | k-1) wherein,

and P _p (k | k) is respectively the dimension expansion state estimation and the dimension expansion state estimation covariance of the current time track p, so as to obtain an unassociated updated track list at the current time, and the specific expression is as follows:

in step B2, the unscented Kalman filtering method is used to list the correlation results

The observation data associated with each track in the navigation system is filtered to obtain the dimension expansion state estimation and the dimension expansion state estimation covariance of each track associated with each observation data at the current moment, and the specific formula is as follows:

in the formula (I), the compound is shown in the specification,

and

respectively, the observation prediction covariance and the observation prediction error of the track-observation data association combination;

and P _p (k | k-1) are the extended dimension state prediction and extended dimension state prediction covariance, respectively, for track p,

is the cross covariance between the dimension-expanding state and the observed data, which are respectively expressed as:

P _p (k|k-1)＝F(k-1)·P _p (k-1|k-1)·F(k-1)′+Q _p (k-1) (29)

in the formula, Q _p (k-1) is the extended dimensional process noise covariance of track p, m _x Is the dimension of the expanded-dimension state vector,

is the weight corresponding to the ith dimension-expanding state estimation sampling point, F (k-1) is the dimension-expanding state transition matrix,

is the ith observation prediction sampling point of the flight path p

And observation prediction

The specific expression is given in equation (10) of step 102,

is the ith dimension expansion state prediction sampling point and the dimension expansion state prediction of the flight path p

The specific formula of the difference value of (c) is as follows:

according to equations (28) - (31), the dimension-extended state estimation and the dimension-extended state estimation covariance of each track in the association result list at the current time can be calculated through equations (26) and (27), respectively, and an associated track list at the current time can be obtained, wherein the specific expression of the associated track list is as follows:

for step 108:

in some embodiments, step 108 may include the steps of:

obtaining the fused track; the fused track comprises fused dimension expansion state estimation and fused dimension expansion state estimation covariance;

and replacing the corresponding track in the associated track list by each fused track with the updating times larger than the first set value to obtain a fused associated track list.

In this step, first, the associated track list obtained in the judgment step 106

Whether the updating times of each flight path p are more than 4 or not is judged, and the space-time deviation estimation covariance of all flight paths with the updating times of more than 4 at the current moment are extracted.

The relation between the extended dimension state estimation and the space-time deviation estimation of the flight path is as follows:

and

respectively a basic state estimate and a space-time offset estimate of the flight path p,

is the basic state estimate covariance of the flight path p,

is the cross-covariance of the basic state estimate and the space-time offset estimate of the flight path p,

is the track p space-time offset estimate covariance.

Extracting space-time deviation estimation of all tracks with updating times larger than 4 at current moment

Sum space-time offset estimation covariance

And performing fusion, namely calculating the space-time deviation estimation after each flight track is fused and the covariance of the space-time deviation estimation after each flight track is fused according to the following formula:

wherein the content of the first and second substances,

in the formula (I), the compound is shown in the specification,

and P _b,f (k | k) is the covariance of the track-fused space-time offset estimate and the fused space-time offset estimate, w _p Is the weight of the flight path p space-time offset estimate, I _p Is the inverse matrix of the space-time offset estimate covariance for track p, and I is the sum of the inverse matrices of the space-time offset estimate covariance for all tracks in the list.

Then, replacing the original space-time deviation estimation in the dimension-extended state estimation at the current time with the fused space-time deviation estimation of each track with the updating times larger than 4 to obtain the dimension-extended state estimation of each track after fusion, wherein the specific expression is as follows:

then, calculating the covariance between the space-time deviation estimation after each track fusion and the corresponding basic state estimation by adopting a non-track transformation method to obtain the covariance of the dimension-extended state estimation after each track fusion, and specifically calculating according to the following formula:

and

respectively, the basic state estimate corresponding to the flight path p

And basic state estimation covariance

Estimates the sample points and weights for the basis states,

is a dimension of the basic state estimation and,

and

is corresponding to the fused space-time offset estimation

And the fused space-time offset estimation covariance P _b,f (k | k) fused space-time offset estimation sampling points and weights, m _b Is the dimensionality of the space-time offset estimate;

and

and the calculation method in step 100

And

the calculation method is similar, and the description is omitted here; in a similar manner to that described above,

and

the calculation method is not described in detail.

Then, the fused track with each updating time larger than 4 can be obtained, and the expressions (36) and (38) are respectively the fused dimension-expanding state estimation of each track and the expression of the fused dimension-expanding state estimation covariance.

Finally, replacing the corresponding track in the associated track list obtained in the step 106 by each fused track with the updating times larger than 4 to obtain a fused associated track list, wherein the expression of the fused associated track list is as follows:

with respect to step 110:

updating the track obtained in step 106 and not related to the current timeListing

And the associated track list obtained in step 108

Merging, wherein the expressions of the merged list are as follows:

in some embodiments, managing the tracks in the merged list to obtain the surviving track list at the current time may include the following steps:

determining the updating times of each track in the merged list, and if the updating times are equal to a second set value and the updating times which are not associated with the observation data are greater than the specified times, removing the track from the merged list; the designated times are less than a second set value;

if the updating times are larger than a second set value and the latest set times are not related to the observation data, the flight path is withdrawn from the merged list;

and obtaining a survival track list at the current moment.

In the embodiment of the invention, the updating times of each track in the merged list are judged, and if the updating times are not more than 4, the track is a temporary track; for the temporary track with the updating times being exactly equal to 4, adopting 3/4 logic method to judge, namely checking that the track is not distributed with observation data in 4 updating times for more than 1, withdrawing the track from the merged list, and counting the number n of tracks _t (k-1) minus 1; if the track has been assigned observation data at least 3 times out of 4 updates, the temporary track is successfully initiated and converted to a confirmation track.

If the number of updating times is more than 4, the flight path is a confirmed flight path, whether the flight path is allocated with observation data at least once in the last 5 updating times is checked, if so, the flight path is continuously maintained to be the confirmed flight pathThe track is processed at the subsequent time, if not, the track is no longer maintained or leaves the observation area of the sensor, the track is withdrawn from the merged list, and the number n of the tracks is counted _t (k-1) minus 1.

Suppose that after the above processing, the total cancellation is carried out

An individual track; screening the merged list to obtain a survival track list at the current moment

Is of the form:

wherein the content of the first and second substances,

with respect to step 112:

in an embodiment of the present invention, the list of unassociated observed data is based on step 104

Generating new flight paths by adopting a single-point initialization method for each observation data, and establishing a new flight path list

And is compared with the survival track list obtained in step 100

Merging to generate the final complete track list T at the current moment _k The specific process is as follows:

since the observation data are distance and angle observation data of polar coordinates, and the basic state of the flight path is position and speed data of a cartesian coordinate system, each observation data of an unassociated observation data list needs to be subjected to deflection-free conversion to construct a new flight path.

First, for

Each observation in (1)

Firstly, the non-deflection position measurement of the data in a Cartesian coordinate system is calculated by a non-deflection method

And no deflection change position measurement covariance

The concrete formula is as follows:

wherein the content of the first and second substances,

in the formula (I), the compound is shown in the specification,

and

are respectively observation data

The distance and angle observation data in (1),

and

there is no deflection in the x and y directions respectively to change the position observation data,

is an error compensation factor, mu ^u Is the mean of the transition position errors;

is the observed variance of the position in the x-direction,

is the position observation variance in the y-direction,

and

is the cross-covariance of the position observations in the x-direction and the y-direction;

is a compensation factor.

Subsequently, the unbiased transformed position observation data and position observation covariance in equations (39) - (44) are used to establish the extended dimension state estimate and extended dimension state estimate covariance for track p, where the list of unassociated observation data is used

The serial number of the first observation data establishing track is set as

That is, the maximum track number of the surviving track list is added with 1, the number of the subsequent newly generated track is processed according to p +1, and the specific formula of the dimension expansion state estimation and the dimension expansion state estimation covariance of the track p is as follows:

wherein the content of the first and second substances,

is a basic state estimate of the flight path p,

is a space-time offset estimate of the flight path P, P _p (k | k) is the basic state estimate covariance corresponding to the flight path p,

is the space-time offset estimation covariance of the flight path p; the 4 variables are of the form:

wherein v is _max Is the maximum possible speed of the vehicle,

is the covariance of the spatial offset of the individual sensors,

is the variance, Δ r, of the individual sensor time offsets _max ，Δθ _max And Δ t _max The largest possible distance deviation, angle deviation and time deviation, respectively.

From the above, the extended dimension state estimation and extended dimension state estimation covariance of the new track established by using the observation data in the unassociated observation data list can be obtained by constructing 4 variables of equations (47) - (50) from the unbiased converted position observation data and position observation covariance in equations (39) - (44) and substituting the 4 variables into equations (45) and (46).

Using the unassociated observation data list in turn according to the above process

Each observation data of the system is used for creating a new flight path, and a new flight path list is finally obtained

The specific expression is as follows:

finally, the surviving tracks are listed

And list of new flight paths

Merging to generate the final complete track list T at the current moment _k The specific expression is as follows:

is the number of tracks in the track list at the current time.

In the embodiment of the invention, under the condition that the asynchronous multi-sensor has space-time deviation and uncertainty of an observation source, the scheme can simultaneously realize data association, space-time deviation compensation and multi-target state estimation fusion processing; the processing steps are repeated, so that the space-time deviation of the multiple sensors can be compensated at the moment when k is 1, 2.

In order to verify the effectiveness of the multi-sensor combined space-time deviation calibration and multi-target association fusion (JDABCF) method, simulation data is used for carrying out Monte Carlo experiments, the performance of the algorithm is evaluated according to Root Mean Square Errors (RMSEs), and the optimal performance which can be achieved by the method is measured by adopting a posterior Cramer-Rao lower bound (PCRLB). The methods used for comparison were the EM-UKF method, which did not take into account the time deviation, and the JDABCF method without feedback fusion treatment (JDABCF-NFF), which did not perform the feedback fusion treatment. The purpose of comparison with these two methods is to illustrate the necessity of the JDABCF method to handle space-time offsets simultaneously and to perform a feedback fusion step.

In the simulation experiment, 4 asynchronous sensors are adopted to track multiple targets, and the 4 sensors are respectively positioned at the positions of (-5000m,5000m), (8000m, -6000m), (8000m,5000m) and (-5000m, -6000 m). The sampling periods of the 4 sensors are respectively 5s, 2s and 2sAnd 1s, the initial sampling instants being 0s, 4s, 6s and 4s, respectively. The sensor 1 is set as a reference sensor. The observed noise covariance of each sensor is R _s (k)＝diag((20m) ² ,(0.01rad) ² ). The sensors each having a distance deviation deltar _s 50m and an angular deviation Δ θ _s 0.03 rad. The remaining parameters are set as follows.

The detection probabilities of the 4 sensors are the same, and the value is 0.95. The time stamp delays of the 4 sensors are 3s, 1s, 1s and 0.5s, respectively, so that the time offsets of the sensors 2, 3 and 4 with respect to the sensor 1 are Δ t, respectively _2,1 ＝2s，Δt _3,1 2s and Δ t _4,1 2.5 s. The 4 objects appear in the observation area from the moment t-0 s, with initial positions (-3000m,8m/s,1000m, -12m/s), (0m,15m/s,1000m,8m/s), (1000m,12m/s,500m,10m/s) and (5000m, -13m/s, -1500m,8m/s), respectively. The process noise standard deviations of the 4 targets were all 0.001m/s ² . In addition, the clutter is evenly distributed in the observation area, and the number of clutter at each time is determined by a poisson distribution variable with a mean value of 30.

Considering that the estimation performance of the multi-sensor space-time bias is similar under the same filtering framework, the invention gives the average result of the multi-sensor space-time bias estimation as the final space-time bias estimation result. Similarly, the present invention gives the average result of the multi-target state estimation as the final target state estimation result.

As shown in fig. 2-7, are graphs comparing the results of the three methods. Wherein FIG. 2 shows the time offset estimates RMSEs for the three methods; FIG. 3 shows distance deviation estimates RMSEs for three methods; FIG. 4 shows the angular deviation estimates RMSEs for three methods; FIG. 5 shows target position estimates RMSEs for three methods; FIG. 6 shows target speed estimates RMSEs for three methods; fig. 7 shows the correct association probabilities for the three methods.

As can be seen from FIGS. 2-6, the RMSEs results for the EM-UKF method are greater than those for the JDABCF method. This is because the EM-UKF method does not take into account and compensate for the time offset, resulting in estimation performance inferior to the JDABCF method. The JDABCF-NFF method has a certain performance improvement, but is still inferior to the JDABCF method. By contrast, the JDABCF method provided by the invention can effectively compensate the space-time deviation to realize accurate fusion estimation of the target state. Fig. 7 gives the probability of correct association, where the JDABCF method still shows the best effect. These results demonstrate the effectiveness of the JDABCF method in simultaneously compensating for space-time offset and performing a feedback fusion step. It is worth noting that the EM-UKF method performs multiple E-M iterations and smoothing to update the initial state estimation result, which results in faster RMSEs convergence in the initial stage of the method and smoother RMSEs curve.

As shown in fig. 8 and 9, an embodiment of the present invention provides a multi-sensor joint space-time offset calibration and multi-target association fusion apparatus. The device embodiments may be implemented by software, or by hardware, or by a combination of hardware and software. In terms of hardware, as shown in fig. 8, for a hardware architecture diagram of an electronic device in which a multi-sensor joint space-time offset calibration and multi-target association fusion apparatus provided in the embodiment of the present invention is located, in addition to the processor, the memory, the network interface, and the non-volatile memory shown in fig. 8, the electronic device in which the apparatus is located in the embodiment may also generally include other hardware, such as a forwarding chip responsible for processing a packet, and the like. Taking a software implementation as an example, as shown in fig. 9, as a logical device, a CPU of an electronic device in which the device is located reads a corresponding computer program in a non-volatile memory into a memory and runs the computer program.

As shown in fig. 9, the multi-sensor joint space-time offset calibration and multi-target association fusion apparatus provided in this embodiment includes:

a determining unit 901, configured to determine, based on a track list at a previous time, a dimension expansion state estimation sampling point set and a weight set corresponding to each track in the track list; the flight path list comprises dimension expansion state estimation and dimension expansion state estimation covariance of a plurality of flight paths corresponding to a plurality of targets; the dimension expansion state estimation comprises basic state estimation and space-time deviation estimation;

a calculating unit 902, configured to calculate an associated cost of each pair of track-observation data combinations based on the extended dimension state estimation sampling point set and the weight set corresponding to each track at the previous time and the observation data list at the current time; the observation data list comprises a plurality of observation data measured by the sensor on the target or the clutter;

the association unit 903 is configured to associate the flight path with the observation data based on the association cost and a one-to-one constraint relationship between the flight path and the observation data, so as to obtain an association result list, an unassociated flight path list, and an unassociated observation data list;

an updating unit 904, configured to update the tracks in the unassociated track list and the associated result list respectively to obtain an unassociated updated track list at the current time and an associated track list at the current time;

the fusion unit 905 is configured to perform feedback fusion processing of space-time deviation on the associated track list to obtain a fused associated track list;

the management unit 906 is configured to merge the unassociated updated track list and the fused associated track list, and manage the tracks in the merged list to obtain a surviving track list at the current time;

a generating unit 907, configured to perform non-deflection transformation on each observation data in the unassociated observation data list to construct an extended dimension state estimation and an extended dimension state estimation covariance corresponding to a new track, so as to obtain a new track list, and merge the new track list and the surviving track list to obtain a track list at the current time.

In an embodiment of the present invention, the computing unit 902 is configured to perform the following operations:

for each pair of track-observation data combinations, performing:

according to the observation function, the extended dimension state estimation sampling point set, the weight set and the observation data in the pair of track-observation data combinations, the observation prediction error and the observation prediction covariance of the pair of track-observation data combinations are determined;

and calculating the association cost of the pair of track-observation data combination according to the Mahalanobis distance.

In one embodiment of the present invention, in the calculating unit 902, the observation function is determined according to the following formula:

where p is the track number, s is the sensor number, X _p Is the extended dimension state estimate of the track, k being the time, (x) _s,p (k),y _s,p (k) For the true position of the track at the time of the observation, (x) _s ,y _s ) Is the position of the sensor s, Δ r _s (k) And Δ θ _s (k) Is the distance deviation and angle deviation, x, of the sensor s _p (k) And y _p (k) Respectively the position of the flight path p in the x and y directions,

and

speed of the flight path p in the x and y directions, Δ t, respectively _s,l (k) Is the time offset of the sensor S relative to the reference sensor l, where S ═ 1.., S and S ≠ l, S being the number of sensors;

the associated cost is determined according to the following formula:

in the formula (I), the compound is shown in the specification,

is the mahalanobis distance of the pair of track-observation data combinations,

In an embodiment of the present invention, the associating unit 903 is configured to perform the following operations:

In an embodiment of the present invention, the updating unit 904 is configured to perform the following operations:

calculating the dimension expansion state prediction and the dimension expansion state prediction covariance of each track in the unassociated track list, and taking the dimension expansion state prediction and the dimension expansion state prediction covariance of each track as the dimension expansion state estimation and the dimension expansion state estimation covariance of each track at the current moment to obtain an unassociated updated track list at the current moment; the unassociated updated track list comprises the dimension expansion state estimation and the dimension expansion state estimation covariance of each track at the current moment;

filtering each observation data in the association result list by adopting an unscented Kalman filtering method to obtain the dimension expansion state estimation and the dimension expansion state estimation covariance of each flight path associated with each observation data at the current moment so as to obtain an associated flight path list at the current moment; the associated track list includes the extended dimension state estimate and the extended dimension state estimate covariance for the current time for each track.

In an embodiment of the present invention, the fusion unit 905 is configured to perform the following operations:

In an embodiment of the present invention, when the management unit 906 performs management on the tracks in the merged list to obtain the surviving track list at the current time, the management unit is configured to perform the following operations:

and obtaining a survival track list at the current moment.

It can be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on a multi-sensor joint space-time offset calibration and multi-target association fusion device. In other embodiments of the present invention, a multi-sensor joint space-time offset calibration and multi-target association fusion apparatus may include more or fewer components than those shown, or some components may be combined, some components may be separated, or different arrangements of components may be used. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

Because the content of information interaction, execution process, and the like among the modules in the device is based on the same concept as the method embodiment of the present invention, specific content can be referred to the description in the method embodiment of the present invention, and is not described herein again.

The embodiment of the invention also provides electronic equipment which comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the multi-sensor combined space-time deviation calibration and multi-target association fusion method in any embodiment of the invention is realized.

The embodiment of the invention also provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the processor is enabled to execute the multi-sensor joint space-time offset calibration and multi-target association fusion method in any embodiment of the invention.

Specifically, a system or an apparatus equipped with a storage medium on which software program codes that realize the functions of any of the above-described embodiments are stored may be provided, and a computer (or a CPU or MPU) of the system or the apparatus is caused to read out and execute the program codes stored in the storage medium.

In this case, the program code itself read from the storage medium can realize the functions of any of the above-described embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

Examples of the storage medium for supplying the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (e.g., CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD + RW), a magnetic tape, a nonvolatile memory card, and a ROM. Alternatively, the program code may be downloaded from a server computer via a communications network.

Further, it should be clear that the functions of any one of the above-described embodiments may be implemented not only by executing the program code read out by the computer, but also by causing an operating system or the like operating on the computer to perform a part or all of the actual operations based on instructions of the program code.

Further, it is to be understood that the program code read out from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion module connected to the computer, and then causes a CPU or the like mounted on the expansion board or the expansion module to perform part or all of the actual operations based on instructions of the program code, thereby realizing the functions of any of the above-described embodiments.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an …" does not exclude the presence of other similar elements in a process, method, article, or apparatus that comprises the element.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A multi-sensor combined space-time offset calibration and multi-target association fusion method is characterized by comprising the following steps:

calculating the association cost of each pair of track-observation data combination based on the dimension-extended state estimation sampling point set and the weight set corresponding to each track at the previous moment and the observation data list at the current moment; the observation data list comprises a plurality of observation data measured by the sensor on the target or the clutter;

2. The method according to claim 1, wherein the calculating the associated cost for each pair of track-observation data combinations based on the set of extended-dimension state estimation sampling points corresponding to each track at the previous time and the set of weights and the observation data list at the current time comprises:

for each pair of track-observation data combinations, performing:

3. The method of claim 2,

the observation function is determined according to the following formula:

and

the correlation cost is determined according to the following formula:

in the formula (I), the compound is shown in the specification,

is the mahalanobis distance of the pair of track-observation data combinations,

4. The method according to claim 1, wherein associating the track with the observation data based on the associated cost and a one-to-one constrained relationship between the track and the observation data to obtain an associated result list, an unassociated track list and an unassociated observation data list comprises:

and determining an association result list, an unassociated track list and an unassociated observation data list by adopting a generalized auction algorithm.

5. The method according to claim 1, wherein the updating the tracks in the unassociated track list and the associated result list respectively to obtain an unassociated updated track list at the current time and an associated track list at the current time comprises:

6. The method according to claim 1, wherein the performing feedback type fusion processing of space-time deviation on the associated track list to obtain a fused associated track list comprises:

and executing the following steps for each track with the updating times larger than a first set value in the associated track list:

7. The method according to any one of claims 1 to 6, wherein the managing the tracks in the merged list to obtain the surviving track list at the current time comprises:

and obtaining a survival track list at the current moment.

8. A multi-sensor combined space-time offset calibration and multi-target association fusion device is characterized by comprising:

9. An electronic device comprising a memory having stored therein a computer program and a processor that, when executing the computer program, implements the method of any of claims 1-7.

10. A computer-readable storage medium, on which a computer program is stored which, when executed in a computer, causes the computer to carry out the method of any one of claims 1-7.