CN121211368B - Multi-mode target fusion and evaluation method - Google Patents
Multi-mode target fusion and evaluation methodInfo
- Publication number
- CN121211368B CN121211368B CN202511755019.1A CN202511755019A CN121211368B CN 121211368 B CN121211368 B CN 121211368B CN 202511755019 A CN202511755019 A CN 202511755019A CN 121211368 B CN121211368 B CN 121211368B
- Authority
- CN
- China
- Prior art keywords
- threat
- data
- target
- sensor
- uncertainty
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Radar Systems Or Details Thereof (AREA)
Abstract
The invention discloses a multi-mode target fusion and evaluation method which comprises the steps of carrying out alignment pretreatment on multi-source observation data to generate multi-source time alignment observation data and multi-source sensor state data, carrying out single-source target detection and initial threat estimation according to the multi-source time alignment observation data and the multi-source sensor state data, constructing single-source target state data and initial target threat estimation data, carrying out multi-sensor target track association and fusion by applying threat consistency constraint to generate multi-sensor fusion target model data based on the single-source target state data, the initial target threat estimation data and the multi-source sensor state data, carrying out multi-sensor collaborative scheduling decision by applying threat reduction benefit estimation to determine multi-sensor scheduling scheme data based on the multi-sensor fusion target model data and the multi-source sensor state data. The invention realizes the deep coupling of association and scheduling to threat assessment tasks, and improves the association accuracy and the effectiveness of scheduling decisions.
Description
Technical Field
The invention belongs to the field of modern monitoring and command control, and particularly relates to a multi-mode target fusion and evaluation method.
Background
In the field of modern monitoring and command control, how to efficiently integrate the observation data of multi-source heterogeneous sensors such as radar, photoelectricity, unmanned aerial vehicle load and the like in the face of increasingly complex electromagnetic and physical environments has become a key for improving situation awareness. The method is accurate and real-time multi-objective fusion and threat assessment, is a core premise for realizing autonomous decision making, guaranteeing the safety of key assets and optimizing resource allocation, and has important research significance and application value.
Current multi-sensor data fusion techniques focus mainly on improving the accuracy of estimation of target states (e.g., position, velocity). For this purpose, researchers have employed state estimation algorithms including kalman filtering (KALMAN FILTERING), particle filtering (PARTICLE FILTERING), and the like. In track association, widely-used methods include multi-hypothesis tracking (MHT) and Joint Probability Data Association (JPDA), which mainly rely on the kinematic characteristics and appearance characteristics of the target for matching. Meanwhile, in the field of sensor management, the existing scheduling strategies also aim at maximizing target tracking accuracy or coverage.
However, the prior art still has significant shortcomings in achieving deep coupling of high-level semantics (e.g., threats) with underlying data (e.g., states). In particular, there is a technical problem in that the scheduling decisions of the sensors are disjointed from the threat assessment task. The existing scheduling model aims at optimizing the state estimation precision (such as position covariance) of a target, and ignores the final purpose of scheduling, namely, reducing the uncertainty of threat assessment. Such state optima are not threat awareness optima, resulting in sensor resources not being prioritized for resolving the most critical threat uncertainty. In addition, the track association process lacks threat continuity constraints. Conventional association costs only consider motion and appearance, resulting in separation of association (data layer) from threat assessment (semantic layer). When the target track crosses or has ambiguity, track error association is easy to occur, so that unreasonable and violent jump of the target threat score occurs on the time sequence, and the stability and the credibility of the global situation are reduced.
Disclosure of Invention
The invention aims to provide a multi-mode target fusion and evaluation method for solving the problems existing in the prior art.
The technical scheme is that the multi-mode target fusion and evaluation method comprises the following steps:
Acquiring multi-source observation data, performing alignment preprocessing, and generating multi-source time alignment observation data and multi-source sensor state data;
Based on the single-source target state data, the initial target threat assessment data and the multi-source sensor state data, threat consistency constraint is applied to execute multi-sensor target track association and fusion, and multi-sensor fusion target model data is generated;
and based on the multi-sensor fusion target model data and the multi-source sensor state data, performing multi-sensor collaborative scheduling decision by applying threat reduction benefit evaluation, and determining multi-sensor scheduling scheme data.
The method has the beneficial effects that the deep coupling of association and scheduling to threat assessment tasks is realized, and the association accuracy and the effectiveness of scheduling decisions are improved.
Drawings
FIG. 1 is a flowchart illustrating steps of a method for multi-modal object fusion and evaluation according to an embodiment of the present application.
Fig. 2 is a flowchart of steps for determining multi-sensor scheduling scheme data according to an embodiment of the present application.
FIG. 3 is a flowchart illustrating steps for constructing threat reduction benefit assessment data in accordance with an embodiment of the application.
Fig. 4 is a flowchart illustrating steps for deriving prediction state covariance according to an embodiment of the application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in detail with reference to specific embodiments. It is noted that embodiments of the present invention may be implemented based on a variety of computing environments and system architectures. For example, the methods described herein may be performed by a processor in a computer system executing computer-executable instructions stored in a memory. The computer system may include, but is not limited to, a server, workstation, or embedded system containing a processor, memory, communication interfaces, and I/O devices. The communication interface is used to communicate with external multi-source aware devices, including illustratively radar, stationary photovoltaics, drone loads, etc.
As shown in fig. 1, a multi-modal target fusion and evaluation method is provided, which includes the following steps:
And acquiring multi-source observation data and performing alignment preprocessing to generate multi-source time alignment observation data and multi-source sensor state data.
In other words, multi-source observation data including radar raw echo data, a fixed photoelectric raw image, and an unmanned aerial vehicle carried raw image are acquired, alignment and quality evaluation processing is performed on the multi-source observation data, and multi-source time alignment observation data and multi-source sensor state data are generated.
In this embodiment, the multi-source observation data is derived from a plurality of preset sensing devices, and may include radar raw echo data, fixed photoelectric raw image data, and unmanned aerial vehicle carried raw image data, for example. The system reads the original observation information from the devices and synchronously acquires the sensor position posture and the time stamp data corresponding to the observation information. The alignment pre-processing can solve the inconsistency of different sensor data in the time reference and the space coordinate system. Specifically, the preprocessing utilizes preset equipment calibration parameter data to execute time synchronization, coordinate unification and quality evaluation processing on various observation data. Time synchronization allows all data to be mapped to a unified time axis, while coordinate unification (e.g., conversion to a unified geodetic coordinate system) ensures comparability of spatial locations. The output multi-source time-aligned observation data and multi-source sensor state data are the structured unified input basis.
In some alternative embodiments, the coordinate unification is not limited to conversion to a geodetic coordinate system, but may be conversion to a specific coordinate system or a relative coordinate system referenced to a certain sensor. In addition, the quality assessment process may also include preliminary screening of data integrity, signal-to-noise ratio, or image sharpness to reject low quality data.
And based on the multi-source time alignment observation data and the multi-source sensor state data, single-source target detection and initial threat estimation are executed, and single-source target state data and initial target threat estimation data are constructed.
In this embodiment, the system performs independent analysis processing on the multisource time-aligned observation data according to sensor sources (e.g., radar, fixed photo-electric, unmanned aerial vehicle). Specifically, the system combines preset radar target detection model parameter data and photoelectric target identification model parameter data to respectively perform target detection, target classification and preliminary state estimation on radar observation, fixed photoelectric observation and unmanned aerial vehicle observation. The output of this step is constructed as two types of critical data, single source target state data and initial target threat assessment data. The single-source target state data is used for describing physical properties of a target, and exemplarily comprises moving elements such as target position, speed, heading and the like, and identifying elements such as target category, identifying confidence and the like. The initial target threat assessment data calculates a preliminary threat score and a corresponding uncertainty estimate for each detected single source target according to a preset initial threat assessment model. These two types of data will be input as the core for the subsequent multi-sensor track association and threat consistency constraint calculations.
Alternatively, the radar target detection model and the photoelectric target recognition model may be deep learning-based models (such as YOLO, fasterR-CNN) or conventional signal/image processing algorithms. The initial threat assessment model may be a simplified rule model at this stage, for example, giving a rough score based only on the class and speed (e.g., high speed) of the target.
And based on the single-source target state data, the initial target threat assessment data and the multi-source sensor state data, performing multi-sensor target track association and fusion by applying threat consistency constraint, and generating multi-sensor fusion target model data.
Specifically, the system reads single-source target state data and initial target threat assessment data, and candidate matching is carried out on multi-source target observations from radars, fixed photoelectricity and unmanned aerial vehicles under a unified time and space coordinate frame. In addition, threat consistency constraints are introduced when track association is performed. In other words, when the system performs global association optimization processing, not only the consistency of the target position and motion and the similarity of the appearance characteristics of the cross-sensor are comprehensively considered, but also the smoothness of the change of the target threat with time is considered, so that the association result is kept continuous and consistent on both physical and semantic (threat) layers. Through association optimization, the system obtains continuous unified target track data across time and across sensors. And carrying out state fusion and threat fusion on each piece of unified target track data, and outputting multi-sensor fusion target model data. Illustratively, the multi-sensor fusion target model data is a comprehensive model comprising a target state time series, a threat scoring time series, and a threat uncertainty time series. Optionally, comprehensive target threat assessment data may also be generated, reflecting the fusion result at the current time. In some embodiments, the global association optimization process may be implemented using a hungarian algorithm, a qiaoke-Wo Erhe south tetter (Jonker-Volgenant, JVC) algorithm, or a multi-hypothesis tracking (MHT) algorithm, etc., where the cost function includes threat consistency cost terms.
And based on the multi-sensor fusion target model data and the multi-source sensor state data, performing multi-sensor collaborative scheduling decision by applying threat reduction benefit evaluation, and determining multi-sensor scheduling scheme data.
Unlike conventional scheduling, which aims only at improving the accuracy of target state estimation, the present embodiment aims at maximizing the accuracy of threat assessment. Specifically, the system reads multi-sensor fusion target model data and comprehensive target threat assessment data, and evaluates observation tasks which can be executed between each sensor and each target by combining multi-source sensor state data and a preset sensor capacity model. Preferably, the threat reduction benefits are quantified by the system evaluating the expected degree of reduction in target threat uncertainty under a given observation as the benefits of that observation task. Based on the constructed threat reduction benefit evaluation data, a collaborative scheduling optimization model is further constructed with the overall threat uncertainty reduction maximized as an optimization objective. The model is constrained by sensor motion constraints, task capacity constraints, and time window constraints when solving. And the model is solved to obtain the optimal observation object, observation time and observation path of each sensor, executable multi-sensor scheduling scheme data is formed, and closed-loop scheduling capability is constructed. Alternatively, the collaborative scheduling optimization model may not only maximize overall revenue, but also minimize uncertainty of the highest threat objective. In addition, the scheduling decision may be performed at a fixed time period or may be triggered to be performed when a global threat situation changes significantly.
As shown in fig. 2, in one possible embodiment, determining the multi-sensor scheduling scheme data includes:
and constructing target threat uncertainty data, wherein the target threat uncertainty data characterizes uncertainty indexes of the current target threat scores.
In this embodiment, knowing only the threat score (e.g., 0.8) of the target is insufficient, and the system also needs to know the confidence or uncertainty of that score. A target of high threat but at the same time with high uncertainty should be a priority object of interest for sensor scheduling. Optionally, the fusion state estimation and covariance information of the multi-sensor fusion target model data are utilized, and the uncertainty index of each target threat score is calculated through a mathematical model by combining the current threat score in the comprehensive target threat assessment data. The indicator may be quantized to a scalar value, such as the current threat score variance Var Threat_k. The final generated target threat uncertainty data (e.g., a list containing [ target number, threat score variance ]) will be used to evaluate observed revenue.
And combining target threat uncertainty data, multi-sensor fusion target model data and multi-source sensor state data, establishing a sensor observation effect model, predicting target threat uncertainty under the assumed observation condition, and forming observed target threat uncertainty prediction data.
Illustratively, this embodiment may be used to evaluate how much threat uncertainty for a particular target (e.g., target k) will drop after observation is complete if the system assigns a certain sensor (e.g., drone a) to observe that target k. For this purpose, the system needs to evaluate each sensor-target combination. Specifically, based on the multisensor state data (e.g., current position, speed of drone a) and multisensor fusion target model data (e.g., current position, speed of target k), the accessibility of the sensor to conduct effective observations of the target during the current decision period is analyzed. If so, deducing the expected updating effect of the observation action on the covariance of the target state according to preset sensor capability model parameter data (for example, the measurement accuracy of the unmanned plane A at a specific distance and angle). And substituting the expected and better state covariance into the threat uncertainty calculation model again to obtain a predicted value, namely the observed threat score variance Var Threat_k_after(s). And summarizing the predicted values to form target threat uncertainty predicted data after observation.
Optionally, the sensor observation effect model may consider not only measurement accuracy but also the information dimension of observation. For example, radar observations may reduce primarily the position/velocity uncertainty of the target, while photoelectric observations may reduce primarily the class uncertainty of the target, both contributing differently to the reduction of the final threat uncertainty. The contributions of these different dimensions may be uniformly measured by the gradient of the threat assessment model.
Comparing the target threat uncertainty data with the observed target threat uncertainty prediction data, calculating the threat uncertainty reduction amount, and constructing threat reduction benefit evaluation data.
Specifically, the system extracts, for each sensor-target combination, its pre-observation current threat score variance Var Threat_k (from the target threat uncertainty data) and the post-observation observed threat score variance Var Threat_k_after(s) (from the post-observation target threat uncertainty prediction data). The difference Var Threat_k-VarThreat_k_after(s) is the threat uncertainty reduction delta U (s, k) of the observation task (s, k) in the threat uncertainty dimension. The reduction Δu (s, k) is defined as threat reduction benefit. All combined benefit values are calculated and packaged to form threat reduction benefit assessment data, which data (e.g., a benefit matrix) is directly input as an objective function of the subsequent schedule optimization.
And constructing a collaborative scheduling optimization model aiming at maximizing the overall threat uncertainty reduction according to the threat reduction income evaluation data, and solving and generating multi-sensor scheduling scheme data.
It can also be said that the multi-sensor scheduling scheme data is generated by constructing a collaborative scheduling optimization model targeting the maximization of the overall threat uncertainty reduction according to threat reduction yield evaluation data and combining multi-source sensor state data and multi-sensor fusion target model data.
In this embodiment, the system weights threat reduction benefit assessment data (i.e., benefit matrix ΔU) as an objective function of the optimization model. The goal of the optimization is to select a set of sensor-target assignment relationships (represented by decision variables x (s, k)) to maximize the overall threat uncertainty reduction (i.e., maxΣΔu (s, k) ×x (s, k)). While maximizing revenue, the optimization model also needs to satisfy a series of realistic constraints derived from multi-source sensor state data (e.g., the task capacity of the sensor s) and scheduling constraint parameter data (e.g., the maximum number of observations that can be accepted by the target k in one cycle). By solving the optimization model (e.g., an integer programming solver or an approximate solution such as a greedy algorithm, a hungary algorithm, etc.), the system obtains an optimal task assignment result. Based on the assignment result, a specific executable observation path and a specific executable opportunity are planned by combining the sensor maneuverability in the multi-source sensor state data and the target position prediction in the multi-sensor fusion target model data to form final multi-sensor scheduling scheme data for being issued to a sensor control system for execution.
In a preferred implementation of the present embodiment, quantification of threat uncertainty is achieved by local linear approximation and covariance propagation. It is assumed that the threat assessment model (used to calculate threat scores Threat) is a function of the target state vector X (including location, speed, class probability, etc.), i.e., threat =g (X). Around the current fusion state X k, the function may be approximated as a linear function by taylor expansion. The gradient vector of the threat assessment model g (X) to the state vector X is calculated and noted as v g k. G k (which has the same dimension as the state vector X) describes the sensitivity of the threat score to changes in the various state components (e.g., speed, distance) and can be derived from preset threat assessment model parameter data.
As shown in fig. 3, in one exemplary embodiment, constructing threat reduction benefit assessment data includes:
And extracting the fusion state covariance in the multi-sensor fusion target model data, and combining the prestored threat assessment model gradient vector to calculate and obtain the current threat scoring variance contained in the target threat uncertainty data.
Specifically, a fusion state covariance matrix Σ k of the target k is extracted from the multi-sensor fusion target model data. The matrix Σ k describes the uncertainty of the current estimation of the target state X k. Using covariance propagation law, the current threat score variance Var Threat_k (i.e., the core indicator of the target threat uncertainty data) is calculated as Var Threat_k≈▽gk T*Σk*▽gk. Wherein T denotes the vector/matrix transpose. It can be seen that this formula combines the uncertainty of the state estimate (Σ k) with the sensitivity of the threat model to state (.v. g k), quantifying the uncertainty of the threat score.
The predicted state covariance under the hypothetical observations is derived.
In this embodiment, on the basis of the current threat score variance, prediction is performed, and the prediction state covariance Σ state_after (s, k) is derived. In a preferred implementation, as shown in FIG. 4, deriving the predicted state covariance includes analyzing the observability of the sensor and the target based on the multi-source sensor state data and the preset sensor capability model parameter data, predicting the measurement accuracy under the assumption of the observation, constructing an observed measurement error covariance matrix, and applying a state covariance update formula in combination with the fused state covariance and the observed measurement error covariance matrix to derive the predicted state covariance. Specifically, based on the multi-source sensor state data and the sensor capability model parameter data, the sensor s's observed reachability and observed geometry of the target k are analyzed. From the sensor capability model, an observed measurement error covariance matrix R meas (s, k) for the sensor s under the observation geometry is determined. The observed measurement error covariance matrix R meas describes the measurement noise level of the sensor s itself. The system applies a state covariance update formula (e.g., the covariance update portion of the kalman update formula) and combines the fusion state covariance Σ k with the observed measurement error covariance matrix R meas (s, k) to derive the predicted state covariance Σ state_after (s, k). An exemplary updated formula is Σ state_after=(I-Kgain*Hobs)*Σk, where I is the identity matrix, H obs is the observation matrix, and K gain is the kalman gain ,Kgain=Σk*Hobs T*(Hobs*Σk*Hobs T+Rmeas)-1.
And replacing the fusion state covariance by using the prediction state covariance, multiplexing the threat assessment model gradient vector, and calculating to obtain the observed threat scoring variance contained in the observed target threat uncertainty prediction data.
Specifically, after the predicted state covariance Σ state_after (s, k) is obtained, the system multiplexes the same threat assessment model gradient vector v g k, calculates the observed threat score variance Var Threat_k_after(s)≈▽gk T*Σstate_after(s,k)*▽gk.
And calculating the difference value between the current threat score variance and the observed threat score variance as a threat uncertainty reduction amount to generate threat reduction benefit evaluation data.
In this embodiment, the difference between the current threat score variance and the observed threat score variance is calculated. Further, to ensure rationality of the benefit (to avoid negative benefit due to model nonlinearity or computational noise), obtaining the threat uncertainty reduction further includes performing a non-negative truncation process on the difference between the current threat score variance and the observed threat score variance to ensure that the threat uncertainty reduction is non-negative. Specifically, the threat uncertainty reduction amount Δu (s, k) is calculated as Δu (s, k) =max (0, var Threat_k-VarThreat_k_after (s)).
In one specific numerical case, it is assumed that the state vector X of the object k is only two-dimensional [ position; velocity ], its fusion state covariance Σ k = [ [4,0], [0,1] ]. Let g k=[0.1,0.5]T be the threat assessment model gradient vector (indicating that the threat is more sensitive to speed than to location). The current threat score variance is calculated as Var Threat_k=[0.1,0.5]*[[4,0],[0,1]]*[0.1,0.5]T = 0.04+0.25 = 0.29. Assuming that the sensor s can only accurately measure speed, its observed measurement error covariance matrix R meas is small on the speed component, resulting in an updated predicted state covariance Σ state_after (s, k) = [ [4,0], [0,0.2] ] (speed variance decreases from 1 to 0.2). The observed threat score variance was calculated as Var Threat_k_after(s)=[0.1,0.5]*[[4,0],[0,0.2]]*[0.1,0.5]T = 0.04+0.05 = 0.09. The threat uncertainty reduction is calculated as Δu (s, k) =max (0, var Threat_k-VarThreat_k_after (s))=max (0,0.29-0.09) =0.20. The 0.20 is the threat reduction benefit of the sensor s to the target k, and is filled into the threat reduction benefit evaluation data matrix for the subsequent optimization decision.
According to one aspect of the application, solving for generating multi-sensor scheduling scheme data includes:
A task assignment optimization model weighted by a threat uncertainty reduction in threat reduction benefit assessment data is constructed, the task assignment optimization model targeting maximizing the overall threat uncertainty reduction.
In this embodiment, the system takes threat reduction benefit assessment data (whose core is the benefit matrix Δu (s, k)) as a core input. And constructing a task assignment optimization model. The model introduces a decision variable x (s, k), which is a binary variable, e.g., x (s, k) =1 indicates that the sensor s is assigned to observe the target k in the current decision period, and x (s, k) =0 indicates that no sensor s is assigned. The objective function of the optimization model is set to maximize the sum of threat reduction benefits for all selected tasks, which can be expressed in mathematical form as max Σ sΣk (Δu (s, k) ×x (s, k)). Wherein Σ s and Σ k represent summing of all sensors and all targets, respectively.
And applying scheduling constraint parameter data in the task assignment optimization model, and then solving to obtain a multi-sensor task assignment result, wherein the scheduling constraint parameter data at least comprises a sensor task capacity constraint and a target observable frequency constraint.
It can also be said that the scheduling constraint parameter data including at least the sensor task capacity constraint and the target observable times constraint is applied in the task assignment optimization model; and solving a task assignment optimization model after the scheduling constraint parameter data is applied to obtain a multi-sensor task assignment result.
In particular, task assignment optimization models are subject to a series of constraints in order for the optimization results to conform to physical and tactical reality. Optionally, the constraints are derived from preset scheduling constraint parameter data or multi-source sensor state data. The first type of constraint is a sensor task capacity constraint, e.g., for any sensor s, the total number of tasks it performs in the same cycle cannot exceed its maximum task capacity Cap sensor(s), i.e., Σ k(x(s,k))≤Capsensor(s). Illustratively, the drone sensor (sensor s=1) may be Cap sensor (1) =2 (i.e. track at most two targets simultaneously) due to its onboard processing capability or communication bandwidth limitations. The second type of constraint is a target observable times constraint, e.g., to avoid redundant observations and resource wastage of multiple sensors on the same target, the total number of times target k is observed in the same period should not exceed its maximum observable times Cap target (k), i.e., Σ s(x(s,k))≤Captarget (k). In most cases Cap target (k) =1 is the preferred setting, meaning that one target is observed by at most one sensor. Furthermore, the task assignment optimization model may also include binary constraints with decision variables x (s, k) of 0 or 1. In some alternative embodiments, more complex constraints may also be imposed. For example, sensor motion constraints (such that the sensor has the ability to reach an observation location within a decision period), energy consumption constraints (such that the total energy consumption of the sensor does not exceed a threshold), or mutual exclusion constraints (e.g., a radar cannot perform a tracking task at the same time when it performs a search task).
After the construction is completed, the system solves the task assignment optimization model. The model is essentially a generalized allocation problem or an integer programming problem. The method of solving the model is various. For example, where constraints and objective functions are relatively simple, standard integer programming solvers (e.g., CPLEX, gurobi) or branch-and-bound methods may be employed for accurate solutions. In some specific simplified cases (e.g., all capacity constraints are 1), the problem can be degenerated to a maximum weight matching problem and solved efficiently using the hungarian algorithm or KM algorithm. In the case of very large problem sizes and high real-time requirements, a greedy algorithm (e.g., repeatedly selecting the (s, k) combination with the highest current benefit and updating the constraint) or a meta-heuristic algorithm (e.g., genetic algorithm, simulated annealing) may also be used to obtain the near-optimal solution. The final output of the solution is a determined decision variable matrix x (s, k), i.e. a multi-sensor task assignment result that specifies which sensor should observe which target.
And planning a sensor observation path and a time based on the multi-sensor task assignment result, the multi-source sensor state data and the multi-sensor fusion target model data, and generating multi-sensor scheduling scheme data.
In this embodiment, the abstract task assignment is translated into a concrete execution action. The system performs path and opportunity planning for multi-sensor task assignment results (e.g., x (1, 3) =1, indicating that assigned sensor 1 observes target 3), in combination with multi-source sensor state data (e.g., current position, speed, maneuver performance of sensor 1) and multi-sensor fusion target model data (e.g., current position and predicted trajectory of target 3). For a fixed sensor (e.g. a fixed light station) the planning may only involve generating control commands directed to specific azimuth and pitch angles and setting the start time of the observation. For mobile sensors (e.g., unmanned aerial vehicles), the planning is more complex, involving computing the optimal or feasible path from the current position of the sensor to the target observable area (while meeting both the observation distance and angle requirements), while avoiding obstacle regions, and meeting fuel or voyage constraints. If one maneuver sensor is assigned to observe multiple targets, the plan also needs to optimize target access order (e.g., solve local traveler problems). After the planning is completed, the control instructions, the flight path (if applicable), the observation time, the observation target and the expected observation mode of all the sensors are packaged to form final multi-sensor scheduling scheme data which can be issued and executed.
In one exemplary embodiment, it is assumed that there are 2 sensors (s=1, 2) and 3 targets (k=1, 2, 3). Threat reduction benefit assessment data ΔU matrix is [ [0.5,0.1,0.8], [0.3,0.6,0.2] ]. Let the scheduling constraint parameter data be set to be a task capacity Cap sensor (1) =1 for sensor 1 and a task capacity Cap sensor (2) =1 for sensor 2. T observable times Cap target (k) =1 (for k=1, 2, 3) for each target. The objective function of the optimization model is max (0.5 x (1, 1) +0.1x (1, 2) +0.8x (1, 3) +0.3x (2, 1) +0.6x (2, 2) +0.2x (2, 3)). The constraint conditions comprise that x (1, 1) +x (1, 2) +x (1, 3) is less than or equal to 1 (sensor 1 capacity), x (2, 1) +x (2, 2) +x (2, 3) is less than or equal to 1 (sensor 2 capacity), x (1, 1) +x (2, 1) is less than or equal to 1 (target 1 capacity), x (1, 2) +x (2, 2) is less than or equal to 1 (target 2 capacity), x (1, 3) +x (2, 3) is less than or equal to 1 (target 3 capacity), and x (s, k) is E {0,1}. By solving the optimization model (the optimal solution can be found by observation), the obtained optimal multi-sensor task assignment results are x (1, 3) =1 and x (2, 2) =1, with the remaining x (s, k) =0. Based on the assignment (sensor 1 observing object 3, sensor 2 observing object 2), the system plans a specific sensor path (if sensor 1 is an unmanned aerial vehicle) or heading (if sensor 2 is an optoelectronic) and timing, generating multi-sensor scheduling scheme data.
In one embodiment of the application, generating multi-sensor fusion target model data includes:
And constructing multi-source target matching candidate data based on the single-source target state data and the multi-source sensor state data, and calculating state consistency cost data and appearance consistency cost data.
Specifically, the system obtains the predicted state (such as predicted position, predicted speed) of all the stored tracks from the historical multi-sensor fusion target model data. At the same time, single source target state data (i.e., new observations) for the current time slice is acquired. A threshold screening process is performed, for example, based on a spatial proximity principle, each predicted track is compared with all new observations, and combinations of predicted locations that are too far from the observed locations (e.g., distances greater than a preset spatial threshold) are eliminated, thereby constructing multi-source target match candidate data. For each pair (track i, observation j) combination in the candidate data, the system calculates a conventional association cost. Including state consistency cost data D state (i, j) reflecting the degree of deviation of the target spatial motion continuity, which may be obtained, for example, by calculating the mahalanobis distance between the predicted state (including position, velocity) of track i and the state of observation j, or reduced to a normalized euclidean distance, such as D state=||ppred_i-pobs_j||/Dnorm, where p pred_i and p obs_j are the predicted position and the observed position, respectively, and D norm is a normalization factor. Also included is appearance consistency cost data D app (i, j) reflecting the degree of consistency of appearance features of the same object under different sensor (e.g., photoelectric, unmanned) observations, which may be calculated, for example, based on cosine similarity (e.g., D app=1-cos(ftrack_i,fobs_j) or L2 distance between historical appearance features associated with track i (e.g., aggregate feature vector f track_i) and appearance features of observation j (f obs_j).
Threat consistency cost data is calculated based on multi-source target match candidate data, single-source target state data, initial target threat assessment data, and historical multi-sensor fusion target model data.
In particular, traditional association methods rely solely on state consistency and appearance consistency cost data, and studies find that the correct target is associated, and its semantic attributes (such as threat level) should also evolve smoothly over time. A plausible match in both space and appearance is suspicious and needs to be penalized if the threat score that resulted in the track jumps from a low threat instant to a high threat. Thus, the system calculates threat consistency cost data D threat (i, j) for each candidate matching pair (track i, observation j). Preferably, the system extracts the historical threat score Threat prev (i) for track i at the last time from the historical multi-sensor fusion target model data, uses the information about observation j (e.g., category, speed, etc.) in the single source target state data and the initial target threat assessment data, assumes that the match holds, and re-evaluates the updated threat score Threat new (i, j) for track i at the current time. Threat consistency cost data D threat (i, j) is ultimately quantized as a function of the magnitude of the change between the historical threat scores and the updated threat scores, to penalize candidate matches that lead to severe, non-smooth transitions in threat scores.
And carrying out weighted fusion on the state consistency cost data, the appearance consistency cost data and the threat consistency cost data according to the cost weight in the preset threat consistency constraint parameter data, and constructing comprehensive association cost.
In this embodiment, after three independent costs (state, appearance, threat) are calculated separately, the system needs to fuse them into a single comprehensive associated cost C total (i, j) for subsequent global optimization. Specifically, three cost weights, namely a state consistency cost weight w state, an appearance consistency cost weight w app and a threat consistency cost weight w threat, are read from preset threat consistency constraint parameter data. All three weights are non-negative and their sum may (but need not) be normalized to 1. They reflect the degree of emphasis on different consistency dimensions. For example, w app can be reduced appropriately while w state and w threat are increased in environments where the optoelectronic appearance features are susceptible to clouding, shadowing, and the like. The composite associated cost C total (i, j) is calculated as a weighted sum of the three costs C total(i,j)=wstate*Dstate(i,j)+wapp*Dapp(i,j)+wthreat*Dthreat (i, j).
And executing global track association optimization based on the comprehensive association cost to generate unified target track data.
Specifically, the system takes a comprehensive association cost C total matrix (the rows and columns of which are tracks and observations respectively) as the input of a global optimization algorithm. The goal of this optimization is to find track-to-observe matching schemes such that the sum of the combined correlation costs for all matching pairs is minimized. Illustratively, this problem can be modeled as a minimum weight perfect match problem for bipartite graphs. Global optimization algorithms include the hungarian algorithm, the JVC (Jonker-Volgenant) algorithm, or the auction algorithm. The optimization process also requires handling of unmatched observations (typically used to initialize new tracks) and unmatched tracks (increasing their unmatched count, marking the tracks as terminated if the number of consecutive unmatched times exceeds a preset termination threshold). The output of the optimization is an explicit correlation result, which is packaged into unified target track data, and comprises a unique number, a time index, a sensor source and a correlation observation list optimized by threat consistency constraint of each target track.
And performing state fusion and threat fusion on the unified target track data to generate multi-sensor fusion target model data.
In this embodiment, after determining the association relationship (i.e., unified target track data), the system performs information fusion on the multi-source observation sequences associated in each track. The fusion comprises two layers, namely state fusion, wherein a system extracts corresponding single-source state estimation and sensor observation error characteristics from single-source target state data, a multi-sensor state fusion method (such as an extended Kalman filter EKF, an unscented Kalman filter UKF or a particle filter) is adopted to carry out fusion calculation on physical states of positions, speeds, heading and the like to obtain fusion target state time sequence data and covariance thereof, and threat fusion is carried out, wherein the system is based on a plurality of initial threat scores and threat uncertainties associated with the flight path and can calculate fusion threat scores and fusion threat uncertainties which take the multi-sensor information into consideration by combining the fused physical states (such as more accurate speed estimation), so as to form fusion threat time sequence data. And merging and packaging the fusion target state time sequence data and the fusion threat time sequence data to obtain multi-sensor fusion target model data. And meanwhile, extracting the fusion results of all the on-orbit tracks at the current moment to generate comprehensive target threat assessment data.
In one possible implementation, constructing initial target threat assessment data includes:
The method comprises the steps of carrying out target detection, classification and preliminary state estimation on single-source observation based on multi-source time alignment observation data and multi-source sensor state data, calculating a preliminary threat score and a corresponding uncertainty estimation for each single-source object according to a preset initial threat estimation model, packaging the preliminary threat score and the corresponding uncertainty estimation to generate initial target threat estimation data, wherein the corresponding uncertainty estimation is used as an observation threat uncertainty in the step of constructing an uncertainty correction factor.
Specifically, to achieve uncertainty correction, the threat is initially quantified in a single source processing stage. While calculating a preliminary threat score for each single source target, a corresponding uncertainty estimate is also calculated. For example, a rule-based threat assessment model (e.g., IF category = specific object AND speed >60km/hTHEN threat = 0.8) may have an uncertainty of the model itself (ambiguity of rule) or an uncertainty of the input (e.g., category identification confidence is only 0.7). These input uncertainties are quantized to the variance of the initial threat score, var threatobs, by uncertainty propagation (such as Monte Carlo simulation, bayesian method, or first order linear approximation). The variance is packaged with the preliminary threat scores in the initial target threat assessment data. The variance value (i.e., observed threat uncertainty) is an input to a subsequent build uncertainty correction factor.
In another embodiment of the present application, computing threat consistency cost data includes:
And extracting historical threat scores corresponding to the tracks based on the multi-source target matching candidate data and the historical multi-sensor fusion target model data to form threat score data at the last moment.
In this embodiment, the threat scoring data at the last time is extracted when calculating the cost for the candidate matching pair (track i, observation j). Specifically, the system retrieves the fusion threat score of the track i at the last moment from the historical multi-sensor fusion target model data (i.e. generated in the last period) according to the track number i in the multi-source target matching candidate data, and marks the fusion threat score as Threat prev (i). The score is stored in the threat score data at the last time as a benchmark for threat smoothness comparisons.
And estimating an updated threat score when the candidate matching is established based on the multi-source target matching candidate data, the initial target threat assessment data, the single-source target state data and the prestored unified threat assessment model parameter data, and forming updated threat score data.
In this embodiment, it is assumed that a match (i, j) is true and a new threat score for track i that would result from the match is predicted. In a preferred implementation manner, updated threat scoring data is formed, and the method comprises the steps of extracting threat related state information corresponding to observation from single-source target state data according to the observation number in multi-source target matching candidate data, wherein the threat related state information comprises target category, speed, distance or course data, substituting threat related state information into a unified threat assessment model defined by unified threat assessment model parameter data, and reevaluating the threat at the current moment of a track in a candidate matching establishment scene to obtain updated threat scoring data. Specifically, the system extracts threat-related state information corresponding to observation j from single-source target state data according to the observation number j in the multi-source target matching candidate data. Threat-related state information is a key physical quantity required to evaluate a threat, and illustratively includes target class data (e.g., a particular object, confidence level 0.9), target speed data (e.g., 70 km/h), target distance data (e.g., 5km, indicating that a threat zone has been entered), and target heading data. The system substitutes the threat related state information into a preset unified threat assessment model (defined by unified threat assessment model parameter data) which is more complex than the initial model, re-assesses the threat of the track i at the current moment, obtains updated threat scores Threat new (i, j), and forms updated threat score data.
And calculating the threat variation amplitude between the updated threat grading data and the threat grading data at the previous moment, and normalizing according to the threat grading range in the unified threat assessment model parameter data to obtain threat consistency cost data.
In one possible embodiment, the threat variation amplitude is calculated and normalized to obtain threat consistency cost data, which specifically includes:
The method comprises the steps of calculating original threat variation between updated threat grading data and last-moment threat grading data, constructing an uncertainty correction factor based on uncertainty of observed threats contained in initial target threat assessment data, uncertainty of historical threats contained in historical multi-sensor fusion target model data and uncertainty weight parameters in threat consistency constraint parameter data, carrying out weighted correction on absolute values of the original threat variation by using the uncertainty correction factor to obtain corrected threat variation amplitude, normalizing the corrected threat variation amplitude according to threat grading range, and generating threat consistency cost data.
Specifically, after the historical threat score Threat prev (i) and the updated threat score Threat new (i, j) at the last time are obtained, the original threat variance Δ Threat raw(i,j)=Threatnew(i,j)-Threatprev (i) is calculated. If observation j itself has a high uncertainty (e.g., var threatobs (j) is high), then the severe threat jump it causes (i.e., delta Threat raw is large) is likely unreliable and should be de-weighted in the correlation decision. Because the original threat variations are not directly used, but rather are subjected to reliability corrections. For this purpose, an uncertainty correction factor Weight uncertainty (i, j) is constructed. Preferably, the correction factor is based on observed threat uncertainty Var threatobs (j) (from initial target threat assessment data), historical threat uncertainty Var threatprev (i) (threat variance at a time on track i extracted from historical multisensor fusion target model data), and preset uncertainty weight parameters α and β (from threat consistency constraint parameter data). An exemplary uncertainty correction factor calculation formula is Weight uncertainty(i,j)=1/(1+α*Varthreatobs(j)+β*Varthreatprev (i)). It can be seen that when the threat uncertainty of the observation or track is high, the denominator becomes large and the correction factor approaches 0. The system applies the correction factor to calculate the corrected threat variation amplitude delta Threat adj(i,j)=|ΔThreatraw(i,j)|*Weightuncertainty (i, j). The corrected threat variation amplitude Δ Threat adj (i, j) is normalized to convert it to a cost value that is scale-comparable to the other costs (D state,Dapp). The threat score range on which the normalization is based (e.g., maximum threat score Threat max =1, minimum threat score Threat min =0, then range 1.0) may be obtained from the unified threat assessment model parameter data. The final threat consistency cost data D threat (i, j) is calculated as D threat(i,j)=ΔThreatadj(i,j)/(Threatmax-Threatmin. the D threat (i, j) value will be used to construct the composite association cost.
In a specific numerical case, assuming that the preliminary threat score of observation j is 0.9, the corresponding uncertainty estimate (i.e., observed threat uncertainty) is calculated to be Var threatobs (j) =0.5. Assuming that track i has a historical threat score of Threat prev (i) =0.6, its historical threat uncertainty is Var threatprev (i) =0.1. Assume that the updated threat score resulting from the re-evaluation (e.g., substitution into a finer model) is Threat new (i, j) =0.9. Let the uncertainty weight parameter α=1, β=1. The original threat variance is calculated as Δ Threat raw =0.9-0.6=0.3. The uncertainty correction factor is calculated as Weight uncertainty =1/(1+1×0.5+1×0.1) =1/1.6=0.625. The corrected threat change amplitude is calculated as Δ Threat adj = |0.3|0.625=0.1875. Let threat score range (Threat max-Threatmin) =1.0. The final threat consistency cost is calculated as D threat (i, j) =0.1875/1.0=0.1875. In contrast, if the uncertainty of observation j 'is extremely high, for example Var threatobs (j')=4.0, even if it leads to the same threat jump (Δ Threat raw =0.3), its correction factor will become Weight uncertainty =1/(1+14.0+10.1) =1/5.1≡0.196. the threat change amplitude after correction is only |0.3| 0.196 approximately 0.0588. And by utilizing uncertainty information, the interference of unreliable observation on threat association is effectively inhibited.
In a preferred embodiment, the process of generating unified target track data includes constructing an extended comprehensive association cost matrix. The rows of the matrix correspond to all known historical tracks and the columns correspond to all new observations of the current time slice. Element C total (i, j) in the matrix is the comprehensive association cost of the matching of track i and observation j. To handle track re-creation, termination and missed detection, the matrix is further extended by adding an option for each row (track) to not match or terminate tracks and setting a termination track Cost parameter Cost terminate_track for it, adding an option for each column (observation) to create a track and setting a new track Cost parameter Cost new_track for it. These cost parameters may be obtained from threat compliance constraint parameter data. And (3) taking the minimum total cost as an optimization target, and executing global optimal matching solution on the expanded comprehensive association cost matrix. The solution may be performed using a variety of algorithms, such as the Hungary algorithm, the JVC (Jonker-Volgenant) algorithm, or the auction algorithm. And updating the track state and generating unified target track data according to the solving result of the global optimal matching. The method comprises the steps of adding observation information to an observation list of a track i and preparing for state updating if the track i is matched with the observation j, continuing only a predicted state of the track i and updating a unmatched count of the track i if the track i is matched with the track j, marking the state of the track i as terminated if the track i is matched with the track j, and assigning a new track number and creating a new track for the track if the track j is matched with the track j.
In some preferred embodiments, a posterior correction mechanism is also provided in order to further improve the smoothness and robustness of the correlation results over the threat time series. The mechanism is triggered after the unified target track data is generated and before the state fusion and threat fusion are performed. Specifically:
and calculating threat time sequence fluctuation measurement data of the track based on unified target track data and historical multi-sensor fusion target model data.
In this embodiment, the system performs a smoothness check on the unified target track data. Each on-track is traversed and a threat scoring sequence for the track within the last sliding window (e.g., the last 5 time slices) is extracted from the historical multi-sensor fusion target model data based on its associated observation list. The system calculates threat time sequence fluctuation metric data based on the sequence, e.g., calculating a variance, a maximum adjacent difference, or a number of hops for the sequence.
When the threat time sequence fluctuation measurement data exceeds a preset fluctuation index, combining threat consistency cost data and unified target track data, and identifying suspicious observation association causing threat mutation.
Specifically, when the threat time series fluctuation metric data exceeds a preset fluctuation index (for example, the variance is greater than 0.2, or the adjacent difference is greater than 0.5), the system determines that an abnormal jump exists in the threat sequence of the track. At this point, the correlation record of the track within the fluctuation window (from the unified target track data) is rechecked, and the suspicious observation correlation (e.g., the correlation with extremely high value of D threat) that causes the severe jump of the threat is identified in combination with the threat consistency cost data (D threat).
And executing local posterior correction on the suspicious observation association to generate corrected unified target track data, wherein the step of executing state fusion and threat fusion comprises the step of executing state fusion and threat fusion on the corrected unified target track data.
In this embodiment, local posterior correction is performed on suspicious observation associations. The correction is a local, limited range reevaluation. For example, the system may attempt to disassociate the suspicious observation from the track and attempt to replace it with a suboptimal candidate match (if any) or directly mark the observation as not matching (i.e., as a new track treatment) while marking the original track as not matching at the time slice. The system will re-evaluate the locally adjusted track threat time series fluctuation index. If the local posterior correction can significantly reduce threat time series fluctuation metric data (e.g., variance reduction by more than 50%) without significantly increasing (or only slightly increasing) the composite correlation cost for the time slice, the system accepts the local adjustment and generates a corrected unified target track data. When the optimal scheme is adopted, the corrected unified target track data is transmitted for subsequent state fusion and threat fusion, so that finally output multi-sensor fusion target model data has higher time continuity and reliability in threat dimension.
According to one aspect of the application, threat consistency cost data is calculated, and the method can also be used for reading multi-source target matching candidate data and historical multi-sensor fusion target model data pre-stored in a system, and analyzing track numbers and time indexes related to each candidate matching. And according to the method, for each track number, a fused threat time sequence recorded in the historical multi-sensor fused target model data is utilized, according to the current time slice index corresponding to the candidate matching, a historical threat score Threat prev (i) of the last moment is obtained through forward searching, and if part of tracks do not have effective threat scores at the last moment, complementation is carried out according to an initialization rule (such as using the last effective threat score or using the average threat score of the current time slice) preset in threat consistency constraint parameter data. And generating threat scoring data at the last moment in one-to-one correspondence with each candidate matching pair, wherein the data comprises a corresponding relation between the track number and the historical threat score Threat prev (i). And reading multi-source target matching candidate data, initial target threat assessment data, single-source target state data and preset unified threat assessment model parameter data. For each pair of candidate matches, extracting threat related state information such as target category data, target speed data, target distance data, target heading data and the like corresponding to the observation from single-source target state data according to the observation number in the candidate matches, and simultaneously extracting initial threat score and initial threat uncertainty data corresponding to the observation from initial target threat assessment data to be used as a measure for the reliability of the current observation. Substituting the state information into a unified threat assessment model, and re-assessing the threat at the current moment of the track in the scene where the candidate matching is established through calculation processes such as type scoring, speed scoring, distance scoring, heading scoring and the like in the model to obtain an updated threat score Threat new (i, j). In this process, the threat score upper limit Threat max and threat score lower limit Threat min in the unified threat assessment model parameter data will be read synchronously for subsequent normalization of threat variation magnitudes. Updated threat score intermediate data is generated in one-to-one correspondence with each candidate match pair, which records threat scores Threat new (i, j) when the candidate match is established. And reading threat scoring data at the previous moment, updating threat scoring intermediate data and threat uncertainty data contained in the initial target threat assessment data, and simultaneously reading uncertainty weight parameters in threat consistency constraint parameter data. For each pair of candidate matches, historical threat scores are obtained from threat score data at the previous moment according to the track numbers, threat uncertainty corresponding to the current observation is obtained from initial target threat assessment data according to the observation numbers, for example, threat score variance Var Threat_obs (j) or equivalent uncertainty index is used for representing, and when needed, historical threat uncertainty Var Threat_prev (i) of the track at the previous moment can be obtained from historical multi-sensor fusion target model data to be used as a supplement. The original threat variance under candidate matches is calculated as delta Threat raw(i,j)=Threatnew(i,j)-Threatprev (i). To avoid unreliable observations from unduly affecting threat variation metrics, uncertainty modifier weights uncertainty (i, j), e.g., weight uncertainty(i,j)=1/(1+α*Varthreatobs(j)+β*Varthreatprev (i)), are constructed based on observed threat uncertainties and historical threat uncertainties. After obtaining the uncertainty correction factor, the corrected threat variation amplitude is calculated as delta Threat adj(i,j)=|ΔThreatraw(i,j)|*Weightuncertainty (i, j). Corrected threat variation amplitude data corresponding to each candidate matching pair is generated, which integrates threat variation size and observation reliability. The threat change amplitude data after correction is read, the threat score upper limit Threat max and the threat score lower limit Threat min in the threat assessment model parameter data are unified, and the threat change amplitude after correction is converted into dimensionless threat consistency cost value according to the normalization rule in the threat consistency constraint parameter data. Specifically, for each pair of candidate matches, normalization is performed using the formula D threat(i,j)=ΔThreatadj(i,j)/(Threatmax-Threatmin). After normalization, if necessary, a lower-limit truncation may be performed on the too-small threat consistency cost value according to a threshold in the threat consistency constraint parameter data to suppress numerical noise, but the cost value after the truncation processing is still used as a part of the threat consistency cost data. And finally outputting threat consistency cost data.
In accordance with another aspect of the application, threat reduction benefit assessment data is generated, which may also be reading target threat uncertainty data, and observed target threat uncertainty prediction data. For each sensor and target combination, extracting the current threat score variance Var Threat_before (k) of the target from the target threat uncertainty data according to the target number, and extracting the predicted threat score variance Var Threat_after (s, k) of the target under the condition that the sensor executes observation from the observed target threat uncertainty prediction data according to the combination of the sensor number and the target number. And matching the threat uncertainty before and after the observation of each sensor and the target combination through the indexing and matching operation to obtain threat uncertainty comparison intermediate data, wherein the threat uncertainty after the observation, the threat uncertainty before the observation and the threat uncertainty after the observation comprise the target number and the sensor number. The threat uncertainty is read against the intermediate data. For each sensor and target combination, a threat uncertainty reduction corresponding to the combination is calculated from threat uncertainty data recorded therein, ΔVar Threat(s,k)=VarThreat_before(k)-VarThreat_after (s, k), wherein threat score variance Var Threat_before (k) represents threat uncertainty of target k at the current time, threat score variance Var Threat_after (s, k) represents predicted threat uncertainty after sensor s performs observation, both derived from threat uncertainty vs. intermediate data. In order to avoid the interference of an unexpected scheme of increasing threat uncertainty caused by observation on subsequent optimization, for the combination that the threat uncertainty reduction is smaller than zero or the absolute value is lower than a preset threshold, setting the corresponding threat uncertainty reduction to zero according to preset threat reduction benefit cutoff parameter data, so that no forward benefit or negligible benefit exists in the threat reduction dimension of the observation action. This truncation process generates threat uncertainty reduction data for each combination where only the non-negative, practically meaningful threat reductions remain. And reading threat uncertainty reduction data and preset threat reduction benefit normalization parameter data. And counting threat uncertainty reduction amounts of all the sensor and target combinations to obtain the largest threat uncertainty reduction amount Max Δ and the smallest non-zero threat uncertainty reduction amount Min Δ in all the combinations in the current decision period, and using the largest threat uncertainty reduction amount Max Δ and the smallest non-zero threat uncertainty reduction amount Min Δ in the current decision period for subsequent normalization calculation. For each sensor and target combination, its threat uncertainty reduction Δvar Threat (s, k) is mapped to a normalized threat reduction benefit value Δu norm(s,k)：ΔUnorm (s, k) = {0, if Δvar Threat(s,k)=0;(ΔVarThreat(s,k)-MinΔ)/(MaxΔ-MinΔ), if Δvar Threat (s, k) >0}, where the normalized threat reduction benefit Δu norm (s, k) is located within the interval [0,1] to represent the relative effectiveness of the observation action in terms of threat uncertainty reduction relative to other candidate observations. If Max Δ is too close to Min Δ, the strategy in the threat reduction revenue normalization parameter data may be modified to a simplified normalization, e.g., directly normalized to a maximum value, to avoid numerical instability. The normalized threat reduction benefit data is read, along with sensor identification information in the target threat uncertainty data and the multi-source sensor state data. For each sensor and target combination, the normalized threat reduction benefit is associated with the corresponding sensor number, target number, forming threat reduction benefit assessment matrix data indexed by sensor as row and indexed by target as column, or an equivalent list structure. In the process, some non-executable combinations (such as observable areas where the sensor cannot reach a certain target in the current period) can be marked according to the scheduling constraint parameter data, and the corresponding threat reduction benefits are directly set to zero or marked as unavailable, so that the threat reduction benefit assessment data is ensured to be consistent with the constraint conditions of the task assignment optimization model constructed later.
In general, the multi-modal target fusion and evaluation method provided by the application comprises the steps of aligning multi-source observation data and carrying out single-source processing. In the track association stage, threat consistency cost is built, weighted and fused with state and appearance cost, global association optimization is carried out to ensure smoothness of threat scores, in the collaborative scheduling stage, a threat reduction benefit evaluation model is built, expected reduction of threat uncertainty caused by observation actions is quantified through fusion of state covariance and threat evaluation gradient, an optimization model aiming at maximization of the benefit is built, and multi-sensor scheduling scheme data is solved and generated. The deep coupling of association and scheduling to threat assessment tasks is realized, and the association accuracy and the effectiveness of scheduling decisions are improved.
The invention carries out mathematical quantification on the threat uncertainty of the high-level semantics by combining the target fusion state covariance and the threat assessment model gradient, and can foreseeably predict the effect of reducing the uncertainty by different observation actions. By using the benefits as optimization targets to carry out cooperative scheduling, the sensor resources are preferentially used for solving the most critical threat cognitive ambiguity, and the deep coupling of scheduling decision and evaluation tasks is realized. Threat consistency costs are introduced in the global association optimization. The cost and the traditional state and appearance cost form a comprehensive association cost function, so that association decisions must be simultaneously satisfied with the smoothness of kinematics and threat assessment on time sequence. The severe jump of threat score caused by ambiguity or misassociation is effectively restrained, and the stability and credibility of situation results are ensured. Meanwhile, an uncertainty weighting mechanism is further introduced, so that excessive interference of unreliable observation on threat continuity is avoided, and the robustness of the system is improved.
The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to the specific details of the above embodiments, and various equivalent changes can be made to the technical solution of the present invention within the scope of the technical concept of the present invention, and all the equivalent changes belong to the protection scope of the present invention.
Claims (8)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202511755019.1A CN121211368B (en) | 2025-11-26 | 2025-11-26 | Multi-mode target fusion and evaluation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202511755019.1A CN121211368B (en) | 2025-11-26 | 2025-11-26 | Multi-mode target fusion and evaluation method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN121211368A CN121211368A (en) | 2025-12-26 |
| CN121211368B true CN121211368B (en) | 2026-03-06 |
Family
ID=98112124
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202511755019.1A Active CN121211368B (en) | 2025-11-26 | 2025-11-26 | Multi-mode target fusion and evaluation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN121211368B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110866887A (en) * | 2019-11-04 | 2020-03-06 | 深圳市唯特视科技有限公司 | Target situation fusion sensing method and system based on multiple sensors |
| CN120805088A (en) * | 2025-09-16 | 2025-10-17 | 辽宁云盾网力科技有限公司 | Behavior analysis early warning method and system based on AI situation awareness |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011001162A1 (en) * | 2009-06-29 | 2011-01-06 | Bae Systems Plc | Estimating a state of at least one target using a plurality of sensors |
| CN109357696B (en) * | 2018-09-28 | 2020-10-23 | 西南电子技术研究所（中国电子科技集团公司第十研究所） | Multi-source sensor information fusion closed-loop testing framework |
| CN117828527B (en) * | 2023-12-29 | 2025-06-13 | 中国空间技术研究院 | Multi-source data fusion and situation generation method and system |
-
2025
- 2025-11-26 CN CN202511755019.1A patent/CN121211368B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110866887A (en) * | 2019-11-04 | 2020-03-06 | 深圳市唯特视科技有限公司 | Target situation fusion sensing method and system based on multiple sensors |
| CN120805088A (en) * | 2025-09-16 | 2025-10-17 | 辽宁云盾网力科技有限公司 | Behavior analysis early warning method and system based on AI situation awareness |
Also Published As
| Publication number | Publication date |
|---|---|
| CN121211368A (en) | 2025-12-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115731268B (en) | Unmanned aerial vehicle multi-target tracking method based on vision/millimeter wave radar information fusion | |
| CN110866887A (en) | Target situation fusion sensing method and system based on multiple sensors | |
| CN120121039B (en) | Unmanned aerial vehicle-based high-precision topographic mapping system and method | |
| CN116736330A (en) | A method of obtaining robot laser odometry based on dynamic target tracking | |
| Cao et al. | Correlation-based tracking of multiple targets with hierarchical layered structure | |
| Ren et al. | Improved shape-based distance method for correlation analysis of multi-radar data fusion in self-driving vehicle | |
| CN117928559A (en) | Unmanned aerial vehicle path planning method under threat avoidance based on reinforcement learning | |
| CN120610568A (en) | Obstacle avoidance method and system for electric UAV based on multimodal perception and reinforcement learning | |
| CN120314926A (en) | Track fusion method based on incremental update Gaussian mixture model and multi-state transition filter | |
| CN119916806A (en) | Unmanned surface vehicle path planning method and system | |
| Zhao et al. | A novel transformer-aided multiple model algorithm for maneuvering target tracking | |
| CN120521616B (en) | A robust autonomous navigation and obstacle avoidance method for underwater inspection robots | |
| CN121211368B (en) | Multi-mode target fusion and evaluation method | |
| CN120927022A (en) | Unmanned vehicle field scene passable region segmentation and navigation method based on air-ground coordination | |
| CN120913453A (en) | Low-altitude aircraft state monitoring method and system based on 5G base station iron tower | |
| CN117724059A (en) | Multi-source sensor fusion track correction method based on Kalman filter algorithm | |
| Zhang et al. | Vision-based uav positioning method assisted by relative attitude classification | |
| Zhang et al. | CALFUSE-KAN: A multi-scale feature fusion network for aircraft trajectory prediction | |
| CN121028788B (en) | Multi-agent cooperative collision prevention picking method based on digital twin and deep reinforcement learning | |
| CN121256654B (en) | A Method and System for Bridge and Tunnel Defect Detection Based on UAV-Edge Computing Collaboration | |
| Yin et al. | Evaluation of Autonomous Navigation Robustness Based on Analytic Hierarchy Process | |
| CN119935248B (en) | Amphibious ship target detection system and method | |
| CN120030499B (en) | A fusion method of multimodal data complementation, error correction and loss tolerance mechanism | |
| CN119903293B (en) | An intelligent flight mode recognition method based on IFPRM-SBLFS deep learning | |
| CN121635392A (en) | UAV cooperative tracking system based on multimodal optoelectronic fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |