A THREAT ASSESSMENT SYSTEM AND PROCESS
FIELD OF THE INVENTION
The present invention relates to a threat assessment system and process.
BACKGROUND
As described in Pearl, Judea, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann, San Mateo, CA, 1988 ("Pearl"), Bayesian networks can be used to predict or infer outcomes in the physical world represented by one or more variables whose values are unknown (referred to herein as 'unknown variables') based on observations of other variables having some causal relationship with the unknown variables. In this context, inference is a process of generating probabilities for the unknown variables based upon predetermined causal relationships between those variables and other variables whose values are known or at least estimated with some uncertainty.
Prior art processes for inferring outcomes have used Bayesian belief functions with causal networks, junction tree algorithms, and Pearl's belief propagation, as described in Pearl. However, the resulting inferred outcomes are analytically cumbersome and typically require considerable human interpretation to be used in practical context. For example, in a military context, Bayesian networks have been used to predict the location and type of intruder aircraft detected by multiple sensors and other sources of information. However, the resulting information nevertheless requires substantial inteφretation and evaluation by a suitably experienced human to determine, for example, whether any action is required in response to a perceived threat of the intruder aircraft to one or more assets such as ground targets.
Furthermore, although Bayesian belief networks can support backward propagation of evidence, (e.g., where an outcome is known and it is desired to update the causal relationships in the network), the computational requirements of prior art methods increase
exponentially with the complexity of the Bayesian network, making backward propagation infeasible in situations of practical complexity. Consequently, the Bayesian networks that have been developed are undesirably limited in the number of variables that can be included.
It is desired, therefore, to provide a threat assessment system and process and a causal network that alleviate one or more of the above difficulties, or at least provide a useful alternative.
SUMMARY OF THE INVENTION
In accordance with the present invention, there is provided a threat assessment system, including: one or more linearisation modules for generating by linear approximation a conditional probability distribution of a first continuous variable on the basis of a non-linear causal relationship between said first continuous variable and a continuous state variable representing a state of a threatening entity, and for generating by linear approximation a conditional probability distribution of a second continuous variable representing a threat posed by said threatening entity to an asset on the basis of a non-linear causal relationship between said second continuous variable, said first continuous variable, and a discrete state variable representing a state of said threatening entity; a multiplier module for generating, on the basis of said conditional probability distributions, a joint belief function for assessing said threat; and a belief function generator for generating a belief function for said second continuous variable on the basis of said joint belief function.
The present invention also provides a threat assessment system, including: a causal network module representing causal relationships between variables for assessing a threat posed by a threatening entity to an asset, said causal relationships including one or more non-linear relationships; one or more parameterisation modules for generating conditional probability distributions for said variables, including generating by linear approximation one or more conditional probability distributions representing said one or more non-linear relationships; and a multiplier module for generating a joint belief function for assessing said threat on the basis of the conditional probability distributions for said variables.
The present invention also provides a threat assessment system for assessing at least one threat posed at least one threatening entity to at least one asset, the system including a capability generator for generating one or more capability values representing respective capabilities of one or more threatening entities to threaten one or more assets; an intent generator for generating one or more intent values representing intent of said one or more threatening entities to threaten said one or more assets; and a threat assessment module for generating one or more threat values representing respective threats posed by said one or more threatening entities to said one or more assets on the basis of said capability values and said intent values.
The present invention also provides a threat assessment system for assessing at least one threat posed at least one threatening entity to at least one asset, the system being adapted to generate conditional probability distributions representing relationships between nodes of a causal network representing said at least one threat, said nodes including at least one entity state node representing a state of a corresponding threatening entity, criteria nodes representing variables dependent on said state and being child nodes of said at least one entity node, and at least one threat node being a child node of said criteria nodes, said at least one threat node representing a threat posed by said at least one threatening entity to a corresponding asset.
The present invention also provides a threat assessment system for assessing at least one threat posed at least one threatening entity to at least one asset, the system being adapted to generate conditional probability distributions representing relationships between nodes of a causal network, said causal network including: at least one entity node representing a state of a corresponding threatening entity; a capability node for each entity-asset pair representing the capability of a corresponding threatening entity to threaten a corresponding asset, the capability node being a child node of the entity node of the corresponding threatening entity; an intent node for each entity-asset pair representing the intent of the corresponding threatening entity to attack the corresponding asset, the intent node being a child node of the entity node of the corresponding threatening entity; and at least one threat node representing a threat posed by a corresponding threatening entity to a corresponding asset, said at least one threat node being a child node of the corresponding capability node and the corresponding intent node.
The present invention also provides a threat assessment system, including: a causal network module representing causal relationships between variables for assessing a threat posed by a threatening entity to an asset; one or more parameterisation modules for generating conditional probability distributions representing linear relationships of said causal relationships one or more linearisation modules for generating by linear approximation conditional probability distributions representing non-linear relationships of said causal relationships; and a multiplier for generating a joint belief function for assessing said threat by multiplying the conditional probability distributions representing said relationships.
The present invention also provides a threat assessment process, including: generating by linear approximation a conditional probability distribution of a first continuous variable on the basis of a non-linear causal relationship between said first continuous variable and a continuous state variable representing a state of a threatening entity; generating by linear approximation a conditional probability distribution of a second continuous variable representing a threat posed by said threatening entity to an asset on the basis of a non-linear causal relationship between said second continuous variable, said first continuous variable, and a discrete state variable representing a state of said threatening entity; generating, on the basis of said conditional probability distributions, a joint belief function for assessing said threat; and generating a belief function for said second continuous variable on the basis of said joint belief function to assess said threat.
The present invention also provides a threat assessment process, including: generating one or more capability values representing respective capabilities of one or more threatening entities to threaten one or more assets; generating one or more intent values representing intent of said one or more threatening entities to threaten said one or more assets; and generating one or more threat values representing respective threats posed by said one or more threatening entities to said one or more assets on the basis of said capability values and said intent values.
The present invention also provides a threat assessment process, including: determining causal relationships between variables for assessing a threat posed by a threatening entity to an asset, said causal relationships including one or more non-linear relationships; generating conditional probability distributions for said variables, including generating by linear approximation one or more conditional probability distributions representing said one or more non-linear relationships; and generating a joint belief function for assessing said threat on the basis of the' conditional probability distributions for said variables.
The present invention also provides a threat assessment process, including: receiving tracking and- identification data including conditional probability distributions of kinematic data and type data of a threatening entity, said kinematic data representing a location and velocity of the threatening entity, and said type data representing a type of said threatening entity selected from a plurality of entity types; generating, on the basis of the conditional probability distributions of kinematic data and a location of an asset, conditional probability distributions of entity-asset data representing separation of the threatening entity and the asset and an angle between the velocity vector of the threatening entity and a vector joining the threatening entity and the asset; generating, on the basis of the conditional probability distributions of entity-asset data, a conditional probability distribution of threat data with respect to the entity-asset data and the type data of the threatening entity; multiplying the conditional probability distributions of entity-asset data, the conditional probability distribution of threat data, the conditional probability distributions of kinetic data, and the conditional probability distribution of type data to generate a joint belief function of said threat data, said entity-asset data, said kinetic data, and said type data; and generating, from said joint belief function, respective belief functions for one or more of said threat data, said entity-asset data, said kinetic data, and said type data.
BRIEF DESCRIPTION OF THE DRAWINGS
Preferred embodiments of the present invention are hereinafter described, by way of example only, with reference to the accompanying drawings, wherein: Figure 1 is a criteria based causal network for assessing threat in accordance with a preferred embodiment of the present invention; Figure 2 is a Bayesian belief network for assessing threat in accordance with a preferred embodiment of the present invention, illustrating the contributions of continuous and discrete nodes to threat; Figure 3 is a block diagram of a preferred embodiment of a threat assessment system; Figure 4 is a flow diagram of a threat assessment process of the system; Figure 5 is a flow diagram of a threat estimation process of the threat assessment process; Figure 6 is a schematic diagram of a threat scenario wherein a threatening entity or intruder U, approaches an asset Ay, Figure 7 is a causal model used by the threat assessment system to represent the relationships between data from sensors, capabilities of one or more intruders with respect to one or more assets, intents of those intruders with respect to those assets, and the resulting threats to each asset; Figure 8 is an image of a surveillance region containing assets, illustrating the actual paths taken by two intruder aircraft, referred to as targets 1 and 2; Figure 9 is an image of the surveillance region of Figure 8, illustrating the paths taken by the two intruder aircraft as generated by a tracking and data fusion system and provided as input to the threat assessment system; Figures 10 and 11 are graphs of the height and velocity estimates, respectively, for the two intruder aircraft as a function of time step, as generated by the tracking and data fusion system and provided as input to the threat assessment system;
Figure 12 includes graphs of the variances of the location and velocity components as a function of time step for target 1 , as generated by the tracking and data fusion system and provided as input to the threat assessment system; Figure 13 includes graphs of the variances of the location and velocity components as a function of time step for target 2, as generated by the tracking and data fusion system and provided as input to the threat assessment system; Figures 14 and 5 include graphs of the aircraft type estimates for target 1 and target 2 of Figure 9, respectively, as generated by the tracking and data fusion system and provided as input to the threat assessment system; Figure 16 includes graphs of the means of the two intermediate variable components as a function of time step, as generated by the threat assessment system; and Figure 17 includes graphs of the means and variances of the threats posed by both intruder aircraft to each of the two assets as a function of time step, as generated by the threat assessment system.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
The threat posed by an entity to one or more assets can be estimated using a causal network, as shown in Figure 1. The state of the entity, represented by node X, is determined from independent sources of information or measurement nodes Si, S"2, ..., Sm that are parents to node X. At the output end of the network, threat variables zj, Z2, ..., zn represent the level of threat to assets Aι, A2, .... A„ respectively. The threat variables are determined using criteria variables cj, C2, .... cr applied to the entity state The causal relationships between the entity state, criteria, and threat variables are in general non-linear.
In general, the entity state X includes continuous and discrete components that can be represented by one or more continuous variables Xκ and one or more discrete variables Xp, as shown for a single entity and asset pair in Figure 2. A continuous variable is a variable whose value is real (i.e., XK e 5K) and continuous, whereas a discrete variable is a variable whose
value is restricted to one of a limited number of possible values. The continuous variables typically include one or more variables representing respective rates of change of one or more other continuous state variables. Corresponding continuous and discrete criteria nodes YK and YD are generated from XK and XD, respectively, and are also referred to as intermediate nodes. The threat Z posed by the entity to the asset is determined from the intermediate nodes YK and YD.
As shown in Figure 3, a threat assessment system based on the networks of Figures 1 and 2 includes threat assessment modules 302 to 314, including a causal network module 302, two linearisation modules 304, 306, a discrete parameterisation module 308, a multiplier 310, a selective marginalisation module 312, and a belief propagation module 314. A threat assessment process, as shown in Figure 4, allows the threat assessment system to generate probability distributions for one or more variables representing respective threats posed by one or more threatening entities to one or more assets from probability distributions for variables representing the state of the threatening entities received from information sources 316.
In the described embodiment, the threat assessment system is a standard computer system such as an Intel IA-32 or IA-64 based computer system executing a Windows operating system, and the processes executed by the system are implemented by software modules, being the threat assessment modules 302 to 314, stored on non- volatile (e.g., magnetic disk) storage associated with the computer system and executed by one or more processors of the system. Although not shown in Figure 3, the system also includes the Matlab software application, available from http://www.mathworks.com/products/matlab/. and Muφhy's Bayes Net Toolbox for Matlab (BNT), as described at http://www.ai.mit.edu/~muφhvk Software BNT/bnt.html. and the threat assessment modules 302 to 314 are based on matrix functions provided by Matlab and BNT. However, it will be apparent that the processes can alternatively be implemented entirely by dedicated software modules written in a programming language such as Fortran and omitting the BNT and Matlab components. Moreover, it will be apparent to those skilled in the art that the
components of the threat assessment system can be distributed over a variety of locations, and that at least parts of the processes executed by the system can alternatively be implemented by dedicated hardware components, such as application-specific integrated circuits (ASICs).
The threat assessment process and system are described below with reference to application of the system and process to air defence, wherein the threat posed to one or more physical assets by one or more intruding aircraft is assessed. However, it should be understood that the threat assessment process and system are not limited to an air defence scenario, but can be applied equally to threat assessment for land and sea scenarios with minimal adaptation. Moreover, the threat assessment process and system can also be used in non-defence related fields. For example, in economics, a network with the structure represented in Figures 1 and 2 can be used to monitor the health of an economy by estimating values for economic variables such as interest rates, building activities, employment rates, currency exchange rates, cost and availability of energy and other resources, climatic factors, etc. In a manufacturing or chemical processing plant, the threat assessment process can be used to monitor specific variables that can affect the health or output capacity of the plant through the use of sensors that are deployed within the plant. In environmental monitoring, the threat assessment process can be applied to assess and control threats to the health of a vital environmental region by identifying and tracking health indicators through the use of environmental data from a wide range of sensors and/or other sources of information. Accordingly, the threat assessment system and process are described below, both in general terms, and also with reference to application.
In order to use the threat assessment process to assess the threat posed by a threatening entity, the first step is to identify the variables relevant, at step 402 of the threat assessment process, as shown in Figure 4. In an air defence scenario, a geographical area contains a distribution of assets that are to be defended against intruder aircraft of different types equipped to launch various types of weapons. An air defence commander has at his or her disposal a number of interceptors of varying capabilities that can be launched to intercept the intruding aircraft on
the basis of knowledge of the asset locations and an assessment of the level of threat posed by the intruding aircraft. The information sources 316 includes a tracking and data fusion system that continually generates track and identification data providing a comprehensive description of each aircraft in the surveillance region, based on input from a network of sensors Si, S2, ■■■, Sm and possibly other sources of information, as described in N. N. Okello and D. W. McMichael, "Capabilities and limitations of data fusion in AEW&C," Tech. Rep. 26/98, CSSIP (The Cooperative Research Centre for Sensor Signal and Information Processing), September 1998 ("Okello 1998"), and N. Okello, P. Scoullar, and S. Challa, "Association and Identity Inference for the Air Picture Compilation Problem," Tech. Rep. CR 16/00, CSSIP, July 2000 ("Okello2000"). The sensors themselves may be included in the assets being protected.
The track and identification data generated by the tracking and data fusion system includes observable kinematic and discrete state estimates and associated uncertainties. Referring to Figure 2, the kinematic estimates are provided by a continuous matrix variable XK = [ ξ(k),ξ(k),η(k),ή(k),ζ(k),ζ(k) J that represents the kinematic component of the target (aircraft) state, with ξ(k), η(λ), and ζ(k) being the three orthogonal spatial coordinates of the intruder aircraft at time step k, and ξ(k),ή(k), (k) being the corresponding velocity components. The discrete variable xD = Tq e {Ty, ..., T^r TA^-I} represents the discrete component of target state: Tq identifies the type of intruder aircraft from a list of NT possible aircraft types, with Tq = 7/v n representing all unknown aircraft types. These seven variables are associated with respective probability distributions.
In general, a surveillance region includes Nu unknown entities, referred to as intruders, and NA assets, where Ut, i = I, ..., Ny is the z'-th intruder, and Ay, j - 1, ..., N^ is they'-th asset. Figure 5 is a schematic illustration of a surveillance region with an intruder (7, approaching a number of assets, illustrating the geometrical relationship between a stationary asset Aj with a known location, and the intruder Ut travelling at velocity vt and equipped with a weapon system whose range envelope is semicircular with radius rL. In order to evaluate the threat
posed by an intruder U, on asset Ay, the variables that control the threat level given an intruder-asset pair (U„ Aj) are determined.
In the described defence application, the threat assessment system uses a general inference network, as shown in Figure 7, and the inference networks in Figures 2 and 3 to infer the threat to assets A,, j = 1, ..., N^ based on separate evaluations of variables representing an intruder's capability Cυ and intent Iy with respect to each asset, determined from track and identification data for the intruder U, generated by the tracking and data fusion system from sensors and any other available relevant information. Returning to Figure 5, once the relevant variables - have been determined, the causal relationships between these variables is determined at step 404.
The level of threat posed to the asset Aj by the intruder U, depends on the intruder-to-asset Δ range ry, its rate of change r — d rtJ I dt and the maximum range rι of the intruder's weapon system. This dependence is non-linear. Intuitively, the threat to an asset posed by a very distant intruder is essentially non-existent or very low, and should increase as the intruder approaches the asset. The threat level should then reach a maximum when the asset falls within the range of the intruder's weapon system, i.e., when the intruder is able to overlay its weapon range envelope over the asset. Accordingly, the intruder's capability to threaten the asset is defined as: zc <x 9c ({rL, rij Λ) = I x' * lr«l < r* (1) ( τ^τ , if |ry | ≥ r
An intruder with a semi-circular frontal weapon envelope of radius rι is deemed to pose no threat if it is receding with respect to the stationary asset, i.e., if r > 0. However, when r < 0 , the threat level is considered to be proportional to cosθ, where θv is the angle between the intruder velocity vector and the intruder-to-asset range vector rυ. Accordingly, the intruder's intent to threaten the asset A} is defined as:
The threat /,, to an asset A
} by an intruder U, is then defined as the product of its intent and capability components, as follows:
li3 = kgc(r
L, r
t3)gι(f
tJ , v
3) if |r„| < r
L and r
tJ < 0
l ϊ(føf)
'
0, if r
%3 > 0 and 0 < |r
y | < oo
or, equivalently, cos θ , if |ry | < rL and |0t-7 | < _ 2 — ^ 2 1%3 — cos θl3 , if |r | > r and |^ | < (4) 0, if \θtJ I > f and 0 < \rtJ \ < ∞
where -π < θy < π is the angle between the ;-th intruder velocity vector v, and the intruder-to- asset range vector ry , and k = 1 is a constant. Thus the threat level ItJ e [0, 1] is a non-linear discontinuous function of the variables rϋ , Qυ , and rx.
Returning to the inference network of Figure 2, the capability and intent components of threat are represented by an intermediate continuous variable or node y* = [ |r„(*)| θ„(A) ]3 (5)
that represents the total continuous information that is required for threat assessment. In alternative embodiments, this node may be one of several possible criteria nodes, and in the general case there will be one or more continuous nodes and one or more discrete nodes.
T
If xA= [ξA ηA CA] is the location of the y'-th asset, then the intruder-to-asset range vector is
Furthermore, if v,(A;) = [ ξ(k) ή(k) ζ(k) ]
τis the velocity vector of the intruder, and
y is the angle between the intruder velocity vector v/ft) and the intruder-to-asset range vector r,
y , then
is the intermediate vector that links the kinematic component of the intruder state vector to the threat variable. The variable yκ is therefore a non-linear function of the kinematic component of the target state vector χ( = lxκ k) ^o(fc) ] n the asset location x^, and therefore can be written as:
YK =A*κ(k), xA) (8)
where XD(Λ) is the intruder type selected from the set T= {T\, ..., T^T) 7/yy j }.
The variable yjr
> is discrete and is another of possibly several criteria nodes. Its node Y
D represents the total discrete information that is required for threat assessment. It is related to X
D by the mapping XD — — > Y
D where A is an appropriately dimensioned matrix that defines the relationship between components of XD and those of YD- In the described embodiment, this relationship is deterministic in that knowledge of the intruder type gives complete information about the weapon system, including weapon envelope parameters, and A is a unitary matrix with a one in each row or column and maps the type of intruder aircraft to its weapon type. For the sake of simplicity, A = I, where I is the identity matrix, so that
where δ is the Dirac δ-function. The variable z = Iy is continuous and represents the threat posed by intruder i to asset j. It is related to the criteria variables through equation (4) above, and is a non-linear function of the discrete component, *£>(&), of the intruder state vector x(k).
Having determined the variables relevant to threat assessment at step 402, and the relationships between those variables at step 404, the causal network 302 is configured by defining the above functions for y& z, and XD at step 406. For example, in the described application of the system to air defence, y -f(xκ) is defined as a vector function that depends on the continuous component of intruder state, of which the function components are the range of the intruder from the asset, and the angle between the intruder velocity vector and the bearing of the asset with respect to the intruder; A.XD → yυ is a matrix that maps target type information to weapons systems; and z = g(yκ,yϋ) is a scalar function of the target type (the aircraft type XD determines the weapon launch range) and the intermediate variable y . The intermediate variable yK is one of two dependent variables of the threat. The second dependent variable, yo, represents the weapon system type, which is discrete.
At step 408, any non-linear relationships (i.e., the causal network functions defined at step 406) of the casual relationships determined at step 404 are identified. In this particular military application of the system, the relationships defining the continuous variables y and z are both non-linear. For example, the intermediate node YK is a child of node XK based on a non-linear relationship. Having identified the non-linear relationships, at step 408 the system is configured to linearise those relationships by applying first-order Taylor series approximations to them, as described below. Having configured the threat assessment system, the system is now ready to generate estimates for threat by executing a threat estimation process 412, as shown in Figure 5.
The threat estimation process 412 begins at step 502 by receiving the probability distributions for the continuous variables XK and the discrete variables X from the information sources 316, being in this military application, a level 1 tracking and data fusion system.
The threat assessment system approximates each continuous node by a Gaussian probability distribution, ensuring that the results will be conservative in the sense that a Gaussian distribution represents the worst possible case and will produce the least accurate results. If
more information subsequently becomes known, such as the actual probability distribution of one or more variables, then the results will be more accurate. Nodes X and X , representing the state vector of intruder U„ have no parents and their priors are obtained from the output of the level 1 data fusion system, as described in D.L. Hall and J. Llinas, "An Introduction to Multisensor Data Fusion," Proceedings of the IEEE, vol. 85, No. 1, pp. 6-23, Jan. 1997. The output of the level 1 data fusion system is:
S (k\k) = [x
κ(k)
τ i
D(k) = T
q ]
T, (10) where T
q = max
Tp{P (T
p\W
k),p = 1, ..., N
T, N
T + 1], then the input node priors are: p(κ
κ(k)\W
k) = N [x (rc); x (rc|fc), P(fc|fc)] (11) and p(xo(A:)|W
rfc) = [P(r
1 |W
r*) . . . P(T
Nτ+l \W
k) (12) i.e., in this case, the input node prior for XK is a Gaussian distribution with mean x
κ (k\k) and covariance matrix
, where the notation (k\k) indicates that the value of the corresponding variable is for time k and taking into account all known information up to time k, and the notation N(x; μ σ) represents a Normal or Gaussian distribution of variable C with mean μ and standard deviation σ, and π denotes the set of all sensor measurements up to time k. The tracking and data fusion system provides the data, represented by equation (10), (11) and (12) as described in Okellol998 and Okello2000.
At steps 504 and 506, an approximate conditional probability distribution (CPD) for nodes Y and Z is determined, based on the linear Gaussian approximation. Specifically, at step 504, a first-order Taylor series approximation of equation (8) about xκ(k) = xκ(k\k) is used to generate the following conditional Gaussian distribution from p(xκ\ W )'. with the time index k omitted for notational convenience, yields
ΎK
W (XA-, XA) + (XA: - XΛΓ)
This approximation therefore transforms y from a deterministic to a random variable, and so
where E denotes expectation value, or
where
Wγ
κ E df (17) XK Ldx | ΛT=XK K
If xκ is approximated by a Gaussian distribution, then the covariance matrix of the variable yK is given by:
∑γκ = ATP(k \ k)A (18)
where
The conditional probability density function between the continuous nodes X and Y^ can therefore be approximated by a conditional Gaussian distribution and takes on the form
where the mean μy^ the regression (weight) matrix Wγκ, and the covariance matrix ∑γκ are defined in equations (16), (17), and (18), respectively.
The output node Z is a child to a discrete node Y and a continuous node YK and so, at step 506, the following conditional probability density function is generated: p(z\yκ, VD) = N |z; μz - Wzyκ, ∑z (21)
where
diag [ ∑
2(l) . Σ (ΛΓ
T + I) ] (23)
Thus the components of the NT +1 dimensional node XD give rise to respective components of the mean μχ and covariance ∑z of the one-dimensional node Z.
At step 508, the conditional probability distribution of any discrete intermediate variables is generated. As described above, in this military application, this distribution is represented by a delta function. However, other applications of the system will in general use alternative mappings from input discrete variables to intermediate discrete variables.
In this case, the node variable Z is nonlinear and discontinuous with respect to YK, and so Z conditioned on YK has three possible modes as defined in equation (4). Consequently, the conditional probability distribution of the variable z is evaluated for each of these three modes.
For the first mode of equation (4), and for p= 1,2, ...,Nτ+ I,
z = cos θ. υ 9 (yκ,p)
This approximation transforms z from a deterministic to a random variable, and so
where μz(p) E dgι(yκ,p) κ,p)- \yκ ==yyκκΥK (26) YK 9i(y dy K dgι(y κ,p) Wz(p) EΫκ yκ=yκ (27) &yκ ∑Z(P) ARJ-ΘA1 , and A = 9gι(yκ,p), iyκ=yκ' dy K (28)
Similarly, for the second mode of equation (4): r
L z = — rcos θ
tj = 92( κ,P) (29)
■yl and so
where
~dg
2(yκ,p) W
z(p)
EΫκ \yκ=yκ (32) dy K ARrβA , and A = g2(yκ, p) Vz(p) (33) dy
κ \yκ=yκ -
For the third mode of equation (4) however, z - 0 and so μ p) = 0, (34) Wz(p) = [0 0], (35) and ∑z(p) = ε2, (36)
for/? = 1, .., Nr + 1, where ε is a small number.
Given the conditional probability distributions of equations (9), (11), (12), (20), and (21), and the Bayesian belief network in Figure 2, at step 510 a joint belief function for threat assessment is generated by the multiplier 310 as the product of all these conditional probability distributions, as follows: Be\(z, yK, yD,xK, xD) = p(z, yK, yD, xK, xD\Wk) = p(xκ\Wk)p(xD\Wk)p(yκ\xκ)p(yD\xD)p(z\yκ, yD) (37)
At step 512, belief functions for individual variables are generated from the joint belief function, as described below. The belief functions are generated by either the selective marginalisation module 312 or the belief propagation module 314, depending upon the variables in the threat assessment network.
Selective marginalisation, as described below, is preferred under all conditions. Selective marginalisation is computationally more direct and more economical than Pearl's belief propagation. Either process can be used if evidence is inserted at nodes other than the root (i.e., initial continuous and discrete) nodes.
The selective marginalisation module 312 generates a closed form expression of the belief function of each node variable by summing out any other discrete variables and integrating out any other continuous variables from the joint belief function of the network given by equation (37). This process of selective marginalisation is referred to herein as the direct integration process.
The direct integration process is performed as follows. From the joint belief function of equation (37), the distribution of any node variable conditioned on the measurement set Vr* is obtained by integrating and or summing out all the other node variables, as follows: Bel(z)
∑ / p(xκ\W
k)p(x
D\W
k)p(y
κ\xκ)p(yD\xD)p(z\yκ, yD)dxκdy
κ (38) X τDn ..VVDπ
JχK <VK
Following a process of substitution, completion of squares and maximization, the conditional mean and covariance of z take on the form μz\i = A^B
3 = Σ
zli(Σ^
Yκ W
z(i)A^M
2 + ∑-l
Yκ τμ
z(i)) (39)
The belief function for the intermediate function Y can also be evaluated by integrating out all the other variables within the joint belief function, as follows: Bel(yκ) = p(yκ\W
k)
= ∑ / P(xκ\W
k)p(y
D\W
k)p(y
κ\x
κ)p(z\y
κ, y
D)dx
κdz yo
Jx" '
z
Hence the mean and covariance of Y are
As described above, as an alternative to the selective marginalisation or direct integration process of equations (38) to (40), the belief functions can be generated by the belief propagation module 314, which applies belief propagation, as described in Pearl, to the Bayesian belief network of Figure 2, as described below.
Consider the Bayesian belief network of Figure 2. When node YD receives no evidence, as in the described embodiments, then XD and YD are equivalent, since A = /. In the derivation that follows, the variable XD is used in place of YD- However, it should be understood that in the more general case YD should be used. The nodes XK and XD are root nodes with no parents, and therefore can be written as forward propagation messages π, as follows:
π(xκ) ≡ p(xK\Wk) = N(xK; x, P) (44) ττ(xD) ≡ p(xD\Wk) = [P(T1 \Wk) P(T2\Wk) . . . P(TN+1 \Wk) } . (45)
The output node Z is a childless continuous node that cannot be instantiated but whose value lies in [0, 1]. Thus the backward propagation message λ(z) ~ U[0, 1] i.e., has a uniform distribution over [0,1], and this appropriately expresses the level of knowledge available on the variable Z. Alternatively, the interval restriction on the variable Z can be relaxed in order to take advantage of the Gaussian distribution by assuming that λ(z) ~ N(z; μz, ∑z), where μz = 0.5 and ∑z is a large number. From equation (4.45) in Pearl,
Bel(z) = αλ(z) pT / p(z\yκ, xD)πz(yκ)πz(xD)dyκ (46) XD J VK
where ziy) and UZ XD) are messages from parents Y and X respectively.
Now consider the message UZ(XD) that -; sends to Z. From equation (4.45) in Pearl, πz(xD) = aπ(xD) = [P(T1\Wk) P(T2\Wk) ... P(TN+1\Wk)}. (4?)
Consider again the message πγ(xκ) that j sends to Y. Applying equation (4.45) in Pearl, πγ
κ(x
κ) = aπ(x
κ) = π(x
κ) = N(x
K;x,P)
Now consider the message πz(yκ) that Y sends to Z. Applying equation (4.45) in Pearl followed by the integral version of equation (4.38) in Pearl, the following equation is obtained: πz{yκ) = aιπ(yf) = π(y
κ) = (49)
eXP[
_ 2^
A" ~ £)
T∑Xκ(
Xκ ~ x)\
dxκ where
B
1 = ∑
x l κ* + W?
κΣ?
κlXκ(y
κ-μ
Yκ). (52)
The conditional distribution p(z\y, XD) can be approximated by a linear Gaussian and written in the form: p(z\yκ,i) = N(z;μz(i) + Wz(i)yκ,∑zγκ ) (53)
Substituting equations (47) and (50) into (46) yields
Bel(z) = αexp[--(z- 0)T∑o1(z-μo)]
Bel(z) = aexp[--(z-μ
0)
τ∑
0 1(z-μ
0)] { -
μz(i))
-(∑ x -
- WΪ
κ∑
y Xκμ
Yκ)
= a
where ^3 = [Zzi
γκ,i-Z
zl
γκ,iW
z(i)Aϊ
1W$(i)∑-l
Yκι + Σ^] (55)
The belief function in equation (54) is a Gaussian mixture whose summand densities have means and variances given by μz\i = ∑
zi{∑zϊ
Yκtiμzi) + ∑zl
YκtiW
zi)A
2-
1M
2 + ∑c
1μ
0} (57) ∑
z„ = A = [∑-]
Yκi -
+ ∑o
1]
"1 (58)
and so the mean and covariance of such a mixture is given by iV
T4-l
∑Z + (μ
zli-z)(
μ Z\i
if) (60)
If ∑o is large, then
ι-l ∑
zli = A→ = [∑^
Yκ.-∑-]
Yκ.W
2(iA^W^(i∑-]
Yκ. (62)
and so the evidence at node Z does not enter the belief function for node Z.
Thus an estimate of the threat Z to one or more assets is obtained from equations (57) and (58) by substituting values from equations (11) and (12), which themselves rely on equations (20) and (21), which rely on equations (16), (17), and (18), and (22) and (23), respectively.
Repeating the above steps for node Y,
where λziy) is the message that Z sends to Y. Applying equation (4.44) in Pearl, λz(yκ) = β λ(z) p(z\xD, yκ)πz(xD)dz (64)
where Z(XD) is given in equation (47) and λ(z) ~ N(z; μz,∑z) where μz = 0.5 and ∑z is a large number to reflect the lack of knowledge on the uninitialized childless node variable Z.
The integrals in equations (63) and (64) are then integrated, after which equation (64) is substituted into equation (63). Thus:
= otλz(yκ){ exp [ - -{y
κ - μ
Yκ - Wγ
κx)
τ γl
Xκ (y
κ - μ
Yκ - Wγ
κx
κ)} exp [ - - (x
κ - x)
T∑
Xκ (XK - x)] dx
κ j = aλ
z(yκ){exp [ - [-BΪA
1B
1 +
τΣ
χ 1x + (
κ - μ
Yl<)
τΣ
γκ 1 lXκ (y
κ - μ
Yκ)] (65) where
Now rearranging the exponent in equation (65) in order to expose the quadratic in y, the following is obtained: - μ
Yκ))
+x
τ∑
χ 1x + (y
κ - μ
Yκ)
τΣγ
κ 1 Xκ(y
κ - μ
Yκ) } = aλ
z(yκ) exp { - \ [VK^
XK - ^y
κlχ
κWy
κA^W^Σ
γ^
Xκ)y
κ
- W?
κ∑
γl
]χκμy
κ) +x
τΣ
x 1 κx +
μYκΣ
γκ i lXκμ
Yκ]}. (68)
But ι λz{yκ) = β P{ yκ, D{ι))πz{xD{ι))dz
= β z(») + W
z{ι)yκ) + μj∑o V }]P(W
fc)
(69) where = (∑i,V
Kι, + ∑o
1) (
70> B
2 = (∑zly
Ktl μz(i) + Wz i)yκ) + ∑o
1 o).
(?1)
Now rearranging the exponent in equation (69) in order to expose the quadratic in y, provides
+(μz(ι) +
Wτ+1 . = β ∑ exp[--{y
T(w
T(ι)∑
zl
Yκ!tWz -Wz(τ
τ∑
zt
Yκ,1A^∑z
\γ
κ,
tWz(ι))y
κ t=l -2y
κ τ(w
z(ι)
τ∑-
zl
Yκ A ∑
z-\
Yκ μz{ι) + ∑o Vo) - W
z(τ)
τ∑^
Yκιtμz(ι))
~(
∑z|γ
κ,,μz(
l) +
+ ∑o Vo) +μ
z(l)
τ∑
zl
Yκtμz(τ)+μϊ∑o
1μo}]P(T,\W
k) (72)
Now substituting (72) into (68), and writing the exponent in the form of a quadratic in y, provides: - W?
κΣ
Y Xκμ
Yκ)
+x
T∑
x i κ£ +
+ ∑rjVo)
N
T+1 = aβ ∑ exp { - - [(y
κ - A BS)
TMVK ~ A
3 XB
3) - B^A^B
3
+x
τ∑
x ϊ κx +
+ ∑rj Vo) +μ
z(i)
T∑
zl
Yκ μz(i) + μrT∑o Vo] }P(T,\W
k) (73)
The belief function in equation (73) is a Gaussian mixture with the means and covariances of the summands given by Y κ, \ι A^B3 (74) JYκ \i = (75)
where / = 1 , ... , Nτ+ 1 , and so the mean and covariance of such a mixture is given by:
where
Any node within a Bayesian network can be instantiated with evidence. Consider the case where it becomes known that the variable YK has a value yκ~ This piece of information can be injected as evidence eγ ~ v by inserting an auxiliary child node V] that directs this evidence backwards towards node YK. Similarly, child nodes 2 and V3 can be inserted to direct evidence el ~ v and eγ ~ v towards nodes Z and Y , respectively.
For example, using selective marginalisation and taking structural information into account,
Bel(yχ) = ∑ / Bel(xκ,XD,yκ,yD,z,υ-ι,V2,υ3)dxκdzdυιdυ2
= ∑ / ap{eγ
κv^)p(
vAyκ)p{yκ\xκ)p{e
ZVi\v2)v{v2\z)v{z\yι<,yD) y ,
D,v
3 Jxκ,z,υ
l,<'
2,v
3 P(
eγ
Dv,\
v3)p{v \yD)p(yD\xD)p( κ\e
Xκ:γ
κ)p( D\e
Xl,γ
D dxκdzdυιdv2 = oιp{eγ
κVi\y
κ) ∑ p(e
ZV2\z)p(ey
nV3\y
D)p(z\yκ,y
D)p(y
κ\xκ)p(y
D\x
D)p(xκ\e
Xκγκ)p(x
D\e
XDyD)dzdx
κ VD,XD
Jz<XK =
aP(
eΫ
κv
1\yκ) / P{ezv
z)P(
z\yκ
<yD)p(yκ\xκ)p(xκ\e
XκYκ)p(eγ
DV y
D)p(y
D\e
XDYD)dzdx
κ Jz'
x>< =
αP(
eΫ
κv, \Vκ) ∑ I p(ez
V,\z)p(z\y
κ,y
D)p(yD\eγ
DV3,e
XDYD)p(yκ\xκ)p{xκ\e
XκYκ)dzdx
κ VD
•''•
X'< =
αP(
ey
κ il2κ)∑P(2/D|ey
Dv
3,eJ
oyi:)) VD
Hence the Belief function for v
# can be written as:
P(xκ, xp,yκ, yp, z, vι,v , υ
3, e
XκYκ .
ex
Dγ
Ω<
eγ
κ v
1 »
eΫ
κv^
eΫ
κv
3) P(
eX
K YK '
β D YD '
βYκ Vi '
βY
κ V
2 '
eY
K V
3 ) ~
av{
eγ
κv ^) i.
υ yκ) {yκ\xκ)pezv
2 v2)p('"'≥\ ) {z\y
κ,y
D) P(
eΫ
Dv
3\
v3)p(v3\yD)p(yD\xD)p(xκ\e
XκYκ)p(x
D\e
XDYD)
Equation (81) follows from (80) because the evidence at node YD is known with probability 1 and therefore forces its value onto the node variable. The second term in equation (80) drops out because evaluation of the belief function fory
κ implies that no hard evidence is available foτ κ; otherwise, the node variable takes on the value of the hard evidence.
Similarly, for node Z:
Bel(z) = Bel(x
κ, XD,yκ,yD,z,vι,υ ,v
3)dxκ yκdυ-
Ldυ
2
(
eΫ
Dv
3\
υ3p
υ3\yD)pyD\xD)p(xκ\e
XκYκ)px
D\ %
' DYD)dxκdyκdv
1dυ2
= P(
eΫ
DV3\yD) {yD\
XDYD)p(e^
vΛz) / p(eγ
κVl\yκ)p(yκ\xκ)p(xκ\e^
κYκ)p(z\y
κ,y
D)dx
κdy
κ vo J
χκ,yκ
= )dx
κdy
κ
VD JXK
= ∑ y
D\eγ
Dy3,e
XDYD) {ez
V2\z)p(z\eγ
κVι,y
D) (83)
As above, equation (84) follows from (83) because the evidence at node YD is dominant and therefore forces its value onto the node variable. Furthermore, it is assumed that the evidence at node Z does not exist; otherwise, it takes on the value of the evidence.
Similarly, for node YD,
Belførj) = ∑ / Be\(x
κ, XD, yκ, yD, z, vι, V2, v
3)dx
κdyκdυιdυ2
XD <V3 JXK ,VK ,Z,V1 ,V2 =
κdyκdvιdυ2 =
p(yD\xD)p(xκ\e
XκYκ)p(x
D\e
XDYD)dzdx
κdyκ = p(e
Vl |2/κ)p(ez
V- )p(y
A-|a;
A:)p(z|2
Λr,2 )p(x
Λ:|eJ-
κy
Λ.)rfzr α;
Λ:dy
Λ-
- z)dzdx
κdy
κ y
D)dx
κdy
κ
Again equation (86) follows from (85) because it is assumed that evidence at YD does not exist; otherwise, o takes on the value of the evidence. Clearly, Bel(yD) does not depend on evidence at node XK.
The threat assessment process described above generates inferred values that do not require further human inteφretation to estimate threat, due to the inclusion of the relevant inferential criteria, including those previously reserved for 'human judgement,' in the process via the criteria based causal networks of Figures 1 and 7. In comparison to prior art processes based on junction tree methods, the threat assessment process is able to generate inferred values using both continuous and mixed nodes with greatly reduced computational complexity, particularly in back-propagation of evidence. The functions representing causal relationships can be discontinuous, as in the military applications described herein, and the distributions of input variables do not have to be Gaussian distributions.
For each potential threat observed in an operational space, the threat relationships are represented by a full Bayesian belief network that is instantiated in real time with data from a real-time data base (in the above defence scenario, populated by a level 1 tracking and data fusion system). All relationships within the operational space are represented dynamically in real-time. The processing load associated with the threat assessment process scales linearly with the number of threatening entities and assets.
EXAMPLE
A level 1 fusion surveillance picture was generated from data supplied by a network of sensors, trackers, and data fusion processes, as described in Okello 1998 and Okello2000. This level 1 fusion surveillance picture was then used as input to the threat assessment system.
Figure 8 shows the actual movements or 'ground-truths' in two dimensions of two intruders (targets 1 and 2) of known type in the vicinity of two stationary assets 802, 804. The intruder of type 1 has a constant speed of 600 km/hr and maintains a constant altitude of 10000 m over a spherical earth while continuously emitting a signal that categorizes it as a type 1 target. The intruder of type 3 has a constant speed of 1000 km/hr and maintains a constant altitude of 9000 m while continuously emitting a signal that categorizes it as a type 3 target. This provides a simple multi-intruder multi-asset threat scenario. The numbers on the intruder ground-truths are target birth and death times in seconds measured from radar and electronic support measure (ESM) activation time. The target type information is related through a lookup table to the type of weapon carried by a platform of the identified type.
The scenario of Figure 8 was used to generate radar and ESM measurements at separate locations within the surveillance area. These were then processed by local trackers and the resulting sensor-level tracks were then fused to obtain a surveillance picture in which each tracked entity is comprehensively described in terms of its continuous kinematic and discrete type estimates. Figure 9 shows the resulting track estimates generated by a Cartesian-based
multitarget IMM-tracker that processes measurements from Radar 1. The Figure shows a first track 900 taken by one aircraft referred to as "target 1 ", and a second path taken by the other aircraft, referred to as "target 2". Figure 10 is a graph of the corresponding height estimates, with a first solid line 1002 representing the height or altitude of target 1 estimated at around 9,000 m, and a second solid line 1004 representing the height of target 2, estimated at around 10,000 m. Figure 11 is a graph of the corresponding speed estimates with the speed 1102 of target 1 estimated at 1,000 km h"1, and the speed 1104 of target 2 estimated at around 600 km h"1. Figures 12 and 13 are graphs of the corresponding location and speed variances for targets 1 and 2, respectively.
The discrete component of each ESM measurement is a vector of aircraft type probabilities with components given by V(Tp\w(k)), p = 1, ..., Nr , where N7- = 3 is the number of aircraft types in the ESM library. Aircraft of unknown types are lumped under target type probability
P(7V
r+l|w(&))
. In this example, target 1 is of type 1 and the ESM type measurements originating from this target are generated from a Dirichlet distribution with the parameter vector α = [5 2 2 2], Similarly, target 2 is of type 3 and the ESM type measurements originating from this target are generated from a Dirichlet distribution with parameter vector α = [2 2 5 2]. Furthermore, any clutter measurement is generated based on the parameter vector α = [2 2 2 5]. Thus while the kinematic component of the ESM are processed using the modified polar coordinates-based bearings-only tracker, these discrete type measurements are processed using a Bayesian filter. The output of such a filter is a vector of type probabilities with components given by P(T
p\Jf^), p = 1, ..., Nτ+ 1 where W is the set of all measurements up to time k. We assume that track-to-measurement association and track-to-track association are possible. Figures 10 and 11 show target type probabilities for tracks 1 and 4, respectively. These plots were generated by a Bayesian filter following the processing of ESM type measurements, as described in Okello 1998 and Okello2000. It was assumed that the target types differ only in the size of their weapon envelopes, and that the target types 77, T2, 7}, and T
4 have semi-circular weapon envelopes with radii of 50 km, 40 km, 60 km, and 2 km, respectively.
For comparison purposes, the same set of level 1 data described above was independently processed by the belief propagation process of the belief propagation module 314 and the direct integration process of the selective marginalisation module 312. Figures 16 and 17 show numerical results generated by the belief propagation process. However, identical results (not shown) were generated by the direct integration process.
Figure 16 includes four graphs of the two components of the intermediate node y for each of the two possible target types, as determined from the level 1 tracking data and fusion input data of Figures 9 to 15. The top left graph 1602 presents, as a function of time step k, the mean value of the first component of the intermediate variable y (refer to Equations 5 and 8), being the length of the intruder-to-asset range vector ry for the first intruder target 1 , and the top right graph 1604 presents the second component of y, being the angle between the intruder velocity vector and the intruder-to-asset range vector rυ. The bottom left graph 1606 and bottom right graph 1608 are equivalent graphs for the second intruder, target 2. Figure 17 includes four graphs showing the mean and covariances of the threat to each of the two stationary assets 802,804 . The top left graph shows that the threat 1702 to the first asset rises to a first peak 1700 having a value of around 0.5 near time step 80 as a consequence of target 1 approaching asset 1 , the threat 1702 rapidly decreasing as target 1 passes asset 1. A second peak 1704 at around time step 500 is due to target 2 approaching and then passing asset 1. Similarly, the bottom left graph shows that the threat 1706 to asset 2 is initially high due to the close proximity and orientation of target 1, rapidly decreasing as target 1 passes asset 2. The threat due to the approach of target 2 increases gradually to a peak at around time step 300 due to the approach of target 2, and rapidly decreases to zero as target 2 heads away from asset 2.
The threat assessment process thus allows the threat posed by one or more threatening entities to one or more assets to be automatically assessed in real-time without requiring human judgement or involvement. The threat values thus generated by the threat assessment system can be used to prepare a suitable response to these threats.
The observation that the belief propagation process and the direct integration process give identical results suggests that these processes are sufficiently versatile to handle a wide range of complex problems. In particular, these processes can be used to solve Bayesian network problems having a mixture of continuous and discrete nodes; in cases where the continuous nodes are not Gaussian, conservative results based on a Gaussian approximation are easily obtainable. Furthermore, it has been demonstrated that non-linearities and discontinuities in the conditional dependence between connected nodes is not an obstacle when using any of the processes described herein.
Many modifications will be apparent to those skilled in the art without departing from the scope of the present invention as herein described with reference to the accompanying drawings.