US8346711B2 - Method for identifying multi-input multi-output Hammerstein models - Google Patents

Method for identifying multi-input multi-output Hammerstein models Download PDF

Info

Publication number
US8346711B2
US8346711B2 US12/591,603 US59160309A US8346711B2 US 8346711 B2 US8346711 B2 US 8346711B2 US 59160309 A US59160309 A US 59160309A US 8346711 B2 US8346711 B2 US 8346711B2
Authority
US
United States
Prior art keywords
processor
output
hammerstein
instructions
set
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US12/591,603
Other versions
US20110125684A1 (en
Inventor
Hussain N. Al-Duwaish
Syed Z. Rizvi
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
King Fahd University of Petroleum and Minerals
Original Assignee
King Fahd University of Petroleum and Minerals
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by King Fahd University of Petroleum and Minerals filed Critical King Fahd University of Petroleum and Minerals
Priority to US12/591,603 priority Critical patent/US8346711B2/en
Assigned to KING FAHD UNIV. OF PETROLEUM & MINERALS reassignment KING FAHD UNIV. OF PETROLEUM & MINERALS ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: AL-DUWAISH, HUSSAIN N., RIZVI, SYED Z.
Publication of US20110125684A1 publication Critical patent/US20110125684A1/en
Application granted granted Critical
Publication of US8346711B2 publication Critical patent/US8346711B2/en
Application status is Active legal-status Critical
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computer systems based on biological models
    • G06N3/02Computer systems based on biological models using neural network models
    • G06N3/04Architectures, e.g. interconnection topology
    • G06N3/0481Non-linear activation functions, e.g. sigmoids, thresholds

Abstract

The method for the identifying of multiple input, multiple output (MIMO) Hammerstein models that includes modeling of the linear dynamic part of a Hammerstein model with a state-space model, and modeling the nonlinear part of the Hammerstein model with a radial basis function neural network (RBFNN).

Description

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to Hammerstein models, and particularly to a computerized method for identifying multi-input multi-output (MIMO) Hammerstein models for engineering design applications.

2. Description of the Related Art

The Hammerstein Model belongs to a family of block oriented models, and is made up of a memoryless nonlinear part followed by a linear dynamic part. It has been known to effectively represent and approximate several industrial processes, such as, for example, pH neutralization processes, distillation column processes, and heat exchange processes. Hammerstein models have also been used to successfully model nonlinear filters, biological systems, water heaters, and electrical drives.

A significant amount of research has been carried out on identification of Hammerstein models. Systems can be modeled by employing either nonparametric or parametric models. Nonparametric representations involve kernel regression or expansion of series, such as the Volterra series. This results in a theoretically infinite number of model parameters, and is therefore represented in terms of curves, such as step responses or bode diagrams. Parametric representations, such as state-space models, are more compact, as they have fewer parameters and the nonlinearity is expressed as a linear combination of finite and known functions.

Development of nonlinear models is the critical step in the application of nonlinear model based control strategies. Nonlinear behavior is prominent in the dynamic behavior of physical systems. Most physical devices have nonlinear characteristics outside a limited linear range. In most chemical processes, for example, understanding the nonlinear characteristics is important for designing controllers that regulate the process. It is rather difficult, yet necessary, to select a reasonable structure for the nonlinear model to capture the process nonlinearities. The nonlinear model used for control purposes should be as simple as possible, warranting minimal computational load and, at the same time, retaining most of the nonlinear dynamic characteristics of the system. The following convention has been used in what follows: upper case variables in bold represent matrices, lower case bold variables represent vectors, and lower case regular (i.e., non-bold) variables represent scalar quantities.

Many model structures have been proposed for the identification of nonlinear systems. The nonlinear static block followed by a dynamic block in the Hammerstein structure has been found to be a simple and effective representation for capturing the dynamics of typical chemical engineering processes such as distillation columns and heat exchangers, for example. Nonlinear system identification involves the following tasks: Structure selection, including selection of suitable nonlinear model structures and the number of model parameters; input sequence design, including the determination of the input sequence u(t) which is injected into the system to generate the output sequence y(t); noise modeling, which includes the determination of the dynamic model which generates the noise input w(t); parameter estimation, which includes estimation of the remaining model parameters from the dynamic system data u(t) and y(t), and the noise input w(t); and model validation, including the comparison of system data and model predictions for data not used in model development.

Hammerstein systems can be modeled by employing either nonparametric or parametric models. Nonparametric models represent the system in terms of curves resulting from expansion of series, such as the Volterra series or kernel regression. In practice, these curves are sampled, often leading to a large number of parameters. Parametric representations, such as state-space models, are more compact and have fewer parameters, while the nonlinearity is expressed as a linear combination of finite and known functions. In parametric identification, the Hammerstein model is represented by the following set of equations:
y(t)=−a 1 y(t−1)− . . . −a n y(t−n)+b o v(t)+ . . . +b m v(t−m)  (1)
v(t)=c 1 u(t)+c 2 u 2(t)+ . . . +c L u L(t)  (2)
where v(t) describes the nonlinearity, L is the order of the nonlinearity, and y(t) and u(t) are the outputs and inputs of the system.

In MIMO Hammerstein models, as noted above, a nonlinear system is represented as a nonlinear memory-less subsystem f(.), followed by a linear dynamic part. The input sequence u(t) and the output sequence y(t) are accessible to measurements, but the intermediate signal sequence v(t) is not. As shown in FIG. 1B, the static nonlinear element scales the inputs u(t) and transforms these inputs to v(t) through a nonlinear arbitrary function f(u). The dynamics of the system are modeled by a linear transfer function, whose outputs are y(t).

Many different techniques have been proposed for the black-box estimation of Hammerstein systems from input-output measurements. These techniques mainly differ in the way that static nonlinearity is represented and in the type of optimization problem that is finally obtained. In parametric approaches, the static nonlinearity is expressed in a finite number of parameters. Both iterative and non-iterative methods have been used for determination of the parameters of the static-nonlinear and linear-dynamic parts of the model. Typical techniques, however, are extremely costly in terms of computational time and energy.

Additionally, most techniques designed to deal with Hammerstein models focus purely on single-input single-output (SISO) models. Identification of MIMO systems, however, is a problem which has not been well explored. Identification based on prediction error methods (PEM), for example, is a complicated function of the system parameters, and has to be solved by iterative descent methods, which may get stuck into local minima. Further, optimization methods need an initial estimate for a canonical parametrization model; i.e. models with minimal numbers of parameters, which might not be easy to provide.

It has been shown that this minimal parametrization can lead to several problems. PEM have, therefore, inherent difficulties with MIMO system identification. More recent studies have also shown that maximum likelihood criterion results in a non-convex optimization problem in which global optimization is not guaranteed. Subspace identification methods (SIM) do not need nonlinear optimization techniques, nor do these methods need to impose to the system a canonical form. Subspace methods therefore do not suffer from the inconveniences encountered in applying PEM methods to MIMO system identification. Thus, it would be desirable to make use of this advantage, modeling the linear dynamic subsystem of the Hammerstein model with a state-space model rather than polynomial models. Thus, a method for identifying multi-input multi-output Hammerstein models solving the aforementioned problems is desired.

SUMMARY OF THE INVENTION

The present invention relates to the identification of multi-input multi-output (MIMO) Hammerstein models and, in particular, to a method utilizing a particle swarm optimization (PSO) subspace algorithm. The inventive method includes modeling of the linear dynamic part of a Hammerstein model with a state-space model, and modeling the nonlinear part of the Hammerstein model with a radial basis function neural network (RBFNN). Accurate identification of a Hammerstein model requires that output error between the actual and estimated systems be minimized, thus the problem of identification is, in essence, an optimization problem. The PSO-subspace algorithm of the present method is an optimization algorithm. Particle swarm optimization (PSO), typically known for its heuristic search capabilities, is used for estimating the parameters of the RBFNN. Parameters of the linear part are estimated using a numerical algorithm for subspace state-space system identification (N4SID).

The present method includes the following steps: (a) estimating an initial set of state space matrices A, B, C, and D using nonlinear data acquired from the plant (these matrices are estimated using the methods of subspace identification); (b) initializing a swarm of particles with a random population of possible radial basis function neural network weights; (c) initializing a particle swarm optimization method with a random population of possible radial basis function neural network weights; (d) calculating a global best set of weights which minimizes an output error measure; (e) estimating sets of radial basis function neural network outputs v(t) for all values of t based upon the global best set of weights; (f) estimating the state-space matrices A, B, C, and D from the radial basis function neural network outputs estimated in step (e) and a set of original system outputs y(t) for values of t; (g) calculating a set of system outputs ŷ(t) from the estimated state-space matrices A, B, C, and D in step (f); (h) calculating the output error measure; and (i) repeating steps (c) to (h) if the calculated output error measure is greater than a pre-selected threshhold error measure.

These and other features of the present invention will become readily apparent upon further review of the following specification and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a diagrammatic representation of a Hammerstein model with separate nonlinearities.

FIG. 1B is a diagrammatic representation of a Hammerstein model with combined nonlinearities.

FIG. 2 is a diagrammatic overview of a general Hammerstein model.

FIG. 3 is a diagrammatic representation of a radial basis function neural network.

FIG. 4 is a flow chart illustrating the method for identifying multi-input multi-output Hammerstein models according to the present invention.

FIG. 5 is a graph illustrating a first nonlinearity of a Hammerstein model associated with a first example of the method for identifying multi-input multi-output Hammerstein models according to the present invention.

FIG. 6 is a graph illustrating a second nonlinearity of a Hammerstein model associated with a first example of the method for identifying multi-input multi-output Hammerstein models according to the present invention.

FIG. 7 is a graph illustrating a step response of a linear subsystem of the Hammerstein model associated with the first example of the method for identifying multi-input multi-output Hammerstein models according to the present invention, specifically identifying a first output.

FIG. 8 is a graph illustrating a step response of a linear subsystem of the Hammerstein model associated with the first example of the method for identifying multi-input multi-output Hammerstein models according to the present invention, specifically identifying a second output.

FIG. 9 is a graph illustrating a complete step response of the Hammerstein model associated with the first example of the method for identifying multi-input multi-output Hammerstein models according to the present invention.

FIG. 10 is a graph illustrating a mean squared output error of the Hammerstein model associated with the first example of the method for identifying multi-input multi-output Hammerstein models according to the present invention.

FIG. 11 is a graph illustrating a first nonlinearity of a Hammerstein model associated with a second example of the method for identifying multi-input multi-output Hammerstein models according to the present invention.

FIG. 12 is a graph illustrating a second nonlinearity of a Hammerstein model associated with the second example of the method for identifying multi-input multi-output Hammerstein models according to the present invention.

FIG. 13 is a graph illustrating a step response of a linear subsystem of the Hammerstein model associated with the second example of the method for identifying multi-input multi-output Hammerstein models according to the present invention, specifically identifying a first output.

FIG. 14 is a graph illustrating a step response of a linear subsystem of the Hammerstein model associated with the second example of the method for identifying multi-input multi-output Hammerstein models according to the present invention, specifically identifying a second output.

FIG. 15 is a graph illustrating a complete step response of the Hammerstein model associated with the second example of the method for identifying multi-input multi-output Hammerstein models according to the present invention.

FIG. 16 is a graph illustrating a mean squared output error of the Hammerstein model associated with the second example of the method for identifying multi-input multi-output Hammerstein models according to the present invention.

FIG. 17 is a block diagram of a system for implementing the method for identifying multi-input multi-output Hammerstein models according to the present invention.

Similar reference characters denote corresponding features consistently throughout the attached drawings.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention relates to the identification of MIMO Hammerstein models and, in particular, to a method utilizing a particle swarm optimization (PSO) subspace algorithm. The method includes modeling of the linear dynamic part of a Hammerstein model with a state-space model, and modeling the nonlinear part of the Hammerstein model with a radial basis function neural network (RBFNN). Accurate identification of a Hammerstein model requires that output error between the actual and estimated systems be minimized, thus the problem of identification is, in essence, an optimization problem. The PSO-subspace algorithm of the present method is an optimization algorithm. Particle swarm optimization (PSO), typically known for its heuristic search capabilities, is used for estimating the parameters of the RBFNN. Parameters of the linear part are estimated using a numerical algorithm for subspace state-space system identification (N4SID).

As noted above, the linear dynamic part of the Hammerstein model is modeled by a state-space model. The static nonlinear part of the Hammerstein model is modeled using RBFNN. The external inputs to the system u(t) are fed to the RBFNN, as best illustrated in FIG. 3. The output generated by RBFNN v(t) acts as the input to the state-space system, which translates the inputs to system outputs y(t). The intermediate data between the non-linear and linear parts is inaccessible to measurements.

Particularly, considering a MIMO Hammerstein system with p inputs and r outputs, the input layer of the RBFNN takes the system inputs u(t)=[u1(t) . . . up(t)]T and the second layer of the RBFNN, referred to as the “hidden layer”, performs a fixed nonlinear transformation on this data using the basis function φ. The output layer of the RBFNN then implements a linear weightage and supplies the response of the network to the output v(t)=[v1(t) . . . vp(t)]T. Thus, the only adjustable parameters in the RBFNN are the weights of its output layer. This output, v(t) is fed to the linear subsystem whose output is given by:
x(t+1)=Ax(t)+Bv(t)+s(t)  (3a)
ŷ(t)=Cx(t)+Dv(t)+z(t)  (3b)
where v(t)ε

Figure US08346711-20130101-P00001
P×1 and Ŷ(t)ε
Figure US08346711-20130101-P00001
R×1 are the vectors for p inputs and r outputs of the linear subsystem observed at discrete time instant t. Vector z(t)ε
Figure US08346711-20130101-P00001
R×1 is called the measurement noise and s(t)ε
Figure US08346711-20130101-P00001
n×1 is called the process noise, where n denotes the order of the linear subsystem. Both z(t) and s(t) are zero mean, white noise sequences, which have covariance matrices given by:

E [ ( s z ) ( s T z T ) ] = [ Q S S T r ] δ pq ( 4 )
where E denotes expected value and δpq denotes a Kronecker delta function.

Accurate identification of a Hammerstein model requires that the error between the outputs of the actual and estimated systems be minimized. If y(t)=[y1(t)]T denotes the vector for original outputs of the sampled data, and ŷ(t)=[ŷ1(t) . . . ŷr(t)]T denotes the vector for the outputs of the estimated system, then a cost function based on the square of the output error is sought to be minimized, with the cost function being given by:

I = t = 1 N e T ( t ) e ( t ) ( 5 )
where N denotes the number of data points, e(t)=[e1(t) . . . er(t)]T is the vector for output error at discrete time instant t, and is given by e(t)=Y(t)−Ŷ(t).

The PSO plays a large part in training the RBFNN. In a swarm of particles, where each particle represents a candidate value for the weight of RBFNN output layer, the fitness of the particles is the reciprocal of the cost index given in equation (5). Hence, the smaller the sum of output errors, the more fit are the particles. Based on this principle, the PSO updates the position of all the particles moving towards an optimal solution for the weights of RBFNN.

Hammerstein identification is, therefore, solved as an optimization problem in which PSO is used to estimate the parameters of RBFNN, while parameters of the linear subsystem are estimated using the N4SID numerical subspace algorithm. RBFNN is an effective type of neural network that has proved useful in applications such as function approximation and pattern recognition. It should be noted that the static nonlinearity in a MIMO Hammerstein model can be either combined or separate.

FIGS. 1A and 1B show two Hammerstein models, both having p inputs and r outputs. In the system of FIG. 1A, each input to the linear subsystem is a function of a separate nonlinear function. Every nonlinearity is therefore approximated by a separate RBF network. FIG. 3 illustrates an RBFNN with three layers having Q neurons in the hidden layer. The input layer connects the network to its environment. The second layer performs a fixed nonlinear transformation using basis functions. The output layer linearly weighs the response of the network to the output. Thus, the jth nonlinear output vj(t) would depend only on uj(t), and would be modeled by the RBFNN as:

v j ( t ) = i = 1 Q w i ϕ u j ( t ) - c i ( 6 )
where Q is the number of neurons in the hidden layer, ci is the center for the ith neuron of that layer, wi is the weight connecting the ith neuron node to the output layer, φ is the radial basis function, and ∥.∥ denotes the norm.

In the second system of FIG. 1B, the nonlinearity is combined, and therefore the effect of all the inputs is reflected over every nonlinear output vj(t). For a similar network having Q neurons in the hidden layer, the equation for the jth nonlinear output vj(t) in this case is given by:

v j ( t ) = i = 1 Q w ij ϕ u ( t ) - c i ( 7 )
where u(t)ε

Figure US08346711-20130101-P00001
px1 is the input vector for the system shown in FIG. 1B at discrete time t.

Subspace identification is used for estimating the parameters of the linear dynamic part of the model; i.e., the matrices of the state-space model. The present inventive method makes use of the N4SID numerical algorithm. The objective of the algorithm is to determine the order n of the system, and the system matrices Aε

Figure US08346711-20130101-P00001
n×n, Bε
Figure US08346711-20130101-P00001
n×P, Cε
Figure US08346711-20130101-P00001
R×n, D ε
Figure US08346711-20130101-P00001
R×P, Qε
Figure US08346711-20130101-P00001
n×n, Rε
Figure US08346711-20130101-P00001
R×R, and Sε
Figure US08346711-20130101-P00001
n×R (and the Kalman gain matrix K if required), without any prior knowledge of the structure of the system. This is achieved in two main steps: First, model order n and a Kalman filter state sequence of estimates {circumflex over (x)}i,{circumflex over (x)}i+1, . . . ,{circumflex over (x)}i+j are determined by projecting row spaces of data block Hankel matrices, and then applying a singular value decomposition. Next, the solution of a least squares problem is used to obtain the state space matrices A, B, C, and D.

PSO is a heuristic search-based optimzation technique which exhibits behavior of swarm intelligence. PSO differs from other evolutionary algorithms (EAs) in that it changes its population from one iteration to the next. Unlike genetic algorithms (GAs) and other EAs, operators like selection, mutation and crossover are not used to change the population. Existing particles are, instead, modified according to a pair of formulae. PSO thus differs from other EAs in terms of performance. PSO is more robust and faster in comparison, and also provides solutions to complex and nonlinear problems, with relative ease in finding a global minimum. The convergence characteristic of PSO is relatively stable and provides solutions in relatively little time.

PSO begins with a population of particles. Each particle adjusts its position according to its own experience, as well as by the experience of neighboring particles. Each particle is treated as a point in D-dimensional space. The ith particle is represented as:
X i=(x i1 ,x i2 , . . . ,x iD)  (8)
and the best positions of the particles (the position giving the most optimum solution) are recorded as:
P i=(p i1 ,p i2 , . . . ,p iD)  (9)
with the change in position (velocity) of each particle being given as:
V i=(v i1 ,v i2 , . . . ,v iD)  (10)
where the velocity and positions of the particles are updated according to the following pair of equations:
V i n+1 =V i n +c 1 *r i1 n*(P i n −X i n)+c 2 *r i2 n*(P g n −X i n)  (11a)
X i n+1 =X i n +x*V i n+1  (11b)
where c1 and c2 are two positive real constants, the cognitive parameter and the social parameter, respectively. The value of c1 signifies how much a particle trusts its past experiences, and how much it searches for solutions around the local best position, while the value of c2 determines the swarm's attraction toward a global best position.

Higher values of c1 and c2 make the swarm able to react faster to changes, whereas lower values of these parameters make the swarm slower in changing neighborhoods of solutions. The present inventive method makes use of values such that c1≧c2, and c1+c2≦4.

Introducing an inertia weight w and a constriction factor X, the update equations become:
V i n+1 =X(wV i n +c 1 *r i1 n*(P i n −X i n)+c 2 *r i2 n*(P g n −X i n))  (12a)
X i n+1 =X i n +x*V i n+1  (12b)
where w is the inertial weight, and X is the constriction factor, which is used to limit the velocity and help better convergence. The value of c1 signifies how much a particle trusts its past experiences, and how much it is attracted to a local best position while the value of c2 determines the swarm's attraction towards a global best position.

The method relates to Hammerstein model identification, which, in essence, can be summarized as follows: Given a set of N noisy inputs u(t)t=0 N−1 and outputs y(t)t=0 N−1, find the weights of the RBFNN; and find the matrices of the state space model. Since the output y(t) is nonlinear in relation to the input u(t), the calculations are nontrivial. Thus, a recursive algorithm is utilized to update the weights of the neural network for each set of input and output data.

As illustrated in FIG. 4, the method includes the following steps: (1) Estimation of state-space matrices A, B, C and D from original nonlinear data using N4SID subspace identification (i.e., an initial estimate) at step 10; (2) initialization of PSO with random population of possible RBFNN weights (step 12); (3) obtaining a global best set of weights which minimizes the cost index given in equation (5) at step 14; (4) estimation of sets RBFNN outputs v(t) for all values of t once optimum weights are obtained (step 16); (5) estimation of state-space matrices A, B, C, and D from the new neural network outputs estimated in step (4) and sets of original system outputs y(t) for all values of t (this estimate of state-space model is an improvement on the previous estimate); (6) regeneration of sets of output ŷ(t) for all values of t from the new estimate of the complete system; and (7) repetition of steps (2) to (6) if the minimum goal for cost is not reached at step 22.

In a first example, a two input, two output Hammerstein type nonlinear process with two separate static nonlinearities is considered. The first nonlinearity is a saturation nonlinearity, while the second nonlinearity is a tan h/exponential function:

v 1 ( t ) = { - 0.5 if - 0.5 u 1 ( t ) u 1 ( t ) if 0.5 > u 1 ( t ) > - 0.5 0.5 if u 1 ( t ) > 0.5 ( 13 a ) v 2 ( t ) = tanh [ 2 u 2 ( t ) ] 1.5 u ( t ) ( 13 b ) v 2 ( t ) = exp ( u 2 ( t ) ) - 1 exp ( u 2 ( t ) ) + 1 4 > u ( t ) > 1.5 ( 13 c )
with the dynamic linear part being given by the following second-order discrete time state-space system:

[ x 1 ( t + 1 ) x 2 ( t + 1 ) ] = [ 1 1 - 0.5 0 ] [ x 1 ( t ) x 2 ( t ) ] [ 1 0.5 ] v ( t ) y ( t ) = [ 1 0 ] [ x 1 ( t ) x 2 ( t ) ]
where the linear part of the system has eigen values at λ1.2=0.5±0.5i.

The system is identified using a PSO/subspace algorithm. A set of persistently exciting data is generated in the intervals [−1,1] and [0,4] for the two inputs, respectively. The centers of the RBFNNs are initialized with a set of evenly distributed centers in their respective intervals. The PSO/subspace algorithm identifies the system within a few iterations of the algorithm and the mean squared error converges to a final value of 9×10−4. FIGS. 5, 6, 7 and 8 show the estimates of nonlinearities and the step response of the dynamic linear subsystem. FIG. 9 shows the step response of the complete Hammerstein system. The results show that the shape of the nonlinearies have been accurately estimated. The eigenvalues of the estimated linear part are λ1=0.4970±0.4993i.

In a second example of the method, the Hammerstein type nonlinear process includes a static nonlinearity given by:

v 1 ( t ) sin ( u ( t ) ) u ( t ) ( 14 a ) v 2 ( t ) = arctan ( u 2 1.4 ) ( 14 b )
with the dynamic linear part being given by a third-order discrete time state-space system:

[ x 1 ( t + 1 ) x 2 ( t + 1 ) x 3 ( t + 1 ) ] = [ 1.80 1 0 - 1.07 0 1 0.21 0 0 ] [ x 1 ( t ) x 2 ( t ) x 3 ( t ) ] + [ 4.80 1.93 1.21 ] v ( t ) ( 15 a ) y ( t ) = [ 1 0 0 0 1 0.5 ] [ x 1 ( t ) x 2 ( t ) x 3 ( t ) ] ( 15 b )
where, for this example, an RBF network of ten neurons is initialized with centers evenly distributed in the interval [−1.75,1.75]. The linear part of the system has eigen values at λ1=0.7, λ2=0.6, and λ3=0.5.

Persistently exciting data is generated in the intervals [−1.75,1.75] and [−4,4] for the two inputs, respectively. The centers of the RBFNNs are initialized evenly distributed within the input intervals. The PSO/subspace method identifies the system with a good estimate of the nonlinearity shape. The mean squared error converges to a final value of 7×10−4. FIGS. 11, 12, 13, and 14 show the estimates of nonlinearities and the step responses of the linear subsystem. The eigenvalues of the estimated linear subsystem are λ1=0.6997, λ2=0.5952, and λ3=0.4949.

As seen from the plots, the shapes of the nonlinearities have been estimated quite accurately. The method combines the advantages of PSO with those of state-space models. PSO is well know to outperform other EAs in finding global optima, and state-space models can more easily be extended to systems with multiple inputs and outputs, as compared to polynomial models. FIG. 15 illustrates the step response of the complete system of the second example, and FIG. 16 illustrates the mean squared output error for the Hammerstein system of the second example.

In the above, the calculations may be performed by any suitable computer system, such as that diagrammatically shown in FIG. 17. Data is entered into system 100 via any suitable type of user interface 116, and may be stored in memory 112, which may be any suitable type of computer readable and programmable memory. Calculations are performed by processor 114, which may be any suitable type of computer processor and may be displayed to the user on display 118, which may be any suitable type of computer display.

Processor 114 may be associated with, or incorporated into, any suitable type of computing device, for example, a personal computer or a programmable logic controller. The display 118, the processor 114, the memory 112 and any associated computer readable recording media and/or communication transmission media are in communication with one another by any suitable type of data bus, as is well known in the art.

Examples of computer-readable recording media include a magnetic recording apparatus, an optical disk, a magneto-optical disk, and/or a semiconductor memory (for example, RAM, ROM, etc.). Examples of magnetic recording apparatus, which may be used in addition to memory 112, or in place of memory 112, include a hard disk device (HDD), a flexible disk (FD), and a magnetic tape (MT). Examples of the optical disk include a DVD (Digital Versatile Disc), a DVD-RAM, a CD-ROM (Compact Disc-Read Only Memory), and a CD-R (Recordable)/RW.

It is to be understood that the present invention is not limited to the embodiments described above, but encompasses any and all embodiments within the scope of the following claims.

Claims (7)

1. A computerized method for identifying multi-input multi-output Hammerstein models, comprising the steps of:
(a) providing an initial set of nonlinear Hammerstein system data acquired from a plant;
(b) estimating a set of state-space matrices A, B, C and D from the initial set of nonlinear Hammerstein system data;
(c) initializing a swarm of particles with a random population of possible radial basis function neural network weights;
(d) calculating a global best set of weights which minimizes an output error measure;
(e) estimating sets of radial basis function neural network outputs v(t) for all values of t based upon the global best set of weights;
(f) estimating the state-space matrices A, B, C and D from the radial basis function neural network outputs v(t) for all values of t, estimated in step (e), and sets of original system outputs y(t) for all values of t;
(g) calculating sets of system outputs ŷ(t) for all values of t from the estimated state-space matrices A, B, C and D of step (f);
(h) calculating the output error measure; and
(i) repeating steps (c) to (h) if the calculated output error measure is greater than a pre-selected threshold error measure.
2. The method for identifying multi-input multi-output Hammerstein models as recited in claim 1, wherein the set of state-space matrices are estimated by calculating x(t+1)=Ax(t)+Bv(t)+s(t) and ŷ(t)=Cx(t)+Dv(t)+z(t), wherein x(t) represents an input of the Hammerstein system and ŷ(t) represents an output of the Hammerstein system observed at a discrete time instant t, z(t) represents a measurement noise, and s(t) represents a process noise.
3. The method for identifying multi-input multi-output Hammerstein models as recited in claim 2, wherein the output error measure I is calculated as
I = t = 1 N e T ( t ) e ( t ) ,
wherein N represents a number of noisy inputs, and e(t) is a vector for output error at discrete time instant t, where e(t)=Y (t)−Ŷ(t).
4. The method for identifying multi-input multi-output Hammerstein models as recited in claim 3, wherein the radial basis function neural network outputs are calculated as
v ( t ) = i = 1 Q w i ϕ u ( t ) - c i ,
wherein ci represents a center for an ith neuron, wherein i is an integer, Q represents a number of neurons in a hidden layer of the neural network, wi represents a weight connecting the ith neuron node to an output layer, and φ represents a radial basis function.
5. The method for identifying multi-input multi-output Hammerstein models as recited in claim 4, wherein the particle swarm optimization method includes establishing a set of particles and identifying each particle in the set as a point in D-dimensional space, wherein the ith particle is represented as Xi=(xi1, xi2, . . . , xiD) and the most optimal positions of the particles are given by Pi=(pi1, pi2, . . . , piD) with a change in velocity of each of the particles being given by Vi=(vi1, vi2, . . . , viD), such that the velocity and position of each particle is updated as Vi n+1=w*Vi n+c1*ri1 n*(Pi n−Xi n)+c2*ri2 n*(Pg n−Xi n) and Xi n+1=Xi n+x*Vi n+1, wherein w is an inertial weight parameter for limiting velocity, and c1 and c2 represent a cognitive parameter and a social parameter, respectively.
6. A system for identifying Hammerstein models, comprising:
a processor;
computer readable memory coupled to the processor;
a user interface coupled to the processor;
a display coupled to the processor
software stored in the memory and executable by the processor, the software having:
means for estimating a set of state-space matrices A, B, C and D from an initial set of nonlinear Hammerstein system data acquired from a plant;
means for initializing a swarm of particles with a random population of possible radial basis function neural network weights;
means for calculating a global best set of weights which minimizes an output error measure;
means for estimating sets of radial basis function neural network outputs v(t) for all values of t based upon the global best set of weights;
means for estimating the state-space matrices A, B, C and D from the radial basis function neural network outputs v(t) for all values of t, and sets of original system outputs y(t) for all values of t;
means for calculating sets of system outputs ŷ(t) for all values of t from the estimated state-space matrices A, B, C and D; and
means for calculating the output error measure.
7. A computer software product that includes a non-transitory storage medium readable by a processor, the non-transitory storage medium having stored thereon a set of instructions for identifying Hammerstein models, the instructions comprising:
(a) a first sequence of instructions which, when executed by the processor, causes the processor to acquire an initial set of nonlinear system from a plant;
(b) a second sequence of instructions which, when executed by the processor, causes the processor to estimate a set of state-space matrices A, B, C and D from the initial set of nonlinear data using subspace identification;
(c) a third sequence of instructions which, when executed by the processor, causes the processor to initialize a swarm of particles with a random population of possible radial basis function neural network weights;
(d) a fourth sequence of instructions which, when executed by the processor, causes the processor to calculate a global best set of weights which minimizes an output error measure;
(e) a fifth sequence of instructions which, when executed by the processor, causes the processor to estimate sets of radial basis function neural network outputs v(t) for all values of t based upon the global best set of weights;
(f) a sixth sequence of instructions which, when executed by the processor, causes the processor to estimate the state-space matrices A, B, C and D from the radial basis function neural network outputs v(t) for all values of t, estimated in the fifth set of instructions, and sets of original system outputs y(t) for all values of t,
(g) a seventh sequence of instructions which, when executed by the processor, causes the processor to calculate sets of system outputs ŷ(t) for all values of t from the estimated state-space matrices A, B, C and D of the sixth sequence of instructions;
(h) an eighth sequence of instructions which, when executed by the processor, causes the processor to calculate the output error measure; and
(i) a ninth sequence of instructions which, when executed by the processor, causes the processor to repeat the third sequence of instructions of the eighth sequence of instructions if the calculated output error measure is greater than a pre-selected threshold error measure.
US12/591,603 2009-11-24 2009-11-24 Method for identifying multi-input multi-output Hammerstein models Active 2031-06-04 US8346711B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US12/591,603 US8346711B2 (en) 2009-11-24 2009-11-24 Method for identifying multi-input multi-output Hammerstein models

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US12/591,603 US8346711B2 (en) 2009-11-24 2009-11-24 Method for identifying multi-input multi-output Hammerstein models

Publications (2)

Publication Number Publication Date
US20110125684A1 US20110125684A1 (en) 2011-05-26
US8346711B2 true US8346711B2 (en) 2013-01-01

Family

ID=44062816

Family Applications (1)

Application Number Title Priority Date Filing Date
US12/591,603 Active 2031-06-04 US8346711B2 (en) 2009-11-24 2009-11-24 Method for identifying multi-input multi-output Hammerstein models

Country Status (1)

Country Link
US (1) US8346711B2 (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120259600A1 (en) * 2011-04-11 2012-10-11 King Fahd University Of Petroleum And Minerals Method of identifying hammerstein models with known nonlinearity structures using particle swarm optimization
CN104166883A (en) * 2014-08-05 2014-11-26 中国南方电网有限责任公司超高压输电公司检修试验中心 Power grid dispatching dynamic capacity increasing evaluation method based on state multi-dimension comprehensive evaluation
CN104915933A (en) * 2015-06-01 2015-09-16 长安大学 Foggy day image enhancing method based on APSO-BP coupling algorithm
US20170048088A1 (en) * 2015-08-12 2017-02-16 King Fahd University Of Petroleum And Minerals Apparatuses and methodologies for decision feedback equalization using particle swarm optimization
US9824133B1 (en) * 2014-06-20 2017-11-21 Amazon Technologies, Inc. Cloud analytics marketplace
US10162876B1 (en) 2014-06-20 2018-12-25 Amazon Technologies, Inc. Embeddable cloud analytics
US10430438B2 (en) 2014-06-20 2019-10-01 Amazon Technologies, Inc. Dynamic n-dimensional cubes for hosted analytics

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5268496B2 (en) * 2008-08-22 2013-08-21 株式会社東芝 Flow analysis method, flow analysis apparatus, and flow analysis program
US9645558B2 (en) * 2012-09-29 2017-05-09 Operation Technology, Inc. Dynamic parameter tuning using particle swarm optimization
CN102982393B (en) * 2012-11-09 2016-08-10 山东电力集团公司聊城供电公司 A kind of on-line prediction method of electric transmission line dynamic capacity
US20140279778A1 (en) * 2013-03-18 2014-09-18 The Trustees Of Columbia University In The City Of New York Systems and Methods for Time Encoding and Decoding Machines
JP6009106B2 (en) * 2014-02-07 2016-10-19 三菱電機株式会社 System identification device
CN104978602B (en) * 2014-04-04 2018-02-27 西北工业大学 A kind of method for improving photoelectric follow-up tracking accuracy
CN104330084B (en) * 2014-11-13 2017-06-16 东南大学 A kind of submarine navigation device neural network aiding Combinated navigation method
CN106026152B (en) 2016-05-19 2017-06-06 合肥工业大学 A kind of electric automobile accesses the discharge and recharge dispatching method of micro-capacitance sensor
CN106503689A (en) * 2016-11-14 2017-03-15 哈尔滨理工大学 Neutral net local discharge signal mode identification method based on particle cluster algorithm
CN109800921A (en) * 2019-01-30 2019-05-24 北京师范大学 A kind of Regional Fall Wheat yield estimation method based on remote sensing phenology assimilation and particle swarm optimization algorithm

Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5706400A (en) 1995-03-08 1998-01-06 Nec Research Institute, Inc. Fault-tolerant implementation of finite-state automata in recurrent neural networks
US6047221A (en) * 1997-10-03 2000-04-04 Pavilion Technologies, Inc. Method for steady-state identification based upon identified dynamics
US6381504B1 (en) * 1996-05-06 2002-04-30 Pavilion Technologies, Inc. Method for optimizing a plant with multiple inputs
US6487459B1 (en) * 1996-05-06 2002-11-26 Pavilion Technologies, Inc. Method and apparatus for modeling dynamic and steady-state processes for prediction, control and optimization
US6516309B1 (en) 1998-07-17 2003-02-04 Advanced Research & Technology Institute Method and apparatus for evolving a neural network
US20050033122A1 (en) 1998-10-30 2005-02-10 United States Government As Represented By The Secretary Of The Army Method and system for predicting human cognitive performance
US20050196047A1 (en) 2004-02-03 2005-09-08 Yuri Owechko Object recognition system incorporating swarming domain classifiers
US7039475B2 (en) 2002-12-09 2006-05-02 Pavilion Technologies, Inc. System and method of adaptive control of processes with varying dynamics
US20070019865A1 (en) 2005-03-04 2007-01-25 Yuri Owechko Object recognition using a congnitive swarm vision framework with attention mechanisms
US20070142975A1 (en) 2005-12-21 2007-06-21 Pegasus Technologies, Inc. Model based optimization of a single or multiple power generating units
US7272454B2 (en) 2003-06-05 2007-09-18 Fisher-Rosemount Systems, Inc. Multiple-input/multiple-output control blocks with non-linear predictive capabilities
US20080071397A1 (en) 2006-08-01 2008-03-20 Rawlings James B Partial Enumeration Model Predictive Controller
US7418301B2 (en) 1996-05-06 2008-08-26 Pavilion Technologies, Inc. Method and apparatus for approximating gains in dynamic and steady-state processes for prediction, control, and optimization
US20080208778A1 (en) 2002-12-09 2008-08-28 Bijan Sayyar-Rodsari Controlling a non-linear process
US20080293372A1 (en) * 2005-10-31 2008-11-27 University Of Florida Research Foundation, Inc. Optimum Nonlinear Correntropy Filted
US7496414B2 (en) * 2006-09-13 2009-02-24 Rockwell Automation Technologies, Inc. Dynamic controller utilizing a hybrid model

Patent Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5706400A (en) 1995-03-08 1998-01-06 Nec Research Institute, Inc. Fault-tolerant implementation of finite-state automata in recurrent neural networks
US7418301B2 (en) 1996-05-06 2008-08-26 Pavilion Technologies, Inc. Method and apparatus for approximating gains in dynamic and steady-state processes for prediction, control, and optimization
US6381504B1 (en) * 1996-05-06 2002-04-30 Pavilion Technologies, Inc. Method for optimizing a plant with multiple inputs
US20030018399A1 (en) 1996-05-06 2003-01-23 Havener John P. Method for optimizing a plant with multiple inputs
US6950711B2 (en) * 1996-05-06 2005-09-27 Pavilion Technologies Method for optimizing a plant with multiple inputs
US6487459B1 (en) * 1996-05-06 2002-11-26 Pavilion Technologies, Inc. Method and apparatus for modeling dynamic and steady-state processes for prediction, control and optimization
US6047221A (en) * 1997-10-03 2000-04-04 Pavilion Technologies, Inc. Method for steady-state identification based upon identified dynamics
US6516309B1 (en) 1998-07-17 2003-02-04 Advanced Research & Technology Institute Method and apparatus for evolving a neural network
US20050033122A1 (en) 1998-10-30 2005-02-10 United States Government As Represented By The Secretary Of The Army Method and system for predicting human cognitive performance
US20080208778A1 (en) 2002-12-09 2008-08-28 Bijan Sayyar-Rodsari Controlling a non-linear process
US7039475B2 (en) 2002-12-09 2006-05-02 Pavilion Technologies, Inc. System and method of adaptive control of processes with varying dynamics
US7272454B2 (en) 2003-06-05 2007-09-18 Fisher-Rosemount Systems, Inc. Multiple-input/multiple-output control blocks with non-linear predictive capabilities
US20050196047A1 (en) 2004-02-03 2005-09-08 Yuri Owechko Object recognition system incorporating swarming domain classifiers
US20070019865A1 (en) 2005-03-04 2007-01-25 Yuri Owechko Object recognition using a congnitive swarm vision framework with attention mechanisms
US20080293372A1 (en) * 2005-10-31 2008-11-27 University Of Florida Research Foundation, Inc. Optimum Nonlinear Correntropy Filted
US20070142975A1 (en) 2005-12-21 2007-06-21 Pegasus Technologies, Inc. Model based optimization of a single or multiple power generating units
US20080071397A1 (en) 2006-08-01 2008-03-20 Rawlings James B Partial Enumeration Model Predictive Controller
US7496414B2 (en) * 2006-09-13 2009-02-24 Rockwell Automation Technologies, Inc. Dynamic controller utilizing a hybrid model

Non-Patent Citations (12)

* Cited by examiner, † Cited by third party
Title
Al-Duwaish et al. "Use of multilayer feedforward neural networks in identification and control of wiener model", IEEE, 1996, pp. 255-258. *
Giri et al. "Hammerstein model identification" Proceedings fof the 10th mediterranean conference on control and automation, 2002, 9 pages. *
Gomez et al. "Subspace identification of multivariable hammerstein and wiener models", IFAC, 2002, 6 pages. *
Hou, "Hammerstein model identification based on adaptive particle swarm optimization", Workshop on intelligent information technology application, 2007, pp. 137-140. *
Kennedy et al. "Particle swarm optimization", IEEE, 1995, pp. 1942-1948. *
Lin et al. "A mew identification method for Hammerstein model based on PSO", IEEE, 2006, pp. 2184-2188. *
Liu et al. "Nonlinear system identification of Hammerstein and Wiener model using swarm intelligence", IEEE, 2006, pp. 1219-1223. *
Naitali et al. "Hammerstein and Wiener nonlinear models identification using multimodal particel swarm optimizer" Proceedings of the 2006 American control conference, 2006, pp. 2363-2368. *
Nelles et al. "Identification of non-linear dynamic systems-Classical methods versus radial basis function networks", Proceedings of the american control conference, 1996, pp. 3786-3790. *
Nelles et al. "Identification of non-linear dynamic systems—Classical methods versus radial basis function networks", Proceedings of the american control conference, 1996, pp. 3786-3790. *
Salahshoor et al., "A new on-line subspace-based identification algorithm for multivariable Hammerstein models", IEEE, 2008, pp. 4748-4753. *
Van Overschee et al. "N4SID: Subspace algorithms for the identification of combined deterministic-stochastic systems", Automatica, special issue on statistical processing and control, 1992, pp. 1-50. *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120259600A1 (en) * 2011-04-11 2012-10-11 King Fahd University Of Petroleum And Minerals Method of identifying hammerstein models with known nonlinearity structures using particle swarm optimization
US8620631B2 (en) * 2011-04-11 2013-12-31 King Fahd University Of Petroleum And Minerals Method of identifying Hammerstein models with known nonlinearity structures using particle swarm optimization
US10430438B2 (en) 2014-06-20 2019-10-01 Amazon Technologies, Inc. Dynamic n-dimensional cubes for hosted analytics
US9824133B1 (en) * 2014-06-20 2017-11-21 Amazon Technologies, Inc. Cloud analytics marketplace
US10162876B1 (en) 2014-06-20 2018-12-25 Amazon Technologies, Inc. Embeddable cloud analytics
CN104166883A (en) * 2014-08-05 2014-11-26 中国南方电网有限责任公司超高压输电公司检修试验中心 Power grid dispatching dynamic capacity increasing evaluation method based on state multi-dimension comprehensive evaluation
CN104915933B (en) * 2015-06-01 2019-04-05 长安大学 A kind of Misty Image Enhancement Method based on APSO-BP coupling algorithm
CN104915933A (en) * 2015-06-01 2015-09-16 长安大学 Foggy day image enhancing method based on APSO-BP coupling algorithm
US20170048088A1 (en) * 2015-08-12 2017-02-16 King Fahd University Of Petroleum And Minerals Apparatuses and methodologies for decision feedback equalization using particle swarm optimization
US9614699B2 (en) * 2015-08-12 2017-04-04 King Fahd University Of Petroleum And Minerals Apparatuses and methodologies for decision feedback equalization using particle swarm optimization

Also Published As

Publication number Publication date
US20110125684A1 (en) 2011-05-26

Similar Documents

Publication Publication Date Title
Shin et al. Empirical data modeling in software engineering using radial basis functions
DE69822314T2 (en) A method for non-linear and control echzzeitzustandsschätzung
Gomes et al. Comparison of response surface and neural network with other methods for structural reliability analysis
Schwaab et al. Nonlinear parameter estimation through particle swarm optimization
Fang et al. Structural damage detection using neural network with learning rate improvement
Liu et al. A probabilistic fuzzy logic system for modeling and control
Harris et al. Adaptive modelling, estimation and fusion from data: a neurofuzzy approach
US6725208B1 (en) Bayesian neural networks for optimization and control
Kuss et al. Gaussian processes in reinforcement learning
US20130325774A1 (en) Learning stochastic apparatus and methods
Hagan et al. An introduction to the use of neural networks in control systems
Ding et al. Recursive least squares parameter estimation for a class of output nonlinear systems based on the model decomposition
Chen et al. Regularized orthogonal least squares algorithm for constructing radial basis function networks
Worden et al. Identification of pre-sliding and sliding friction dynamics: Grey box and black-box models
Celikoglu Application of radial basis function and generalized regression neural networks in non-linear utility function specification for travel mode choice modelling
Mongillo et al. Online learning with hidden Markov models
Yu et al. Forecasting of hydrologic time series with ridge regression in feature space
Kandepu et al. Constrained state estimation using the unscented Kalman filter
Mirikitani et al. Recursive bayesian recurrent neural networks for time-series modeling
Heskes et al. A theoretical comparison of batch-mode, on-line, cyclic, and almost-cyclic learning
Bodyanskiy et al. Hybrid adaptive wavelet-neuro-fuzzy system for chaotic time series identification
Karimi et al. Haar wavelet-based approach for optimal control of second-order linear systems in time domain
Roweis et al. Learning nonlinear dynamical systems using the expectation-maximization algorithm
Potočnik et al. Nonlinear model predictive control of a cutting process
Iplikci Support vector machines‐based generalized predictive control

Legal Events

Date Code Title Description
AS Assignment

Owner name: KING FAHD UNIV. OF PETROLEUM & MINERALS, SAUDI ARA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:AL-DUWAISH, HUSSAIN N.;RIZVI, SYED Z.;REEL/FRAME:023590/0062

Effective date: 20091026

STCF Information on status: patent grant

Free format text: PATENTED CASE

FPAY Fee payment

Year of fee payment: 4