US6768927B2  Control system  Google Patents
Control system Download PDFInfo
 Publication number
 US6768927B2 US6768927B2 US09820770 US82077001A US6768927B2 US 6768927 B2 US6768927 B2 US 6768927B2 US 09820770 US09820770 US 09820770 US 82077001 A US82077001 A US 82077001A US 6768927 B2 US6768927 B2 US 6768927B2
 Authority
 US
 Grant status
 Grant
 Patent type
 Prior art keywords
 vector
 system
 state
 correcting
 means
 Prior art date
 Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
 Expired  Fee Related, expires
Links
Images
Classifications

 G—PHYSICS
 G05—CONTROLLING; REGULATING
 G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
 G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
 G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
 G05B13/0265—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion
 G05B13/027—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion using neural networks only
Abstract
Description
The invention relates to a control system, to which a state vector representing the states of a controlled system is applied, and which provides a correcting variables vector of optimized correcting variables, the relation between the state vector and the correcting variables vector being defined by a matrix of weights, the weights being derived from an algorithmic solution of an optimization equation.
In control systems presently in use, all measured states of a controlled system are weighted and applied to all correcting variables. The states of the controlled system are combined in a state vector x. The correcting variables are combined in a correcting variables vector u. The weights are represented by a matrix. In order to achieve optimal control behavior, these weights have to be selected appropriately. The control system is optimized. The weights depend on the solution of an optimization equation such as the “state dependent Riccati equation” (SDRE). This solution is represented by a matrix P(x) depending on the state vector.
In accordance with the prior art, the optimization equation, for example the state dependent Riccati equation, is computed offline. From the solution P(x) for the timedependent state vector, a vector of optimal correcting variables u(x) is computed. As x is timedependent, also a timedependent correcting variables vector results. A neural network is trained in a learning process, various state vectors x being applied to the neural network during the learning process. Then the neural network provides the associated correcting variables vectors. These correcting variables vectors u(t) are compared with the optimized correcting variables vectors u _{opt}(t) which result from the offline solution of, for example, the state dependent Riccati equation. The difference represents the learning signal for training the neural network. Then, the neural network thus trained represents the control system trained offline, for example a guidance controller for missiles, which provides associated optimal correcting variables vectors u(t), when timedependent state vectors x(t) are applied thereto.
Training of a neural network is cumbersome and requires the processing of large quantities of data. A control system obtained in this way is inflexible.
It is an object of the invention to improve a control system of the type defined in the beginning.
To this end, equationsolving means for algorithmically solving the optimization equation in real time are provided. The state vector is applied to these equationsolving means. The solution P(x) of the optimization equation is applied to the control system to determine the weights.
It has been found that the optimization equation, such as the state dependent Riccati equation, can be solved, at least substantially, in real time. This yields a solution P(x) for each state vector x(t). This solution is applied to a control system and, therein, determines the weights of the state variables of the state vector x(t) applied also to the control system. The control system generates, therefrom, the optimal correcting variables of the optimal correcting Variables vector u _{opt}(t). With this design of the control system, the data quantities to be processed are smaller. Instead, high computing capacity is required.
The solution of the optimization equation requires a model of the controlled system. This model of the controlled system can be described by an equation
For analytically solving, for example, the state dependent Riccati equation, this function is “factorized”, i.e. is replaced by a form
{dot over (x)}=F( x ) x+G( x ) u,
wherein F and G are matrices depending on the state vector x. This “factorizing” permits only a simplified model of the controlled system. This model may considerably deviate from reality. The real controlled system nearly always contains uncertainties of the parameters and/or nonlinearities, which cannot be modeled in this form or which may, sometimes, not even be known.
In order to deal with these problems, an adaptive model of the controlled system is provided, to which the state vector and the correcting variables vector are applied and which provides an estimated value of the of the state vector. A first vectorial training signal for the adaptive model is represented by the difference of the actual state vector and the estimated value of the state vector. A second vectorial training signal for an adaptive network which is provided on the control system side and to which the state vector and the correcting variables vector are applied is derived from the trained model of the controlled system. This network on the side of the control system provides a correction quantity for correcting the optimal correcting variables vector resulting from the solution of the optimization equation at the control system, whereby an actual correcting variables vector to be applied to the controlled system is formed.
Preferably, to this end, the model of the controlled system has a first matrix (F(x)) which is multiplied by the state vector x, and a second matrix (G(x)) which is multiplied by the correcting variables vector (u). The sum of the state and correcting variables vectors multiplied by the respective matrices, representing the time derivative of the state vector, is integrated to provide a model value of the state vector. An adaptive structure is provided, on the side of the controlled system, which provides a correcting value for correcting the model value of the state variable, this adaptive structure being trained by the first vectorial training signal.
According to another solution, a structure trained offline has an input to which the state vector (x) of the controlled system is applied, and an output which provides a correcting variables vector (u), the correcting variables of the correcting variables vector being applied to the controlled system. The state vector (x) is applied online to the equation solving means, whereby the equation solving means provide an optimal correcting variables vector (u _{opz}(t)). The difference of the correcting variables vector (u(t)) provided by the adaptive structure and of the optimal correcting variables vector (u _{opz}(t)) is applied online to the adaptive structure as a training signal.
The control system has an adaptive structure such as a neural network, which is trained offline to represent an optimal control system. This structure provides stable control, which may, however, not be optimal in some ranges of the state space. In addition thereto, the optimization equation such as the state dependent Riccati equation is solved in real time. The correcting variables vector obtained thereby for the respective timedependent state vector is compared to the correcting variables vector provided by the adaptive structure arid, thereby, serves to continuously further train the adaptive structure “online”.
The weight factors for providing the optimal correcting variables vector may be determined, instead of by the state dependent Riccati equation, also by other optimization equations. The control system may, for example, be a LQcontroller. The described procedure of correcting a model of the control system by means of an adaptive structure such as a neural network and of correcting the optimal correcting variables vector through the control system by means of a second adaptive structure trained by the corrected model of the control system may, if required, also be used with an already available control system operating with proportional navigation or extended proportional navigation, in order to improve the control behavior thereof.
The adaptive structure may be a neural network or a fuzzyneural network or an adaptive fuzzy logic unit. The adaptive structure may also be trained offline with knowledge about the dynamic behavior of the controlled system in a simulation process.
Embodiments of the invention are described hereinbelow with reference to the accompanying drawings:
FIG. 1 shows a block diagram of a control system with “online” solution of the state dependent Riccati equation.
FIG. 2 shows a block diagram of a control system wherein a correction of the model of the controlled system and a correction of the control system effected by training procedures.
FIG. 3 shows a system, wherein a trained neural network used as control system is further trained online by a solution of the state dependent Riccati equation obtained in real time.
FIG. 4 shows a system, wherein a solution of the state dependent Riccati equation is found substantially in real time by means of a neural network imaging this equation.
FIG. 5 shows another system, wherein a solution of the state dependent Riccati equation is found substantially in real time by means of a processor (systolic array processor) arranged in a control loop.
Referring to FIG. 1, numeral 10 designates a controlled system. The controlled system may be a missile which tracks a target, the trajectory of the missile to the target being to be controlled. The controlled system can be described by an equation {dot over (x)}=g(x,u,t), x being a state vector, i.e. a vector of the state variables of the controlled system, u being a correcting variables vector, i.e. a vector of the available correcting variables, t being the time, an g designating a function. The controlled system is affected by disturbances, which are combined in a vector x _{0}, and—with a guidance controller for missiles—by the motions of a target to which the missile is to be guided. The controlled system 10 is controlled by a controlling system 12. The state vector x(t) of the controlled system 10 is applied to the controlling system 12. The state vector x(t) represents the input variables of the controlling system 12 at an input 14. The controlling system, for example a guidance controller of a missile, provides correcting variables, which are combined in a correcting variables vector u. The correcting variables vector u(t) is linked with the state vector x(t) through a matrix, i.e. each correcting variable of the correcting variables vector u is dependent on each state variable of the state vector x through a weight factor, an element of the matrix. The correcting variables of the correcting variables vector u are applied to the controlled system from an output 16 of the controlling system 10. This is symbolized in FIG. 1 by an “input” 18. Now, the weight factors of the matrix of the controlling system 10 are to be selected such that optimal control behavior is achieved. This is indicated by the designation “u _{opt}(t)” in FIG. 1.
In FIG. 1, this optimization of the weight factors is achieved by solving the SDRE (“state dependent Riccati equation”) online. The equation solving means 20 for the online solution of the state dependent Riccati equation are represented by a block 20 in FIG. 1. The timedependent state vector x(t) is applied to an input 22 of the equation solving means 20. The equation solving means 20 provide, as a solution, a matrix P(x). This matrix P(x) is applied to the control system 12, as indicated by arrow 24. Then the control system represents the optimal weight matrix −R^{−1}G^{T}P.
The solution of the state dependent Riccati equation is based on the factoring given above: {dot over (x)}=F(x)x+G(x)u. This factoring may, however, not be able to model the controlled system sufficiently. In the embodiment of FIG. 2, adaptive structures are used to correct the modeling of the controlled system and to correct, in turn, the control system in accordance with this correction.
Referring to FIG. 2, numeral 26 designates a controlling system similar to the controlling system 10 of FIG. 1. The state vector x of a controlled system 30 is applied to the controlling system 26 through an input 28. Equation solving means 32, to which the state vector x is also applied through an input 34, provide a solution P(x) of the state dependent Riccati equation. This solution P(x) is applied to the controlling system 26. Thereby, the weights of the weight matrix of the controlling system 26 are optimized. The controlling system 26 provides a correcting variables vector u _{opt}.
This is substantially the structure of FIG. 1. The controlling system may be a guidance controller which guides a missile to a target, this guidance representing the controlled system.
A model 36 of the controlled system is provided for the controlled system 30, this model underlying the SDRE. The state vector x is multiplied by a Function F(x), which is represented by a block 38. The correcting variables vector u is multiplied by a function G(x), which is represented by a block 40. The products are summed at a summing point 42. This yields the time derivative {dot over ({overscore (x)})} of a model state vector. Integrating this time derivative {dot over ({overscore (x)})} yields the model state vector {overscore (x)} itself. The integration is indicated by a block 42, the initial value {overscore (x_{0} )} being additionally supplied to this block 42.
The state vector {overscore (x)} thus obtained is corrected at a summing point 44 by a vectorial correcting value Δx. This yields an estimated value {circumflex over (x)} of the state vector. The difference δx of the actually measured state vector x and the estimated value {circumflex over (x)} is formed at a summing point 46. This difference δx is applied to the neural network 48 on the side of the controlled system as a training signal. The neural network on the side of the controlled system provides the vectorial correcting value Δx which is applied to the summing point 44. Optionally, also a vectorial correcting value Δ{dot over (x)} may be applied to the summing point 44 by the neural network, as indicated by dashed connection 50. In this way, the model 36 on which the solution of the SDRE is based can be matched with the real controlled system.
This matched model now permits also the controlling system 26 to be matched. To this end, the difference δx is applied not only as a training signal to the neural network on the side of the controlled system but also is backpropagated through this controlled systemside neural network 48, as shown by connection 48. The neural network 48 trained as described receives, at an input 54, the difference δx and provides, at an output 56, a vector δu. This vector δu serves as a training signal for a neural network on the side of the controlling system. This is illustrated by connection 60. The neural network 58 on the side of the controlling system receives the real state vector x at an input 62 and the correcting variables vector u at an input 64, as indicated by connections 66 and 68, respectively.
The adaptive model of the controlled system is designated by numeral 70. The adaptive controlling system is designated by numeral 72. The training procedures take place online.
Referring to FIG. 3, the controlling system comprises a neural network 74. The neural network 74 is trained offline. The neural network 74 receives a state vector x(t) from a controlled system 76, the state vector representing the state variables of the controlled system 76. The trained neural network 74 provides a correcting variables vector u(t), which represents the correcting variables vectors acting on the controlled system. Also here, the controlled system may be the process of tracking a target by a missile. This is conventional technology.
In the embodiment of FIG. 3, equation solving means 78 for solving the SDRE are additionally provided. These equation solving means 78 receive continuously the timedependent state vector x(t) of the controlled system 76. The solution P(x) provided by the equation solving means permits the formation of an optimal correcting variables vector u _{opt}(x) depending on the state vector and, as this state vector x is timedependent, of a timedependent optimal correcting variables vector u _{opt}(t). At a summing point 80, this optimal correcting variables vector u _{opt}(t) is compared with a real correcting variables vector u(t) provided by the neural network 74. The difference serves as a training signal for the neural network 74, whereby the neural network 74 is retrained online.
In this way, the neural network 74 can be trained in real time in additional points of the state space.
FIG. 4 illustrates one mode of solving the SDRE in real time.
Referring to FIG. 4, numeral 82 designates a neural network. This neural network 82 represents a “copy” of the SDRE. The processor elements of the neural network 82 carry out the operation of a scalar product. The SDRE can be represented by scalar products. The output 84 of the neural network 82 should be zero. Therefore, a desired output of “0” is set. The desired output “0” is compared with the actual output at a summing point 88. The difference is a training signal by means of which the neural network 82 is trained, as indicated in FIG. 4. By this training, the weight factors of the neural network are set in such a way that “0” appears at the output 84. The relation between the state vector x at the input of the neural network and the vector “0” at the output is determined by a weight matrix. If the processor elements represent a “copy” of the SDRE, then the weight matrix after the training represents the solution P(x) of the SDRE. This solution is derived from output 90.
Also this training process can take place substantially in real time.
In the embodiment of FIG. 5, the neural network is replaced by a parallel processor for vectormatrix operations (such as a socalled “systolic array processor” ) 92. In this processor, the SDRE, is implemented by appropriate vectormatrix operations. The output 94 should yield “0”. This is represented by a vector “0” at a desired output 96. The difference formed at a summing point 98 is applied to a controller 100. The controller 100 varies the P(x)parameters of this implemented SDRE such that “0” appears at output 94. Then the parameters represent the solution of the SDRE.
Claims (11)
Priority Applications (3)
Application Number  Priority Date  Filing Date  Title 

DE2000117600 DE10017600A1 (en)  20000408  20000408  Flight regulator for aircraft uses algorithm based upon real time solution of optimization equation to determine optimum control parameter based upon state vector and control vector 
DE10017600.3  20000408  
DE10017600  20000408 
Publications (2)
Publication Number  Publication Date 

US20010034560A1 true US20010034560A1 (en)  20011025 
US6768927B2 true US6768927B2 (en)  20040727 
Family
ID=7638108
Family Applications (1)
Application Number  Title  Priority Date  Filing Date 

US09820770 Expired  Fee Related US6768927B2 (en)  20000408  20010329  Control system 
Country Status (3)
Country  Link 

US (1)  US6768927B2 (en) 
EP (1)  EP1148395A3 (en) 
DE (1)  DE10017600A1 (en) 
Cited By (4)
Publication number  Priority date  Publication date  Assignee  Title 

US20050141683A1 (en) *  20031225  20050630  Yoshikazu Ishii  Control and monitoring telecommunication system and method of setting a modulation method 
US20070067050A1 (en) *  20050920  20070322  Honeywell International Inc.  Method for softcomputing supervision of dynamical processes with multiple control objectives 
US20120277888A1 (en) *  20110429  20121101  Georgia Tech Research Corporation  Systems and methods for parameter dependent riccati equation approaches to adaptive control 
US8710411B1 (en) *  20090929  20140429  Lockheed Martin Corporation  Method and system for determining an optimal missile intercept approach direction for correct remote sensortoseeker handover 
Families Citing this family (3)
Publication number  Priority date  Publication date  Assignee  Title 

EP2354541B1 (en) *  20100120  20140917  Siemens Aktiengesellschaft  Wind farm power control based on matrix reflecting a power load distribution between individual wind turbines 
US9095886B2 (en) *  20110627  20150804  University Of Central Florida Research Foundation, Inc.  Mill control system and method for control of metal strip rolling 
WO2013072464A3 (en) *  20111117  20140530  Siemens Aktiengesellschaft  Method and device for controlling a temperature of steam for a steam power plant 
Citations (4)
Publication number  Priority date  Publication date  Assignee  Title 

US5159661A (en) *  19901005  19921027  Energy Conversion Devices, Inc.  Vertically interconnected parallel distributed processor 
US5991525A (en) *  19970822  19991123  Voyan Technology  Method for realtime nonlinear system state estimation and control 
US6064332A (en) *  19940426  20000516  The United States Of America As Represented By The Secretary Of The Air Force  Proportional Guidance (PROGUIDE) and Augmented Proportional Guidance (Augmented PROGUIDE) 
US6244536B1 (en) *  19971126  20010612  The United States Of America As Represented By The Secretary Of The Air Force  Air to air homing missile guidance 
Patent Citations (5)
Publication number  Priority date  Publication date  Assignee  Title 

US5159661A (en) *  19901005  19921027  Energy Conversion Devices, Inc.  Vertically interconnected parallel distributed processor 
US6064332A (en) *  19940426  20000516  The United States Of America As Represented By The Secretary Of The Air Force  Proportional Guidance (PROGUIDE) and Augmented Proportional Guidance (Augmented PROGUIDE) 
US5991525A (en) *  19970822  19991123  Voyan Technology  Method for realtime nonlinear system state estimation and control 
US6285971B1 (en) *  19970822  20010904  Voyan Technology  Method for realtime nonlinear system state estimation and control 
US6244536B1 (en) *  19971126  20010612  The United States Of America As Represented By The Secretary Of The Air Force  Air to air homing missile guidance 
NonPatent Citations (6)
Title 

Cloutier J.R. et al., "StateDependent Riccati Equation Techniques: An Overview", Proceedings of the American Control Conference 932936 (1997), XP002240769. 
He S. et al., "Solving Riccati Differential Equations with Multilayer Neural Networks", Proceedings of the 36<th >Conference on Decision & Control 21992200 (1997), XP002240771. 
He S. et al., "Solving Riccati Differential Equations with Multilayer Neural Networks", Proceedings of the 36th Conference on Decision & Control 21992200 (1997), XP002240771. 
Mracek C.P. et al., "A New Technique for Nonlinear Estimation", Proceedings of the 1996 IEEE International Conference on Control Applications 338343 (1996), XP002240770. 
Neidhoefer J. et al., "Intelligent Control for NearAutonomous Aircraft Missions", Systems, Man, and Cybernetics, IEEE International Conference 31243129 (1998). 
Wise K.A. et al., "Nonlinear Control of Agile Missiles Using State Dependent Riccati Equations", Proceedings of the American Control Conference 379380 (1997), XP001031806. 
Cited By (7)
Publication number  Priority date  Publication date  Assignee  Title 

US20050141683A1 (en) *  20031225  20050630  Yoshikazu Ishii  Control and monitoring telecommunication system and method of setting a modulation method 
US7570748B2 (en) *  20031225  20090804  Hitachi, Ltd.  Control and monitoring telecommunication system and method of setting a modulation method 
US20070067050A1 (en) *  20050920  20070322  Honeywell International Inc.  Method for softcomputing supervision of dynamical processes with multiple control objectives 
US7769474B2 (en)  20050920  20100803  Honeywell International Inc.  Method for softcomputing supervision of dynamical processes with multiple control objectives 
US8710411B1 (en) *  20090929  20140429  Lockheed Martin Corporation  Method and system for determining an optimal missile intercept approach direction for correct remote sensortoseeker handover 
US20120277888A1 (en) *  20110429  20121101  Georgia Tech Research Corporation  Systems and methods for parameter dependent riccati equation approaches to adaptive control 
US9058028B2 (en) *  20110429  20150616  Georgia Tech Research Corporation  Systems and methods for parameter dependent riccati equation approaches to adaptive control 
Also Published As
Publication number  Publication date  Type 

DE10017600A1 (en)  20011011  application 
EP1148395A3 (en)  20030723  application 
US20010034560A1 (en)  20011025  application 
EP1148395A2 (en)  20011024  application 
Similar Documents
Publication  Publication Date  Title 

Sheridan  Three models of preview control  
Papageorgiou et al.  Active Hebbian learning algorithm to train fuzzy cognitive maps  
Werbos  Consistency of HDP applied to a simple reinforcement learning problem  
Park et al.  An optimal tracking neurocontroller for nonlinear dynamic systems  
De Veaux et al.  A comparison of two nonparametric estimation schemes: MARS and neural networks  
Barto  1" 1 Adaptive Critics and the Basal Ganglia  
US5119468A (en)  Apparatus and method for controlling a process using a trained parallel distributed processing network  
SaintDonat et al.  Neural net based model predictive control  
Sutton  Adapting bias by gradient descent: An incremental version of deltabardelta  
US5285377A (en)  Control apparatus structuring system  
Levin et al.  Control of nonlinear dynamical systems using neural networks: Controllability and stabilization  
Tian et al.  Adaptive neurofuzzy control of a flexible manipulator  
US5781700A (en)  Trained Neural network air/fuel control system  
Prokhorov et al.  Adaptive critic designs  
Tomizuka et al.  On the optimal digital state vector feedback controller with integral and preview actions  
US20040249483A1 (en)  Multipleinput/multipleoutput control blocks with nonlinear predictive capabilities  
Karakasoglu et al.  Identification and decentralized adaptive control using dynamical neural networks with application to robotic manipulators  
Anderson  Strategy learning with multilayer connectionist representations  
US5579439A (en)  Fuzzy logic design generator using a neural network to generate fuzzy logic rules and membership functions for use in intelligent systems  
Jenkins et al.  A simplified neural network solution through problem decomposition: The case of the truck backerupper  
US5493631A (en)  Stabilized adaptive neural network based control system  
US4907170A (en)  Inference machine using adaptive polynomial networks  
US5924086A (en)  Method for developing a neural network tool for process identification  
US5579442A (en)  Adaptive kinematic control apparatus  
US5056037A (en)  Analog hardware for learning neural networks 
Legal Events
Date  Code  Title  Description 

AS  Assignment 
Owner name: BODENSEEWERK GERATETECHNIK GMBH, GERMANY Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KROGMANN, UWE;REEL/FRAME:011662/0662 Effective date: 20010305 

FPAY  Fee payment 
Year of fee payment: 4 

REMI  Maintenance fee reminder mailed  
REMI  Maintenance fee reminder mailed  
LAPS  Lapse for failure to pay maintenance fees  
FP  Expired due to failure to pay maintenance fee 
Effective date: 20120727 