CN112286052A - Method for solving industrial control optimal tracking control by using linear system data - Google Patents
Method for solving industrial control optimal tracking control by using linear system data Download PDFInfo
- Publication number
- CN112286052A CN112286052A CN202011015464.1A CN202011015464A CN112286052A CN 112286052 A CN112286052 A CN 112286052A CN 202011015464 A CN202011015464 A CN 202011015464A CN 112286052 A CN112286052 A CN 112286052A
- Authority
- CN
- China
- Prior art keywords
- optimal
- tracking control
- optimal tracking
- controller
- solving
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 230000006870 function Effects 0.000 claims description 20
- 238000011217 control strategy Methods 0.000 claims description 13
- 239000011159 matrix material Substances 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 5
- 238000011156 evaluation Methods 0.000 claims description 3
- 238000013461 design Methods 0.000 abstract description 7
- 238000005259 measurement Methods 0.000 abstract description 4
- 230000003416 augmentation Effects 0.000 abstract description 3
- 238000004519 manufacturing process Methods 0.000 abstract description 2
- 238000004088 simulation Methods 0.000 description 3
- VGGSQFUCUMXWEO-UHFFFAOYSA-N Ethene Chemical compound C=C VGGSQFUCUMXWEO-UHFFFAOYSA-N 0.000 description 1
- 239000005977 Ethylene Substances 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 238000005336 cracking Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
- G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
- G05B13/04—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
- G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
- G05B13/0265—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion
Landscapes
- Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Feedback Control In General (AREA)
Abstract
The invention discloses a method for solving the optimal tracking control of industrial control by utilizing linear system data, which relates to an optimal tracking control method of industrial control. The invention mainly aims at the problem that the optimal controller needs an accurate system model, and designs a method for learning the optimal tracking controller gain without system dynamics knowledge. The invention has the main advantages that the error is tracked by using input, output and measured augmentation variables such as past values of the input and the output, and a state observer does not need to be designed while an approximate optimal tracking controller is obtained. And the system under consideration can track the reference signal by an optimal method when the controller is used under the controller, and the technology can design the controller under the conditions that the controlled system of the industrial process is difficult to accurately model and the system state measurement cost is high or is not measurable.
Description
Technical Field
The invention relates to an optimal tracking control method for industrial control, in particular to a method for solving the optimal tracking control method for industrial control by utilizing linear system data.
Background
An optimal control law is found to ensure tracking of the reference signal by minimizing certain performance indicators that contain tracking errors. The method is a basic problem of a control theory with wide application in industry, medical treatment, aviation and the like, and the rich achievement of optimal tracking control really lays a foundation for a series of future researches. As known in the open literature, the problem of modeless optimal tracking control where system conditions are not available has not been fully studied. Note, however, that when designing an optimal controller, pioneering work requires an accurate system model, and this requirement is rarely available in practice. Data-driven Reinforcement Learning (RL) has attracted increasing attention over the past decades because it is able to find an optimal solution through interaction with an unknown environment. In practice, it is difficult to obtain system model parameters or system states, it is difficult to accurately model the industrial process controlled systems, and the system state measurements are costly or non-measurable, so tracking using measured augmented variables such as inputs, outputs, and their past values is essential.
Disclosure of Invention
The invention aims to provide a method for solving the optimal tracking control of industrial control by using linear system data, and provides an intelligent design scheme of a discrete linear system optimal tracking controller driven by complete data, which utilizes input, output, past values of the input and output and other measurement augmentation variables to track, thereby solving the problems existing in the design of the industrial system controller.
The purpose of the invention is realized by the following technical scheme:
a method for solving industrial control optimal tracking control by using linear system data is characterized by comprising the following processes:
aiming at a linear discrete system, Q learning is applied to an extended form of a system state space, an improved discrete time linear system data driving optimal tracking control algorithm is provided, the algorithm is used for solving the problem of linear quadratic regulation of an extended state space model only by using input, output and measured values, and a data driving method is adopted;
firstly, proposing an optimal tracking control problem, then rewriting a system model into an extended state space model, searching an optimal tracking control strategy by designing differential input, and enabling the output of the system to follow a target reference signal under the condition of not considering system dynamics;
based on an optimal tracking control strategy, an optimal value function based on Q learning is designed;
the algorithm utilizes the measured data to obtain an approximate optimal tracking controller, under the controller, a considered system tracks a reference signal through an optimal method, an output adjusting method is adopted, and an optimal tracking control strategy is learned by directly utilizing measured differential output, differential input and tracking errors; firstly, a relation matrix is constructed, then an optimal value function and an optimal Q function are respectively defined, and a Bellman equation of the Q function is obtained according to a stability control strategy.
The method for solving the optimal tracking control of the industrial control by utilizing the linear system data comprises the following steps of:
(1) establishing a proper relation matrix;
(2) further optimizing the optimal value function according to a stability control strategy, thereby obtaining a Bellman equation of the Q function;
and designing an iterative algorithm on the basis.
The method for solving the optimal tracking control of the industrial control by utilizing the linear system data comprises the following four steps of:
(1) initializing to give a stable controller gain
；
(2) Then by solving the Q function matrix
Performing performance evaluation;
(3) and finally, strategy updating is carried out:
；
(1) (4) when
Is a very small value, the iteration is stopped.
The invention has the advantages and effects that:
because the optimal tracking controller is difficult to obtain in practical application, the method does not need to utilize system model parameters and system states, a new design method is provided, and a Q learning algorithm and an extended state space model are combined, so that for an unknown linear system, the optimal tracking control strategy can be found only by measuring input and output of states and past values of the states and adding tracking errors, and the method can be applied to an industrial system to solve the problem of controller design.
Drawings
FIG. 1 is a general scheme of a linear system data-driven tracking control method based on Q learning;
FIG. 2
The convergence result of (2);
FIG. 3Q traces of the learning algorithm;
the figure 4Q learning algorithm controls the input trajectory.
Detailed Description
The present invention will be described in detail with reference to the embodiments shown in the drawings.
The present invention employs Q-learning to solve the optimal tracking problem, which is first described as minimizing a specific cost function containing the tracking reference signal target with extended state space implementation. Then, a Q-learning algorithm is proposed, which only needs to utilize the measurement data, without the need to design a state observer, so as to obtain an approximately optimal tracking controller under which the considered system can track the reference signal in an optimal way.
The invention converts the optimal tracking control problem of a linear system into the linear quadratic regulation problem of a system expansion state space expression and provides a novel strategy Q learning algorithm based on data driving. A simple extended state-space equation is used to simplify the solution process of the proposed strategy-based Q learning method.
The invention comprises four steps:
the method comprises the following steps: optimal tracking problem for discrete time linear systems
In this section, the optimal tracking problem for DT (discrete time) linear systems is described by an extended state space expression, making the differential inputs the decision variables that drive the system to track the target reference signal. Consider a linear system of DTs described by an input-output state space model.
And establishing a DT linear system described by the input and output state space model.
Wherein
And
is the output and input of the system at the sampling time instant.
And
is a matrix of appropriate dimensions.
Now the two back shift operators
Is defined as (1), i.e.
And
. Then (1) can be rewritten as:
step two: extended state space model transformation
Defining an augmentation vector:
assuming that the target output is a constant reference signal, the output tracking error can be defined such that
And obtaining the desired extended state space model.
The difference variables are compressed by using the tracking error, and the established extended state space model adopts a method of combining dynamic programming and an RL framework to process the optimal tracking control problem instead of using the general state and output equation of the system. The goal is to find the optimal tracking control strategy by designing the differential inputs, so that the output of the system (1)
A target reference signal is tracked.
Wherein
And
is a symmetric matrix, therefore, the optimal tracking control problem can be expressed as:
this sequence finds the optimal tracking control strategy by solving the optimization problem shown in (3) without knowledge of the system.
Step three: strategy Q learning algorithm
First, initialization is performed to give a stable controller gain
Let a
Wherein
Representing an iteration index; then by solving the Q function matrix
And (4) performance evaluation is carried out:
(ii) a And finally, updating the strategy as follows:
when is coming into contact with
And is
And then stop.
Step four: example verification algorithm
In this section, the effectiveness of the proposed strategy Q-learning algorithm is illustrated by an example. For the system (1), selecting
，
，
，
，
And the target signal of reference is
Wherein
By adopting the method, the sum is selected, firstly, the parameter of the system is assumed to be known, and then the optimal Q function matrix and the optimal tracking controller gain are respectively obtained by using a dare command in Matlab:
the algorithm obtains the optimal Q function matrix and the optimal tracking controller gain after 10 iterations. The simulation verifies the effectiveness of the algorithm.
Case simulation:
if the system model is unknown, the generalized process model of the SC-1 ethylene cracking furnace system is as follows:
when in use
When the system model is unknown, the DT linear system described by the input-output state space model can be written as:
using the method of the invention, selection
And
firstly, assuming that the parameters of the system are known, then using the dare command in Matlab to respectively obtain the optimal Q function matrix and the optimal tracking controller gain:
the effectiveness of the method is verified through a series of simulation examples.
Claims (3)
1. A method for solving industrial control optimal tracking control by using linear system data is characterized by comprising the following processes:
aiming at a linear discrete system, Q learning is applied to an extended form of a system state space, an improved discrete time linear system data driving optimal tracking control algorithm is provided, the algorithm is used for solving the problem of linear quadratic regulation of an extended state space model only by using input, output and measured values, and a data driving method is adopted;
firstly, proposing an optimal tracking control problem, then rewriting a system model into an extended state space model, searching an optimal tracking control strategy by designing differential input, and enabling the output of the system to follow a target reference signal under the condition of not considering system dynamics;
based on an optimal tracking control strategy, an optimal value function based on Q learning is designed;
the algorithm utilizes the measured data to obtain an approximate optimal tracking controller, under the controller, a considered system tracks a reference signal through an optimal method, an output adjusting method is adopted, and an optimal tracking control strategy is learned by directly utilizing measured differential output, differential input and tracking errors; firstly, a relation matrix is constructed, then an optimal value function and an optimal Q function are respectively defined, and a Bellman equation of the Q function is obtained according to a stability control strategy.
2. The method for solving the optimal tracking control problem of the industrial control by using the linear system data as claimed in claim 1, wherein the optimal value function based on the Q learning comprises the following steps:
(1) establishing a proper relation matrix;
(2) further optimizing the optimal value function according to a stability control strategy, thereby obtaining a Bellman equation of the Q function;
and designing an iterative algorithm on the basis.
3. The method for solving the industrial control optimal tracking control by using the linear system data as claimed in claim 2, wherein the iterative algorithm comprises the following four steps:
(1) initializing to give a stable controller gain
；
(2) Then by solving the Q function matrix
Performing performance evaluation;
(3) and finally, strategy updating is carried out:
；
(4) when in use
Is a very small value, the iteration is stopped.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011015464.1A CN112286052A (en) | 2020-09-24 | 2020-09-24 | Method for solving industrial control optimal tracking control by using linear system data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011015464.1A CN112286052A (en) | 2020-09-24 | 2020-09-24 | Method for solving industrial control optimal tracking control by using linear system data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112286052A true CN112286052A (en) | 2021-01-29 |
Family
ID=74422922
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011015464.1A Pending CN112286052A (en) | 2020-09-24 | 2020-09-24 | Method for solving industrial control optimal tracking control by using linear system data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112286052A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114237184A (en) * | 2021-12-20 | 2022-03-25 | 杭州电子科技大学 | Method for improving optimized learning control performance of industrial process |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104317321A (en) * | 2014-09-23 | 2015-01-28 | 杭州电子科技大学 | Coking furnace hearth pressure control method based on state-space predictive functional control optimization |
| CN105334751A (en) * | 2015-11-26 | 2016-02-17 | 杭州电子科技大学 | Design method for stability controller of batched injection molding process |
| CN106843171A (en) * | 2016-12-28 | 2017-06-13 | 沈阳化工大学 | A kind of operating and optimization control method based on data-driven version |
-
2020
- 2020-09-24 CN CN202011015464.1A patent/CN112286052A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104317321A (en) * | 2014-09-23 | 2015-01-28 | 杭州电子科技大学 | Coking furnace hearth pressure control method based on state-space predictive functional control optimization |
| CN105334751A (en) * | 2015-11-26 | 2016-02-17 | 杭州电子科技大学 | Design method for stability controller of batched injection molding process |
| CN106843171A (en) * | 2016-12-28 | 2017-06-13 | 沈阳化工大学 | A kind of operating and optimization control method based on data-driven version |
Non-Patent Citations (1)
| Title |
|---|
| 李金娜等: "基于非策略Q-学习的网络控制系统最优跟踪控制", 《控制与决策》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114237184A (en) * | 2021-12-20 | 2022-03-25 | 杭州电子科技大学 | Method for improving optimized learning control performance of industrial process |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108789417B (en) | Reconfigurable mechanical arm cost-preserving decentralized control method based on self-adaptive dynamic programming | |
| Henson et al. | Nonlinear process control | |
| CN107942667B (en) | Injection molding process hybrid 2D tracking control method based on time-varying time lag and interference | |
| Zhou et al. | Deep convolutional neural network based fractional-order terminal sliding-mode control for robotic manipulators | |
| CN105773623A (en) | SCARA robot trajectory tracking control method based on prediction indirect iterative learning | |
| Cho et al. | An indirect model reference adaptive fuzzy control for SISO Takagi–Sugeno model | |
| CN112859604B (en) | Two-dimensional data drive optimal tracking control method for unknown dynamic injection molding process | |
| CN105240846A (en) | Method for controlling combustion process of circulating fluidized bed boiler on basis of multivariable generalized predictive control optimization | |
| CN111399375A (en) | Neural network prediction controller based on nonlinear system | |
| CN112286052A (en) | Method for solving industrial control optimal tracking control by using linear system data | |
| Osman et al. | Predictive functional control with observer (PFC-O) design and loading effects performance for a pneumatic system | |
| CN105353610A (en) | Magnetic-control shape memory alloy actuator modeling method based on KP model | |
| CN114740723A (en) | Software robot robust self-adaptive control method based on disturbance observer | |
| Liu et al. | Adaptive dynamic programming for optimal control of unknown nonlinear discrete-time systems | |
| CN110888323A (en) | Control method for intelligent optimization of switching system | |
| Zips et al. | Torque control of a hydrostatic transmission applied to a wheel loader | |
| CN108536008A (en) | A kind of iterative learning control method of MIMO nonlinear systems | |
| Liu et al. | Data learning‐based model‐free adaptive control and application to an NAO robot | |
| CN115327903B (en) | Off-track strategy optimal tracking control method for two-dimensional state time-lag batch processing process | |
| CN109491354A (en) | A kind of full level of factory performance optimal control method of complex industrial process data-driven | |
| CN114296349B (en) | Hysteresis control method and device for nanometer positioning platform | |
| Mahmoud et al. | Sliding mode control of nonlinear SISO systems with both matched and unmatched disturbances | |
| Yamada et al. | Iterative control of three link manipulator using 2D system control approach | |
| CN117908362B (en) | Composite anti-saturation control method and control system for flexible joint robot | |
| Wang et al. | Multi-step predictive control with TDBP method for pneumatic position servo system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210129 |
|
| RJ01 | Rejection of invention patent application after publication |