CN107490962B - Data-driven optimal control method for servo system - Google Patents

Abstract

The invention adopts a data-driven optimal control method of a servo system, which specifically comprises state error feedback control and adaptive optimization control based on strategy iteration, wherein the state error feedback control can utilize sampling data to calculate feedback gain K in real time; the adaptive optimization controller directly uses the control quantity and the state error quantity obtained by the state error feedback control to carry out iterative computation and gradually approaches to the optimal control quantity; the method does not need to know a specific system model, and only needs a measurable state error amount.

Description

Data-driven optimal control method for servo system

Technical Field

The invention relates to a servo control system of a direct current motor, belongs to the technical field of electromechanical control, and particularly relates to an optimal control method of a data-driven servo system.

Background

In servo control, the system model changes due to the presence of uncertainty factors such as dynamic friction, load variations, and external disturbances. Furthermore, if unknown parameters are present in the control system, the initial estimated values for these parameters may differ significantly from their true values. Therefore, if the system model is not accurately obtained in the control process, it is necessary to develop a control algorithm that does not depend on the system model.

The data-driven control algorithm directly utilizes the sampled data to replace a system model to design the controller, wherein the data-driven self-adaptive dynamic programming algorithm is an effective optimal control method. The data-driven self-adaptive dynamic programming theory integrates methods such as DP, RL and function approximation, and the essence is that the system performance index function is estimated by using online or offline data and adopting a function approximation structure, and then the optimal control strategy is obtained according to the optimality principle.

Disclosure of Invention

In view of the above, the invention adopts a data-driven optimal control method for a servo system, which does not need to know a specific system model and only needs a measurable state error amount, aiming at the problem that the servo system cannot be accurately modeled.

The design idea of the invention is as follows:

setting an error equation of a direct current motor servo system as follows:

wherein u is a control quantity, and e is a state error;

the cost function is defined as follows:

wherein r (e, u) ═ Q (e) + u^TRu, Q are positive definite, R is a positive definite symmetric matrix;

and finally, the control state error e converges to 0 under the condition of optimal cost function, and the servo system is controlled by using the obtained control quantity u.

The technical scheme for realizing the invention is as follows:

the optimal control method of the data-driven servo system comprises two parts of data-driven state error feedback control and data-driven adaptive optimization control:

a first part: data driven state error feedback control

101. Setting K to 1, and initializing system feedback gains K (K-1) and K (K) at the K-1 and K-th moments;

102 applying the system feedback gains K (K-1) and K (K) to the servo system in sequence in a time interval [ kT, (K +1) T ];

103. sampling the state error of the servo system, and calculating a state error relation matrix A (k), g according to the expressions (10), (12) to (15)_e(k) And g_u(k)；

A(k)＝A₀(k)A₁(k) (10)

A₁(k)＝e((k+1/2)T)(e(kT))⁺(12)

A₀(k)＝e((k+1)T)(e((k+1/2)T))⁺(13)

Then, it is possible to obtain:

g_u(k)＝(A₀(k)-A₁(k))(K(k)-K(k-1))⁺(14)

g_e(k)＝A₀(k)-g_u(k)·K(k) (15)

wherein e (kT) and e ((k +1/2) T) represent the system state error at the time of k and (k +1/2) and are represented by the symbol ·⁺Represents a pseudo-inverse of the matrix;

104. calculating K (K +1) and K (K +2) according to equation (11);

wherein, λ represents a set step length and is a constant between 0 and 1; the symbol | | | | represents the Frobenius norm of the matrix.

105. Judging whether k is smaller than a set threshold value, namely whether the k-th time reaches a set cut-off time T1, if so, entering a second part, otherwise, making k equal to k +1, and returning to the step 102;

in the first part, the state error feedback control quantity is continuously adjusted while being applied, and the obtained input u⁰And the state error data e can be used for the next part of the adaptive optimization controller.

A second part: data-driven adaptive optimization control

Setting j to 0, define phi_ki、

u^j，

Is expressed as follows

φ_ki＝φ_i(e(t_k))-φ_i(e(t_k-1))

Wherein j is the number of iterations; phi is a_i(e),(i＝1,...N_w)，ψ_i(e,u),(i＝1,...N_c)，σ_i(e),(i＝1,...N_l) Is a basis function;

as a weight value u⁰In order to feed back the control quantity for the state error,

is u^j(e) Is determined by the estimated value of (c),

as a function u^j(e) An approximation error between the actual value and the estimated value.

201. Calculating phi based on the system state error obtained in the first part_kiAnd

then obtain

202. Based on the values obtained in step 201

Calculating the weight value by the formula (22)

Wherein M is a positive integer.

Calculating the weight value by the formula (23)

Wherein l is the weight

Any possible value of (a);

203. judgment of

If yes, the process goes to step 204, otherwise, j is made j +1, the process returns to step 201, and epsilon is a given threshold;

204、

and the control quantity is an approximate value of the optimal control quantity, and then the servo system is controlled in real time by using the control quantity.

Advantageous effects

The method comprises state error feedback control and adaptive optimization control based on strategy iteration, wherein the state error feedback control can utilize sampling data to calculate feedback gain K in real time; the adaptive optimization controller directly uses the control quantity and the state error quantity obtained by the state error feedback control to carry out iterative computation and gradually approaches to the optimal control quantity; the method does not need to know a specific system model, and only needs a measurable state error amount.

Drawings

FIG. 1 is a flow chart of an optimal control method for a data-driven servo system.

Fig. 2 is a graph of initial feedback gain adjustment.

FIG. 3 is a comparison graph of MATLAB simulation cost functions.

FIG. 4 is a MATLAB simulation state trace comparison diagram.

FIG. 5 is a comparison graph of MATLAB simulation state trajectories obtained by applying two control quantities to the system in real time after the calculation is completed.

Detailed Description

The invention is described in detail below with reference to the figures and the specific examples.

The mathematical model of a certain dc motor servo system can be written as:

wherein x (t) ═ x₁(t),x₂(t)]^TThe state quantity is measurable, and the state quantity is measurable; u is a control quantity; f (x, u) is unknown, satisfying that F (0,0) ═ 0.

The invention designs an optimal control method to enable a motor servo system to reach an expected state x_d＝[x_1d,0]^TI.e. tracking the step signal, and at this time u _d0. The state error is defined as: e ═ x-x_dAnd e₀＝x₀-x_d. The error system equation is:

the invention finally controls the state error to converge to 0.

For optimal control, the cost function needs to be defined as follows:

wherein r (e, u) ═ Q (e) + u^TRu, Q are positive definite, and R is a positive definite symmetric matrix. The following assumptions are made for this cost function:

assume that 1: the value of the cost function (3) is related to the control quantity u, for each e₀There is a unique minimum.

Assume 2: cost function (3) satisfies for arbitrary piecewise continuous control input u

The optimal control system structure of the data-driven servo system is shown in fig. 1, and comprises a state error feedback controller and a self-adaptive optimization controller, which are respectively explained in detail below.

1. A state error feedback controller.

For unknown servo systems, system information is implicit in state errors and input information, and the following data relation can be established through system information sampling:

e((k+1)T)＝g(e(kT),u(kT)) (4)

where g (e, u) is unknown, k is 0,1,2, T is the sampling period, omitting T may simplify equation (4) to:

e (k +1) ═ g (e (k)), u (k)) (5) this equation contains the state error and the input signal, revealing the internal characteristics of the system (2) and can be used directly to search for the appropriate feedback control quantity. Since the system (2) and the system (4) represent the same system, g (0,0) can be obtained as 0.

To make the state error reach 0, applying lagrange's mean theorem to equation (5) at any point e (k) and 0, we can get:

e(k+1)＝g_e(k)e(k)+g_u(k)u(k) (6)

whereinTheta is more than 0 and less than 1. In this regard, the following assumptions are made:

assume that 3: for any k ∈ N, at each time interval [ kT, (k +1) T]Upper, matrix g_u(k) And g_e(k) Remain unchanged.

To ensure that this assumption holds, a sufficiently small sampling period T should be chosen.

Dividing a time interval [ kT, (K +1) T ] into two parts, applying a feedback gain K (K-1) in a first part [ kT, (K +1/2) T ], and applying a feedback gain K (K) in a second part, namely designing a state error feedback control quantity as follows:

from assumptions 3 and equation (6), the state error relationship can be obtained as follows:

e((k+1/2)T)＝[g_e(k)vg_u(k)K(k-1)]e(kT)＝A₁(k)e(kT) (8)

e((k+1)T)＝[g_e(k)+g_u(k)K(k)]e((k+1/2)T)＝A₀(k)e((k+1/2)T) (9)

thus, it is possible to obtain:

e(k+1)＝A₀(k)A₁(k)e(k)＝A(k)e(k) (10)

obviously, the state error e can be converged to 0 by adjusting a (k), and a (k) can be adjusted by adjusting the feedback gain k (k), defining k (k) adjustment rate as follows:

wherein lambda is the step length, lambda is more than 0 and less than 1, a smaller value should be selected to ensure e convergence, but not too small to ensure the search efficiency; the symbol | | · | |, represents the Frobenius norm; the symbol + represents the pseudo-inverse of the matrix. When | | | A (K) | | > 1, the feedback gain K is constantly adjusted according to the formula (11), wherein the required real-time matrix g_u(k) And g_e(k) The following method can be used for the calculation.

State error relationship matrix A₁(k) And A₀(k) The measured state error can be determined by equations (8) and (9):

A₁(k)＝e((k+1/2)T)(e(kT))⁺(12)

A₀(k)＝e((k+1)T)(e((k+1/2)T))⁺(13)

then, it is possible to obtain:

g_u(k)＝(A₀(k)-A₁(k))(K(k)-K(k-1))⁺(14)

g_e(k)＝A₀(k)-g_u(k)·K(k) (15)

when | | | A (k) | | ≧ 1, the feedback gain K (k) is continuously updated in real time according to the equations (11) to (15); when | | a (K) | is < 1, K remains unchanged. In the process, the state is wrongThe difference feedback control quantity is continuously adjusted while being applied, and the obtained input u⁰And the state error data e can be used for the next part of the adaptive optimization controller.

2. An adaptive optimization controller.

And by adopting the data generated in the last process, the part of the adaptive optimization controller adopts an adaptive dynamic programming strategy iteration method to solve the optimal control problem of the servo system.

For the above state error system, the basic strategy iteration method is as follows:

(1) solving for V from^j(e(t))：

r(e，u^j(e))+(V_e ^j(e))^T(f(e，u^j(e)))＝0，V^j(0) Where j is the number of iterations (0), (16).

(2) The control amount is calculated by the following formula:

selecting a time series t_kK is 0,1, so that t is more than 0₁＜t₂＜...＜t_M< ∞, wherein M is a sufficiently large integer. As can be seen from the cost function definition (3):

can be converted into:

the following functional structure is used to approximate the unknown V in the above formula^j(e)，

And u^j(e) Respectively expressed as:

wherein

Is the weight; phi is a_i(e),(i＝1,...N_w)，ψ_i(e,u),(i＝1,...N_c)，σ_i(e),(i＝1,...N_l) Is a basis function;respectively approximating three functions V by a function structure^j(e)，

And u^j(e) The approximation error of (2). Equation (18) can be written as:

wherein

Is the approximation error;

for convenience, the following definitions are made:

φ_ki＝φ_i(e(t_k))-φ_i(e(t_k-1))

then (19) can be abbreviated as:

without loss of generality, the following assumptions were made:

assume 4: for all u^jExistence of

And γ > 0, so that for allThe following inequality holds:

therefore, the weight matrix can be solved according to equation (20) by:

according to the formula (17), the weight can be obtained by solving

Using the existing sampling data, continuously iterating and calculating by the formulas (22) and (23) when t and N are_w、N_cAnd N_lWhen the weights are approaching infinity, the respective optimal values of the weight convergence, i.e., the unknown functions converge to the optimal values thereof.

In summary, the optimal control method of the data-driven servo system is as follows:

a first part: data driven state error feedback control

101. Setting K to 1, and initializing system feedback gains K (K-1) and K (K) at the K-1 and K-th moments;

102 applying the system feedback gains K (K-1) and K (K) to the servo system in sequence in a time interval [ kT, (K +1) T ];

103. sampling the state error of the servo system, and calculating a state error relation matrix A (k), g according to the expressions (10), (12) to (15)_e(k) And g_u(k)；

A(k)＝A₀(k)A₁(k) (10)

A₁(k)＝e((k+1/2)T)(e(kT))⁺(12)

A₀(k)＝e((k+1)T)(e((k+1/2)T))⁺(13)

Then, it is possible to obtain:

g_u(k)＝(A₀(k)-A₁(k))(K(k)-K(k-1))⁺(14)

g_e(k)＝A₀(k)-g_u(k)·K(k) (15)

wherein e (kT) and e ((k +1/2) T) represent the system state error at the time of k and (k +1/2) and are represented by the symbol ·⁺Represents a pseudo-inverse of the matrix;

104. calculating K (K +1) and K (K +2) according to equation (11);

wherein, lambda represents a set step size and is a constant between 0 and 1

105. And judging whether k is smaller than a set threshold value, namely whether the k-th time reaches a set cut-off time T1, if so, entering the second part, otherwise, making k equal to k +1, and returning to the step 102.

After the first part is executed, a series of system state errors e (kT), e ((k +1/2) T) and a system feedback control input u are obtained⁰An adaptive optimization controller for the next section.

A second part: data-driven adaptive optimization control

Setting j to 0, define phi_ki、

u^j，

Is expressed as follows

φ_ki＝φ_i(e(t_k))-φ_i(e(t_k-1))

Wherein j is the number of iterations; phi is a_i(e),(i＝1,...N_w)，ψ_i(e,u),(i＝1,...N_c)，σ_i(e),(i＝1,...N_l) Is a basis function;

is the weight;is u^j(e) Is determined by the estimated value of (c),

as a function u^j(e) An approximation error between the actual value and the estimated value.

201. Calculating phi based on the system state error obtained in the first part_kiAndthen obtain

202. Based on the values obtained in step 201

Calculating the weight value by the formula (22)

Wherein M is a positive integer.

Calculating the weight value by the formula (23)

Wherein l is the weight

Any possible value of (a).

203. Judgment of

If yes, go to step 204, otherwise, let j equal to j +1, return to step 201; each time a return is made, the value calculated in the loop 202 is calculated

Substituting for the calculation in the next loop step 201, ε is the given threshold.

204、

And the control quantity is an approximate value of the optimal control quantity, and then the servo system is controlled in real time by using the control quantity.

The flow chart of the whole process is shown in fig. 2.

And (5) simulating the method by utilizing matlab. Consider a motor servo system with the following second order model:

the model parameters of the system are set as: theta₁＝-2,θ₂＝-5,θ₃＝1,θ₄＝0and K_s＝900。

The cost function is defined as:

the system initial values are set as: x is the number of₀＝[-1 0]^T，K(0)＝[-1 0]，K(1)＝[0-1]。

The first part employs a state error feedback controller, and the phase is set to 1.5 s. Fig. 3 shows a graph of the adjustment of the feedback gain K in this process.

And the second part adopts a self-adaptive optimization controller, directly utilizes the data obtained by the first part, uses a polynomial basis function, and obtains the final approximate optimal control quantity after 17 iterations. Fig. 4 is a graph comparing the initial controlled variable (obtained in the first section) and the cost function corresponding to the approximate optimal controlled variable, and it is obvious that the cost function of the approximate optimal controlled variable is small. Fig. 5 is a comparison diagram of state trajectories obtained by applying two control quantities to the system in real time after the calculation is completed, and it is obvious that the system has a faster convergence rate by applying the approximate optimal control quantity.

In summary, the above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (1)

1. A data-driven servo system optimal control method is suitable for controlling a direct current motor servo system, and a mathematical model of a certain direct current motor servo system can be written as follows:

x(0)＝x₀

wherein x (t) ═ x₁(t),x₂(t)]^TRespectively the output angular position of the motor and the angular speed of the rotor; u is a control quantity; f (x, u) is unknown, satisfying that F (0,0) is 0;

the state error is defined as:

e＝x-x_d

wherein x is_dThe motor servo system reaches an expected state, and x is an output state of the motor servo system and comprises an angular position and a rotor angular speed output by the motor servo system;

the error system equation is:

finally, the error of the control state converges to 0;

for optimal control, the cost function is defined as follows:

wherein r (e, u) ═ Q (e) + u^TRu, Q are positive definite, R is a positive definite symmetric matrix;

assume that 1: the cost function value is related to the control quantity u, for each e₀Has a unique minimum value;

assume 2: cost function satisfied for arbitrary piecewise continuous control input u

Aiming at the direct current motor servo system, the method is characterized by comprising two parts of data-driven state error feedback control and data-driven adaptive optimization control:

a first part: data driven state error feedback control

101. Setting K to 1, and initializing system feedback gains K (K-1) and K (K) at the K-1 and K-th moments;

102. sequentially applying the system feedback gains K (K-1) and K (K) to the servo system over a time interval [ kT, (K +1) T ];

103. sampling the state error of the servo system, and calculating a state error relation matrix A (k), g according to the expressions (10), (12) to (15)_e(k) And g_u(k)；

A(k)＝A₀(k)A₁(k) (10)

A₁(k)＝e((k+1/2)T)(e(kT))⁺(12)

A₀(k)＝e((k+1)T)(e((k+1/2)T))⁺(13)

Then, it is possible to obtain:

g_u(k)＝(A₀(k)-A₁(k))(K(k)-K(k-1))⁺(14)

g_e(k)＝A₀(k)-g_u(k)·K(k) (15)

wherein e (kt) and e ((k +1/2) T) represent the system state error at the k-th time and (k +1/2) th time, and the symbol · + represents the pseudo-inverse of the matrix;

104. calculating K (K +1) according to equation (11);

wherein, λ represents a set step length and is a constant between 0 and 1; the symbol | | | | represents the Frobenius norm of the matrix;

105. judging whether k is smaller than a set threshold value, if so, entering a second part, otherwise, making k equal to k +1, and returning to the step 102;

a second part: data-driven adaptive optimization control

Setting j to 0, define phi_ki、

u^j，

Is expressed as follows

φ_ki＝φ_i(e(t_k))-φ_i(e(t_k-1))

Wherein j is the number of iterations; phi is a_i(e),i＝1,...N_w，ψ_i(e,u),i＝1,...N_c，σ_i(e),i＝1,...N_lIs a basis function;

is the weight; u. of⁰Feedback control quantity for state error;

is u^j(e) Is determined by the estimated value of (c),

as a function u^j(e) An approximation error between the actual value and the estimated value;

201. calculating phi based on the system state error obtained in the first part_kiAnd

then obtain

202. Based on the values obtained in step 201

Calculating the weight value by the formula (22)

Wherein M is a positive integer, r (e, u) ═ Q (e) + u^TRu, Q are positive definite, R is a positive definite symmetric matrix;

calculating the weight value by the formula (23)

Wherein l is the weight

Any possible value of (a);

203. judgment of

If yes, the process goes to step 204, otherwise, j is made j +1, the process returns to step 201, and epsilon is a given threshold;

204、

and the control quantity is an approximate value of the optimal control quantity, and then the servo system is controlled in real time by using the control quantity.

Images