CN103971163A

CN103971163A - Adaptive learning rate wavelet neural network control method based on normalization lowest mean square adaptive filtering

Info

Publication number: CN103971163A
Application number: CN201410195894.4A
Authority: CN
Inventors: 袁赣南; 杜雪; 张瑶; 夏庚磊; 吴迪; 李旺; 贾韧锋; 常帅
Original assignee: Harbin Engineering University
Current assignee: Harbin Engineering University
Priority date: 2014-05-09
Filing date: 2014-05-09
Publication date: 2014-08-06
Anticipated expiration: 2034-05-09
Also published as: CN103971163B

Abstract

The invention relates to the technical field of wavelet neural network optimization, in particular to an adaptive learning rate wavelet neural network control method based on normalized least mean square adaptive filtering. The invention includes: establishing a control system model; unitizing the ownership values of the wavelet network by layers; optimizing the weight of wavelet neurons; solving error signals and training costs; The fuzzy rules of the combined derivative function; determine the membership function; determine the proportion of each fuzzy rule in the value of the derivative function; output the fuzzy system, linearize and display the activation function; determine the induced local domain and neuron output of each neuron; solve Each local gradient function; the output layer adaptively adjusts the learning rate; determines the range of the output layer learning rate; adjusts the learning rate of the hidden layer; trains neuron synaptic weights; outputs tracking control signals; completes closed-loop feedback control. The invention can speed up the convergence speed and reduce the computational complexity.

Description

An Adaptive Learning Rate Wavelet Neural Network Control Method Based on Normalized Least Mean Square Adaptive Filtering

技术领域technical field

本发明涉及小波神经网络优化技术领域，特别涉及一种基于归一化最小均方自适应滤波的自适应学习率小波神经网络控制方法。The invention relates to the technical field of wavelet neural network optimization, in particular to an adaptive learning rate wavelet neural network control method based on normalized least mean square adaptive filtering.

背景技术Background technique

复杂系统多具有不确定性，其系统内部的非线性函数是难以建立的，因此无法采用基于系统结构的方法实现对复杂系统的跟踪控制。而人工神经网络是由人工神经元互联而成的网络系统，它从微观结构和功能上对人脑进行了抽象和简化，可以看作是一个由简单处理单元构成的规模宏大的高度并行处理器，天然具有存储经验知识和使之可用的特性。神经网络与人脑的相似处在于，神经网络所获取的知识是从外界环境学习而来，同时相互连接的神经元间的连接权值用于存储获得的知识。在处理计算上，虽然每个处理单元的功能看似简单，但大量简单处理单元的并行活动使网络在保证较快速度的前提下呈现出丰富的功能，加之神经网络的自适应能力为解决复杂的非线性、不确定、不确知系统的问题开辟了新途径，所以目前人工神经网络已受到非线性系统辨识与分析、控制系统和计算机等许多领域的青睐，并得到广泛应用。在神经网络的研究中，作为前向多层神经网络训练方法的BP算法应用最为广泛，但其还存在一些缺陷，例如学习算法所得到的误差是高维权向量的复杂非线性函数，易陷入局部极小值。小波神经网络(简称小波网络)是建立在小波分析理论基础上的新型神经网络，在隐含层中采用小波函数代替传统神经网络函数作为激励函数，并通过仿射变换建立起小波变换与网络系数之间的连接，结合了小波分析和神经网络的优点，具有很强的学习和泛化能力，在控制中具有静态非线性映射以及动态处理的优势，使其在许多具有非线性、强耦合的复杂系统中也逐步被采纳。然而目前小波网络在复杂系统或大规模网络中还存在跟踪速度慢等问题。实际上这种问题是使用随机梯度的算法在进行权值迭代更新时普遍存在的，其研究成果主要体现在自适应滤波中的最小均方自适应滤波(LMS)及归一化最小均方自适应滤波(NLMS)，但在神经网络领域鲜有这类的研究成果。根据Simon Haykin的理论，LMS和NLMS都可视为建立一个简单线性神经元来设计未知动态系统的一个多输入——单输出模型。由此可知小波网络是具有更复杂拓扑结构的自适应滤波器，若对小波网络施以改进，则可使NLMS或LMS具有应用在小波网络中的可能。Complex systems are often uncertain, and the nonlinear functions inside the system are difficult to establish, so the method based on system structure cannot be used to realize the tracking control of complex systems. The artificial neural network is a network system formed by the interconnection of artificial neurons. It abstracts and simplifies the human brain from the microstructure and function, and can be regarded as a large-scale highly parallel processor composed of simple processing units. , naturally has the characteristics of storing empirical knowledge and making it available. The similarity between the neural network and the human brain is that the knowledge acquired by the neural network is learned from the external environment, and the connection weights between the interconnected neurons are used to store the acquired knowledge. In terms of processing and computing, although the function of each processing unit seems simple, the parallel activities of a large number of simple processing units make the network present rich functions under the premise of ensuring a relatively fast speed. Therefore, artificial neural networks have been favored by many fields such as nonlinear system identification and analysis, control systems and computers, and have been widely used. In the study of neural networks, the BP algorithm is the most widely used as a forward multi-layer neural network training method, but it still has some defects. For example, the error obtained by the learning algorithm is a complex nonlinear function of high-dimensional weight vectors, which is easy to fall into local minimum value. Wavelet neural network (abbreviated as wavelet network) is a new type of neural network based on wavelet analysis theory. In the hidden layer, wavelet function is used instead of traditional neural network function as the excitation function, and wavelet transformation and network coefficients are established through affine transformation. The connection between them, combining the advantages of wavelet analysis and neural network, has strong learning and generalization capabilities, and has the advantages of static nonlinear mapping and dynamic processing in control, making it suitable for many nonlinear and strongly coupled applications. It is also gradually adopted in complex systems. However, wavelet networks still have problems such as slow tracking speed in complex systems or large-scale networks. In fact, this kind of problem is common when the algorithm using stochastic gradient is used for iterative updating of weights. The research results are mainly reflected in the least mean square adaptive filtering (LMS) and normalized least mean square auto- Adaptive filtering (NLMS), but there are few such research results in the field of neural networks. According to Simon Haykin's theory, both LMS and NLMS can be regarded as building a simple linear neuron to design a multi-input-single-output model of an unknown dynamic system. It can be seen that the wavelet network is an adaptive filter with a more complex topology. If the wavelet network is improved, the NLMS or LMS can be applied to the wavelet network.

LMS与NLMS在权值推导过程中的局限性主要表现在它们只适用于线性结构，小波网络的非线性活化函数会使这一过程变得十分复杂。而这一问题早已引起研究神经网络硬件实现的学者们的关注，其中学者Emilio Soria-Olivas在《A Low-Complexity Fuzzy ActivationFunction for Artificial Neural Networks》中采用模糊理论对神经网络的活化函数进行了局部线性回归计算。而相比与传统模糊算法，Takagi和Sugeno于1985年提出的T-S模糊模型的规则后件通常是输入变量的线性函数，这样每条规则就可以包含许多的信息，所以采用较少的规则便可达到控制效果，这就意味着可以使用相对简单的方法来对非线性函数进行回归拟合。因此本发明采用T-S模糊模型对小波网络的活化函数进行局部线性回归拟合，这种方法的结果与原函数具有很高的拟合精度，也从而克服了小波网络复杂的非线性活化函数所带来的计算困难。此外，小波网络还存在多输入多输出以及多层拓扑结构的问题，这些问题使自适应学习率不再像NLMS滤波器那样可以独立的调节，为此本发明提出一种基于NLMS的自适应学习率小波网络控制方法，从而在控制的初始阶段降低了系统误差，提高了控制过程的收敛速度和稳定性。The limitations of LMS and NLMS in the weight derivation process are mainly manifested in that they are only suitable for linear structures, and the nonlinear activation function of wavelet network will make this process very complicated. This problem has already attracted the attention of scholars who study the hardware implementation of neural networks. Among them, the scholar Emilio Soria-Olivas used fuzzy theory in "A Low-Complexity Fuzzy Activation Function for Artificial Neural Networks" to perform local linearization on the activation function of neural networks. regression calculation. Compared with the traditional fuzzy algorithm, the rule consequence of the T-S fuzzy model proposed by Takagi and Sugeno in 1985 is usually a linear function of the input variable, so that each rule can contain a lot of information, so fewer rules can be used. To achieve the control effect, this means that a relatively simple method can be used to perform regression fitting on nonlinear functions. Therefore the present invention adopts T-S fuzzy model to carry out local linear regression fitting to the activation function of wavelet network, the result of this method has very high fitting precision with original function, also thereby overcomes the complex nonlinear activation function of wavelet network. It is difficult to calculate. In addition, the wavelet network also has the problems of multi-input, multi-output and multi-layer topology. These problems make the adaptive learning rate no longer independently adjustable like the NLMS filter. For this reason, this invention proposes an adaptive learning based on NLMS The rate wavelet network control method reduces the system error in the initial stage of control and improves the convergence speed and stability of the control process.

发明内容Contents of the invention

本发明的目的在于提供一种降低系统误差，提高控制过程的收敛性和稳定性，并减小计算复杂度，摆脱了原有固定学习率带来的冗余困扰，避免了发散的问题，提高小波网络在复杂系统控制中的跟踪效率的一种基于归一化最小均方自适应滤波的自适应学习率小波神经网络控制方法。The purpose of the present invention is to provide a method for reducing system errors, improving the convergence and stability of the control process, reducing the computational complexity, getting rid of the redundant troubles brought by the original fixed learning rate, avoiding the problem of divergence, and improving An adaptive learning rate wavelet neural network control method based on normalized least mean square adaptive filtering based on wavelet network tracking efficiency in complex system control.

本发明的目的是这样实现的：The purpose of the present invention is achieved like this:

(1)建立控制系统模型:采用小波网络对增强型PID控制器进行参数整定，令小波网络为MIMO的多层反馈网结构，各神经元函数为活化函数，神经网络的状态空间为：(1) Establish the control system model: use the wavelet network to adjust the parameters of the enhanced PID controller, make the wavelet network a MIMO multi-layer feedback network structure, each neuron function is an activation function, and the state space of the neural network is:

$\{\begin{matrix} {W W}_{k k} = = {W W}_{k k - - 11} + + {φ φ}_{k k} \\ {Z Z}_{k k} = = ψ ψ (({W W}_{k k},, {U u}_{k k})),, \end{matrix}$

其中W_k为权值空间，U_k为网络输入，Z_k为网络输出，φ_k为权值更新函数，ψ(W_k,U_k)为参数化的非线性函数，小波网络的权值空间为W_k，将权值空间中的每个权值生成[-1,1]区间上均匀分布的随机数；Where W _k is the weight space, U _k is the network input, Z _k is the network output, φ _k is the weight update function, ψ(W _k , U _k ) is a parameterized nonlinear function, the weight space of wavelet network For W _k , each weight in the weight space generates a uniformly distributed random number on the [-1,1] interval;

(2)取[-1,1]区间上均匀分布的随机数为权值初始值，并将小波网络的所有权值按层进行单位化；(2) Take the random number evenly distributed on the [-1,1] interval as the initial weight value, and unitize the weight value of the wavelet network by layer;

(3)小波神经元权值优化：以小波函数为激励函数的神经元为中心，将前后两个网络层中的权值分别与小波函数类型、神经元个数进行关联，设第J层中的激励函数为小波函数，I、K分别为J层前后的两层，W_LM、W_MN为单位化后三层间的两个权值矩阵，则将小波函数类型和神经元个数与其关联的表达式为：(3) Wavelet neuron weight optimization: centering on the neuron with the wavelet function as the excitation function, correlate the weights in the front and rear two network layers with the wavelet function type and the number of neurons respectively, and set The activation function of is the wavelet function, I and K are the two layers before and after the J layer respectively, W _LM and W _MN are the two weight matrices between the three layers after unitization, then the type of the wavelet function and the number of neurons are associated with it The expression is:

${W W}_{IJ IJ} = = {K K}_{J J} \cdot &Center Dot; {W W}_{IJ IJ} \sqrt[J J]{I I}$

${W W}_{JK JK} = = {K K}_{J J} \cdot &Center Dot; {W W}_{JK JK} \sqrt[K K]{J J},,$

其中K_J为常值；Where K _J is a constant value;

(4)引入训练样本集{x(n),norm(n)}：依次输入向量x(1)，x(2)……x(n)，记录网络输出z(1)，z(2)……z(n)，求解误差信号e(n)和训练代价ε(n)：(4) Introduce the training sample set {x(n), norm(n)}: input vectors x(1), x(2)...x(n) in sequence, and record network output z(1), z(2) ... z(n), solve the error signal e(n) and the training cost ε(n):

e(n)＝norm(n)-z(n)e(n)=norm(n)-z(n)

$ϵ ϵ ((n no)) = = \frac{11}{22} {e e}^{22} ((n no))$

(5)采用阶梯函数对活化函数的导函数分段处理：将函数分为M段，对活化函数进行拟合，活化函数的每段的斜率对应导数的函数值；(5) Step function is used to segment the derivative function of activation function: the function is divided into M segments, the activation function is fitted, and the slope of each segment of the activation function corresponds to the function value of the derivative;

(6)制定拟合导函数的模糊规则：T-S模型的输入变量：(6) Formulate fuzzy rules for fitting derivative functions: input variables of T-S model:

$x x = = [[{x x}_{11},, {x x}_{22}]] &DoubleRightArrow; &DoubleRightArrow; \{\begin{matrix} {x x}_{11} = = x x ((n no)) \\ {x x}_{22} = = x x ((n no)) - - {b b}^{m m} \end{matrix}$

输出量为导函数值k(n)，其模糊规则形式为：The output quantity is the derivative function value k(n), and its fuzzy rule form is:

${R R}^{i i} : : if if {x x}_{11} is is {A A}_{11}^{i i} and and {x x}_{22} is is {A A}_{22}^{i i},, then then {k k}_{i i} = = {p p}_{i i} {x x}_{11} + + {q q}_{i i} {x x}_{22} + + {r r}_{i i} ((i i = = 1,2 1,2,, . . . . . .,, c c)),,$

其中和表示第i条规则中的模糊集合，b^m代表第m段(m＝1,2,...,M)的左边界，p_i、q_i和r_i是模糊集合的常数；in and Represents the fuzzy set in the i-th rule, b ^m represents the left boundary of the m-th segment (m=1,2,...,M), p _i , q _i and r _i are the constants of the fuzzy set;

(7)确定隶属函数：采用高斯型函数作为隶属函数，各输入变量x_j的隶属度为：(7) Determine the membership function: Gaussian function is used as the membership function, and the membership degree of each input variable x _j is:

$\begin{matrix} {μ μ}_{{A A}_{j j}^{i i}} (({x x}_{j j})) = = exp exp ((- - {((\frac{{x x}_{j j} - - {c c}_{j j}^{i i}}{{σ σ}_{j j}^{i i}}))}^{22})) & j j = = 1,2,3 1,2,3;; i i = = 1,2 1,2,, . . . . . .,, M m \end{matrix}$

式中和分别为隶属度函数的中心和宽度；In the formula and are the center and width of the membership function, respectively;

(8)确定每个模糊规则在导函数值中所占的比重：每条模糊规则对于输入量x＝[x₁,x₂]的适用度μ_i及其激活度为：(8) Determine the proportion of each fuzzy rule in the value of the derivative function: the applicability μ _i and activation degree of each fuzzy rule to the input quantity x=[x ₁ ,x ₂ ] for:

${μ μ}_{i i} = = {Π Π}_{j j = = 11}^{33} {μ μ}_{{A A}_{j j}^{i i}} (({x x}_{j j}))$

${\overset{^^}{μ μ}}_{i i} = = \frac{{μ μ}_{i i}}{{Σ Σ}_{i i = = 11}^{c c} {μ μ}_{i i}};;$

(9)输出T-S模糊系统、线性化显示活化函数：活化函数的线性形式：(9) Output T-S fuzzy system, linearization display activation function: linear form of activation function:

$s the s ((v v ((n no)))) = = \{\begin{matrix} a a,, & v v ((n no)) > > {θ θ}_{11} \\ k k ((n no)) v v ((n no)) + + d d ((n no)),, & {θ θ}_{22} < < v v ((n no)) < < {θ θ}_{11} \\ b b,, & v v ((n no)) < < {θ θ}_{22} \end{matrix},,$

s(x)为线性形式的活化函数，a和b分别为函数的左右边界，θ₁和θ₂分别为边界的阈值，k(n)和d(n)为线性区域系数，λ为常值系数；s(x) is the activation function in linear form, a and b are the left and right boundaries of the function respectively, θ ₁ and θ ₂ are the thresholds of the boundary respectively, k(n) and d(n) are linear region coefficients, and λ is a constant value coefficient;

(10)确定各神经元的诱导局部域及神经元输出：诱导局部域与神经元j的输出信号分别为：(10) Determine the induced local domain and neuron output of each neuron: the induced local domain and the output signal of neuron j are respectively:

${v v}_{j j} ((n no)) = = {Σ Σ}_{i i = = 11}^{I I} {w w}_{ij ij} {x x}_{i i} ((n no)) = = {X x}_{i i}^{T T} ((n no)) {W W}_{ij ij} ((n no))$

其中v_j(n)为诱导局部域，w_ij为权值，x_i(n)为上层神经元输出，W_ij和X_i(n)分别为w_ij和x_i(n)构成的向量，I为上层神经元总数，为j层的活化函数；Where v _j (n) is the induced local domain, w _ij is the weight, xi (n) is the output of the neurons in the upper layer, W _ij _and _Xi (n) are the vectors composed of w _ij and _xi (n), respectively, I is the total number of neurons in the upper layer, is the activation function of layer j;

(11)求解各个局部梯度函数δ_j(n)，局部梯度δ_j(n)为：(11) Solving each local gradient function δ _j (n), the local gradient δ _j (n) is:

步骤(9)的活化函数线性化后，函数δ_jL(n)的线性化表示为：After the activation function of step (9) is linearized, the linearization of the function δ _jL (n) is expressed as:

(12)输出层自适应调整学习率：自适应学习率为：(12) The output layer adaptively adjusts the learning rate: the adaptive learning rate is:

$\begin{matrix} {μ μ}_{k k} ((n no + + 11)) = = g g (({x x}_{11} ((n no)),, {x x}_{22} ((n no)) {. . . . . . x x}_{J J} ((n no)),, {k k}_{k k} ((n no)))) \\ = = 11 / / [[{k k}_{k k}^{22} ((n no)) (({x x}_{11}^{22} ((n no)) + + {x x}_{22}^{22} ((n no)) + + . . . . . . + + {x x}_{j j}^{22} ((n no)))) + + {σ σ}_{v v}^{22}]],, \end{matrix}$

其中k_k(n)＝s′_k(v_k(n))＝常数c_k，σ_v(0＜σ_v＜1)；Where k _k (n)=s′ _k (v _k (n))=constant c _k , σ _v (0<σ _v <1);

(13)确定输出层学习率的范围(13) Determine the range of the learning rate of the output layer

阈值：Threshold:

${μ μ}_{k k} ((n no + + 11)) = = \{\begin{matrix} {μ μ}_{max max} & {μ μ}_{k k} > > {μ μ}_{max max} \\ g g (({x x}_{11} ((n no)) {,, x x}_{22} ((n no)) . . . . . . {x x}_{J J} ((n no)),, {k k}_{k k} ((n no)))) & {μ μ}_{min min} < < {μ μ}_{k k} < < {μ μ}_{max max} \\ {μ μ}_{min min} & {μ μ}_{k k} < < {μ μ}_{min min} \end{matrix};;$

(14)隐层的学习率调节：对隐层的每个神经元均采用相同的学习率：(14) Learning rate adjustment of the hidden layer: the same learning rate is used for each neuron in the hidden layer:

${μ μ}_{j j} ((n no + + 11)) = = \frac{11}{k k} {Σ Σ}_{k k = = 11}^{K K} {μ μ}_{k k} ((n no)),,$

其中μ_j表示隐层第j个神经元的学习率，K为输出层神经元个数；Where μ _j represents the learning rate of the jth neuron in the hidden layer, and K is the number of neurons in the output layer;

(15)训练神经元突触权值：引入步骤(12)和步骤(14)的自适应学习率，局部梯度采用非线性的δ(n)：(15) Training neuron synaptic weights: introduce the adaptive learning rate of step (12) and step (14), and the local gradient adopts nonlinear δ(n):

w(n+1)＝w(n)+Δw(n)w(n+1)=w(n)+Δw(n)

Δw(n)＝μ(n)δ(n)x(n)；Δw(n)=μ(n)δ(n)x(n);

(16)循环次数加1，返回步骤(10)，直至满足停止准则，输出跟踪控制信号；(16) cycle number of times adds 1, returns to step (10), until satisfying stop criterion, output tracking control signal;

(17)将控制信号输入执行机构并与系统进行计算融合，输出被控参数值，与预期量进行比较，完成闭环反馈控制。(17) Input the control signal into the actuator and perform calculation and fusion with the system, output the value of the controlled parameter, compare it with the expected value, and complete the closed-loop feedback control.

本发明的有益效果在于：The beneficial effects of the present invention are:

本发明能够自适应的实时调节学习率，从而加快收敛速度，减小计算复杂度，同时具有更加平滑的迭代曲线。与固定学习率相比，能够提高学习效率。与基于LMS算法的自适应滤波方法相比，本发明采用基于NLMS思想对学习率调节更具有针对性，从而加快算法收敛速度，同时摆脱了原有学习率调节方法带来的冗余困扰，避免了发散的问题，提高控制系统跟踪效率。The invention can self-adaptively adjust the learning rate in real time, so as to speed up the convergence speed, reduce the computational complexity, and have a smoother iteration curve at the same time. Compared with the fixed learning rate, it can improve the learning efficiency. Compared with the adaptive filtering method based on the LMS algorithm, the present invention adopts the idea of NLMS to adjust the learning rate more specifically, thus speeding up the convergence speed of the algorithm, and at the same time getting rid of the redundant trouble caused by the original learning rate adjustment method, avoiding The problem of divergence is solved, and the tracking efficiency of the control system is improved.

附图说明Description of drawings

图1：基于NLMS自适应学习率的小波网络结构图；Figure 1: Wavelet network structure diagram based on NLMS adaptive learning rate;

图2：技术方案流程图；Figure 2: Flowchart of the technical solution;

图3：复杂系统的小波网络跟踪控制原理图；Figure 3: Schematic diagram of wavelet network tracking control for complex systems;

图4：活化函数及导函数；Figure 4: Activation function and derivative function;

图5：T-S模糊推理后活化函数的导函数拟合效果图；Figure 5: The fitting effect diagram of the derivative function of the activation function after T-S fuzzy inference;

图:6：神经元j的输入信号流图；Figure: 6: Input signal flow diagram of neuron j;

图7：非线性函数(1)的跟踪控制拟合曲线与误差信号；Fig. 7: Tracking control fitting curve and error signal of nonlinear function (1);

图8：非线性函数(2)的跟踪控制拟合曲线与误差信号。Figure 8: Tracking control fitting curve and error signal of nonlinear function (2).

具体实施方式Detailed ways

下面结合附图对本发明做进一步描述。The present invention will be further described below in conjunction with the accompanying drawings.

本发明采用基于NLMS思想对学习率调节更具有针对性，这种方法在权值更新过程中可实时更新学习率，从而降低系统误差，提高控制过程的收敛性和稳定性，并减小计算复杂度，摆脱了原有固定学习率带来的冗余困扰，避免了发散的问题，提高小波网络在复杂系统控制中的跟踪效率。本发明所述的自适应调节学习率方法在小波网络在线学习平台中进行，本发明实施方式主要包含以下关键步骤：The present invention adopts the idea based on NLMS to adjust the learning rate more specifically. This method can update the learning rate in real time during the weight update process, thereby reducing system errors, improving the convergence and stability of the control process, and reducing computational complexity. degree, get rid of the redundant trouble caused by the original fixed learning rate, avoid the problem of divergence, and improve the tracking efficiency of wavelet network in complex system control. The self-adaptive adjustment learning rate method of the present invention is carried out in the wavelet network online learning platform, and the embodiment of the present invention mainly comprises the following key steps:

步骤1 建立控制系统模型，采用小波网络对增强型PID控制器进行参数整定，令小波网络为MIMO的多层反馈网结构，各神经元函数为活化函数(如sigmoid函数)，其中小波函数取具有连续可微性质的函数(如Morlet函数)，设置循环停止准则。神经网络的状态空间模型可表述为：Step 1. Establish the control system model, use the wavelet network to tune the parameters of the enhanced PID controller, make the wavelet network a MIMO multi-layer feedback network structure, and each neuron function is an activation function (such as a sigmoid function), where the wavelet function takes Continuously differentiable functions (such as Morlet functions), set the loop stop criterion. The state space model of neural network can be expressed as:

$\{\begin{matrix} {W W}_{k k} = = {W W}_{k k - - 11} + + {φ φ}_{k k} \\ {Z Z}_{k k} = = ψ ψ (({W W}_{k k},, {U u}_{k k})) \end{matrix}$

其中W_k为权值空间，U_k为网络输入，Z_z为网络输出，φ_k为权值更新函数，ψ(W_k,U_k)为参数化的非线性函数。令小波网络的权值空间为W_k，将权值空间中的每个权值生成[-1,1]区间上均匀分布的随机数。Where W _k is the weight space, U _k is the network input, Z _z is the network output, φ _k is the weight update function, and ψ(W _k , U _k ) is a parameterized nonlinear function. Let the weight space of the wavelet network be W _k , and generate random numbers evenly distributed on the [-1,1] interval for each weight in the weight space.

步骤2 在步骤1的基础上，取[-1,1]区间上均匀分布的随机数为权值初始值，并将小波网络的所有权值按层进行单位化。Step 2 On the basis of step 1, take the random number evenly distributed on the [-1,1] interval as the initial weight value, and unitize the weight value of the wavelet network by layer.

步骤3 小波神经元权值优化。以小波函数为激励函数的神经元为中心，将前后两个网络层中的权值与小波函数类型和神经元个数进行关联。设第J层中的激励函数为小波函数，I、K分别为J层前后的两层，此时W_LM、W_MN为单位化后三层间的两个权值矩阵，则将小波函数类型和神经元个数与其关联的表达式为：Step 3 Wavelet neuron weight optimization. With the neuron as the activation function of the wavelet function as the center, the weights in the two network layers before and after are associated with the type of wavelet function and the number of neurons. Assume that the excitation function in the Jth layer is a wavelet function, and I and K are the two layers before and after the J layer respectively. At this time, W _LM and W _MN are two weight matrices between the three layers after unitization, then the wavelet function type The expression associated with the number of neurons is:

${W W}_{JK JK} = = {K K}_{J J} \cdot \cdot {W W}_{JK JK} \sqrt[K K]{J J}$

其中K_J为与小波函数有关的常值，不同的小波函数具有不同的常值。Among them, K _J is a constant value related to the wavelet function, and different wavelet functions have different constant values.

步骤4 引入训练样本集{x(n),norm(n)}。依次输入向量x(1)，x(2)……x(n)，并记录网络输出z(1)，z(2)……z(n)。求解误差信号e(n)和训练代价函数ε(n)：Step 4 introduce the training sample set {x(n), norm(n)}. Input the vectors x(1), x(2)...x(n) in turn, and record the network output z(1), z(2)...z(n). Solve the error signal e(n) and the training cost function ε(n):

e(n)＝norm(n)-z(n)e(n)=norm(n)-z(n)

$ϵ ϵ ((n no)) = = \frac{11}{22} {e e}^{22} ((n no))$

步骤5 采用阶梯函数对活化函数的导函数分段处理。按照导数的变化规律将函数分为M段，从而对活化函数进行拟合。活化函数的每段的斜率对应其导数的函数值，即斜率的分段问题可转化为导函数的分段问题。Step 5 Use the step function to segment the derivative function of the activation function. According to the change law of the derivative, the function is divided into M segments, so as to fit the activation function. The slope of each segment of the activation function corresponds to the function value of its derivative, that is, the segmental problem of the slope can be transformed into the segmental problem of the derivative function.

步骤6 制定拟合导函数的模糊规则。T-S模型采用多输入单输出模型，输入变量：Step 6 Formulate fuzzy rules for fitting derivative functions. The T-S model adopts a multi-input single-output model, and the input variables are:

输出量为导函数值k(n)，其模糊规则有着如下形式：The output quantity is the derivative function value k(n), and its fuzzy rules have the following form:

其中和表示第i条规则中的模糊集合，b^m代表第m段(m＝1,2,...,M)的左边界，p_i、q_i和r_i是与模糊集合有关的常数。in and represents the fuzzy set in the i-th rule, b ^m represents the left boundary of the m-th segment (m=1,2,...,M), p _i , q _i and _ri are constants related to the fuzzy set.

步骤7 确定隶属函数。本发明采用高斯型函数作为隶属函数，各输入变量x_j的隶属度为：Step 7 Determine the membership function. The present invention adopts the Gaussian function as the membership function, and the degree of membership of each input variable _xj is:

式中和分别为隶属度函数的中心和宽度。In the formula and are the center and width of the membership function, respectively.

步骤8 确定每个模糊规则在导函数值中所占的比重。每条模糊规则对于输入量x＝[x₁,x₂]的适用度μ_i及其激活度为：Step 8 Determine the proportion of each fuzzy rule in the value of the derivative function. The applicability μ _i and activation degree of each fuzzy rule to the input quantity x=[x ₁ ,x ₂ ] for:

${\overset{^^}{μ μ}}_{i i} = = \frac{{μ μ}_{i i}}{{Σ Σ}_{i i = = 11}^{c c} {μ μ}_{i i}}$

步骤9 T-S模糊系统输出及活化函数线性化显示。活化函数的线性形式可描述为：Step 9 T-S fuzzy system output and activation function linearization display. The linear form of the activation function can be described as:

$s the s ((v v ((n no)))) = = \{\begin{matrix} a a,, & v v ((n no)) > > {θ θ}_{11} \\ k k ((n no)) v v ((n no)) + + d d ((n no)),, & {θ θ}_{22} < < v v ((n no)) < < {θ θ}_{11} \\ b b,, & v v ((n no)) < < {θ θ}_{22} \end{matrix}$

s(x)为线性形式的活化函数，a和b分别为函数的左右边界，θ₁和θ₂分别为边界的阈值，k(n)和d(n)为线性区域系数，λ为常值系数。s(x) is the activation function in linear form, a and b are the left and right boundaries of the function respectively, θ ₁ and θ ₂ are the thresholds of the boundary respectively, k(n) and d(n) are linear region coefficients, and λ is a constant value coefficient.

步骤10 求解各神经元的诱导局部域及神经元输出。诱导局部域与神经元j的输出信号分别为：Step 10 Solve the induced local domain and neuron output of each neuron. The output signals of the induced local domain and neuron j are:

其中v_j(n)为诱导局部域，w_ij为权值，x_i(n)为上层神经元输出，W_ij和X_i(n)分别为w_ij和x_i(n)构成的向量，I为上层神经元总数，为j层的活化函数。Where v _j (n) is the induced local domain, w _ij is the weight, xi (n) is the output of the neurons in the upper layer, W _ij _and _Xi (n) are the vectors composed of w _ij and _xi (n), respectively, I is the total number of neurons in the upper layer, is the activation function of layer j.

步骤11 求解各个局部梯度函数δ_j(n)。局部梯度δ_j(n)可表示为：Step 11 Solve each local gradient function δ _j (n). The local gradient δ _j (n) can be expressed as:

经步骤9的活化函数线性化后，函数δ_jL(n)的线性化表示为：After the activation function linearization in step 9, the linearization of the function δ _jL (n) is expressed as:

步骤12输出层自适应调整学习率。为增强小波网络权值的学习效率，本发明提出的自适应学习率为：Step 12: The output layer adaptively adjusts the learning rate. In order to enhance the learning efficiency of wavelet network weights, the adaptive learning rate proposed by the present invention is:

$\begin{matrix} {μ μ}_{k k} ((n no + + 11)) = = g g (({x x}_{11} ((n no)),, {x x}_{22} ((n no)) {. . . . . . x x}_{J J} ((n no)),, {k k}_{k k} ((n no)))) \\ = = 11 / / [[{k k}_{k k}^{22} ((n no)) (({x x}_{11}^{22} ((n no)) + + {x x}_{22}^{22} ((n no)) + + . . . . . . + + {x x}_{j j}^{22} ((n no)))) + + {σ σ}_{v v}^{22}]] \end{matrix}$

其中k_k(n)＝s′_k(v_k(n))＝常数c_k，为了避免当输入x(n)较小时，也会很小，这样有可能出现数值计算的困难，因此采用σ_v(0＜σ_v＜1)以克服这个问题。Where k _k (n)=s′ _k (v _k (n))=constant c _k , in order to avoid when the input x(n) is small, It will also be very small, which may cause numerical calculation difficulties, so σ _v (0<σ _v <1) is used to overcome this problem.

步骤13 输出层学习率的范围Step 13 Range of output layer learning rate

为了保证μ_k(n+1)的有效性，本发明对其加以阈值的限制：In order to ensure the validity of μ _k (n+1), the present invention imposes a threshold limit on it:

${μ μ}_{k k} ((n no + + 11)) = = \{\begin{matrix} {μ μ}_{max max} & {μ μ}_{k k} > > {μ μ}_{max max} \\ g g (({x x}_{11} ((n no)) {,, x x}_{22} ((n no)) . . . . . . {x x}_{J J} ((n no)),, {k k}_{k k} ((n no)))) & {μ μ}_{min min} < < {μ μ}_{k k} < < {μ μ}_{max max} \\ {μ μ}_{min min} & {μ μ}_{k k} < < {μ μ}_{min min} \end{matrix}$

步骤14 隐层的学习率调节。本发明在步骤12输出层学习率自适应调节的基础上，对隐层的每个神经元均采用相同的学习率：Step 14 Adjust the learning rate of the hidden layer. The present invention adopts the same learning rate to each neuron of the hidden layer on the basis of step 12 output layer learning rate adaptive adjustment:

${μ μ}_{j j} ((n no + + 11)) = = \frac{11}{k k} {Σ Σ}_{k k = = 11}^{K K} {μ μ}_{k k} ((n no))$

其中μ_j表示隐层第j个神经元的学习率，K为输出层神经元个数。Among them, μ _j represents the learning rate of the jth neuron in the hidden layer, and K is the number of neurons in the output layer.

步骤15 神经元突触权值的训练过程。突触权值的调整过程中引入步骤12和步骤14的自适应学习率，但为保持活化函数原有的优势，其局部梯度仍采用非线性的δ(n)：Step 15 Training process of neuron synaptic weights. The adaptive learning rate of step 12 and step 14 is introduced in the adjustment process of synaptic weights, but in order to maintain the original advantages of the activation function, its local gradient still adopts nonlinear δ(n):

w(n+1)＝w(n)+Δw(n)w(n+1)=w(n)+Δw(n)

Δw(n)＝μ(n)δ(n)x(n)Δw(n)=μ(n)δ(n)x(n)

步骤16 循环次数加1，返回步骤10，直至满足停止准则，输出跟踪控制信号。Step 16 Increase the number of cycles by 1, return to step 10, and output the tracking control signal until the stop criterion is met.

步骤17将控制信号输入执行机构并与系统进行计算融合，在一定的外界干扰条件下，输出被控参数值，并与预期量进行比较，完成闭环反馈控制的一个过程。Step 17: Input the control signal into the actuator and perform calculation and fusion with the system. Under certain external disturbance conditions, output the value of the controlled parameter and compare it with the expected value to complete a process of closed-loop feedback control.

本发明提出一种应用于小波神经网络T-S模糊化活化函数及自适应学习率调节方法，建立一种基于NLMS的小波神经网络学习率自适应调节方法。该方法的具体实施包括建立控制模型及小波神经网络模型，活化函数导函数的T-S模糊推理，构造输出层和隐层的自适应学习率函数模型等关键内容。本发明所述的学习率自适应调节方法在神经网络在线学习平台中进行，图1所示是小波网络的系统结构图。下面将按照流程详述本发明提出的技术方案的具体实施过程(如图2所示)。该实施方式主要包含以下几个关键内容：The invention proposes a T-S fuzzy activation function and an adaptive learning rate adjustment method applied to the wavelet neural network, and establishes an NLMS-based wavelet neural network learning rate adaptive adjustment method. The specific implementation of this method includes establishing control model and wavelet neural network model, T-S fuzzy inference of activation function derivative, constructing output layer and hidden layer adaptive learning rate function model and other key contents. The learning rate self-adaptive adjustment method described in the present invention is carried out in the neural network online learning platform, and Fig. 1 shows the system structure diagram of the wavelet network. The specific implementation process of the technical solution proposed by the present invention will be described in detail below according to the flow chart (as shown in FIG. 2 ). This implementation mainly includes the following key contents:

步骤1 建立控制系统模型(如图3所示)，采用小波网络对增强型PID控制器进行参数整定，从而实现复杂系统的跟踪控制。令小波网络采用MISO的多层反馈网结构，各神经元函数为活化函数(如sigmoid函数、logistic函数等)，其中小波函数取具有连续可微性质的函数(如Morlet函数)，设置循环停止准则。小波网络的状态空间模型可表述为：Step 1 Establish the control system model (as shown in Figure 3), and use the wavelet network to tune the parameters of the enhanced PID controller, so as to realize the tracking control of the complex system. Let the wavelet network adopt the multi-layer feedback network structure of MISO, each neuron function is an activation function (such as sigmoid function, logistic function, etc.), and the wavelet function is a function with continuous differentiability (such as Morlet function), and the loop stop criterion is set . The state space model of wavelet network can be expressed as:

其中W_k为权值空间，U_k为网络输入，Z_k为网络输出，φ_k为权值更新函数，ψ(W_k,U_k)为参数化的非线性函数。令小波网络的权值空间为W_k，将权值空间中的每个权值生成[-1,1]区间上均匀分布的随机数。Where W _k is the weight space, U _k is the network input, Z _k is the network output, φ _k is the weight update function, and ψ(W _k , U _k ) is a parameterized nonlinear function. Let the weight space of the wavelet network be W _k , and generate random numbers evenly distributed on the [-1,1] interval for each weight in the weight space.

步骤2 在步骤1的基础上，取[-1,1]区间上均匀分布的随机数为权值初始值，并将小波网络的所有权值按层进行单位化。例如设W_MN为由w_mn(m＝1...M；n＝1...N)组成的第M层至第N层间的权值矩阵，则单位化后的权值矩阵W_MN为：Step 2 On the basis of step 1, take the random number evenly distributed on the [-1,1] interval as the initial weight value, and unitize the weight value of the wavelet network by layer. For example, let W _MN be the weight matrix between the Mth layer and the Nth layer composed of w _mn (m=1...M; n=1...N), then the unitized weight matrix W _MN for:

${W W}_{MN MN} = = \frac{{W W}_{MN MN}}{\sqrt{{Σ Σ}_{m m = = 11}^{M m} {Σ Σ}_{n no = = 11}^{N N} {w w}_{mn mn}^{22}}}$

步骤3 小波神经元权值优化，这一步骤是以小波函数为激励函数的神经元为中心，将前后两个网络层中的权值与小波函数类型和神经元个数进行关联。在小波网络中，令某个隐层中的激励函数为小波函数，例如本发明采用Morlet函数，设第J层中的激励函数为小波函数，I、K分别为J层前后的两层，此时W_LM、W_MN为单位化后三层间的两个权值矩阵，则将小波函数类型和神经元个数与其关联的表达式为：Step 3 Wavelet neuron weight optimization, this step is centered on the neuron with the wavelet function as the excitation function, and correlates the weights in the front and rear two network layers with the wavelet function type and the number of neurons. In the wavelet network, the activation function in a certain hidden layer is a wavelet function. For example, the present invention adopts the Morlet function, and the activation function in the J layer is a wavelet function, and I and K are respectively two layers before and after the J layer. Here When W _LM and W _MN are two weight matrices between the three layers after normalization, the expression for correlating the wavelet function type and the number of neurons with it is:

${W W}_{IJ IJ} = = {K K}_{J J} \cdot \cdot {W W}_{IJ IJ} \sqrt[J J]{I I}$

${W W}_{JK JK} = = {K K}_{J J} \cdot \cdot {W W}_{JK JK} \sqrt[K K]{J J}$

e(n)＝norm(n)-z(n)e(n)=norm(n)-z(n)

$ϵ ϵ ((n no)) = = \frac{11}{22} {Σe Σ e}^{22} ((n no))$

步骤5 采用阶梯函数对活化函数的导函数分段处理。采用分段线性逼近策略，其基本思想是用一系列折现来逼近活化函数的导数。本发明采用的是阶梯函数进行逼近，即按照导数的变化规律将函数分为M段，从而对活化函数进行拟合。活化函数的每段的斜率对应其导数的函数值，即斜率的分段问题可转化为导函数的分段问题。Step 5 Use the step function to segment the derivative function of the activation function. Using a piecewise linear approximation strategy, the basic idea is to approximate the derivative of the activation function with a series of discounts. The present invention uses a step function for approximation, that is, the function is divided into M sections according to the change rule of the derivative, so as to fit the activation function. The slope of each segment of the activation function corresponds to the function value of its derivative, that is, the segmental problem of the slope can be transformed into the segmental problem of the derivative function.

步骤6 制定拟合导函数的模糊规则。令T-S模糊模型为多输入单输出系统，其两个清晰输入变量分别为：Step 6 Formulate fuzzy rules for fitting derivative functions. Let the T-S fuzzy model be a multiple-input single-output system, and its two clear input variables are:

模糊系统清晰输出量为导函数值k(n)，其模糊规则有着如下形式：The clear output of the fuzzy system is the derivative function value k(n), and its fuzzy rules have the following form:

其中和表示第i条规则中的模糊集合，本发明满足为神经元输入信号，即活化函数输入信号，其计算方法在以后的步骤中有描述，设导函数被分割成M段，b^m代表第m段(m＝1,2,...,M)的左边界。p_i、q_i和r_i是与模糊集合有关的常数，它们是函数固有特性的反映。in and Represents the fuzzy set in the i-th rule, the present invention satisfies It is the input signal of the neuron, that is, the input signal of the activation function, and its calculation method is described in the following steps. Let the derivative function be divided into M segments, and b ^m represents the mth segment (m=1,2,...,M ) on the left border. p _i , q _i and _ri are constants related to fuzzy sets, which reflect the inherent characteristics of functions.

步骤7 确定隶属函数。本发明采用高斯型函数作为隶属函数，根据已设定的模糊规则，输入量x＝[x₁,x₂]中各输入变量x_j的隶属度为：Step 7 Determine the membership function. The present invention adopts the Gaussian function as the membership function, and according to the set fuzzy rules, the membership degree of each input variable x _j in the input quantity x=[x ₁ , x ₂ ] is:

步骤8 确定每个模糊规则在导函数值中所占的比重。由模糊推理可得到每条模糊规则对于输入量x＝[x₁,x₂]的适用度μ_i及其归一算法得到每条模糊规则的激活度分别表示为：Step 8 Determine the proportion of each fuzzy rule in the value of the derivative function. From fuzzy reasoning, the applicability μ _i of each fuzzy rule to the input quantity x=[x ₁ , x ₂ ] can be obtained and the activation degree of each fuzzy rule can be obtained by its normalization algorithm Respectively expressed as:

步骤9 T-S模糊系统输出及活化函数线性化。根据步骤6至步骤8可得到导函数的结果为：Step 9 T-S fuzzy system output and activation function linearization. According to step 6 to step 8, the result of the derivative function can be obtained as follows:

$((k k)) n no = = {Σ Σ}_{i i = = 11}^{c c} {\overset{^^}{μ μ}}_{i i} ((n no)) \cdot \cdot {k k}_{i i} ((n no))$

本文采用的活化函数均为有界函数，且具有连续可微性，线性形式可描述为：The activation functions used in this paper are all bounded functions and have continuous differentiability. The linear form can be described as:

s(·)为线性形式的活化函数，由上式可知，活化函数被分成三个部分，左右边界和线性拟合区域，a和b分别为函数的左右边界，θ₁和θ₂分别为边界的阈值，k(n)和d(n)为线性区域系数，λ为常值系数。图4表示隐层的小波函数与输出层的sigmoid函数及导函数，图5描述了导函数在T-S模糊推理后的拟合效果。s( ) is the activation function in linear form. It can be seen from the above formula that the activation function is divided into three parts, the left and right boundaries and the linear fitting area, a and b are the left and right boundaries of the function, respectively, and θ ₁ and θ ₂ are the boundaries The threshold value of , k(n) and d(n) are coefficients in the linear region, and λ is a constant coefficient. Figure 4 shows the wavelet function of the hidden layer, the sigmoid function and the derivative function of the output layer, and Figure 5 describes the fitting effect of the derivative function after TS fuzzy reasoning.

步骤10 求解各神经元的诱导局部域及神经元输出，其中输入层的诱导局部域为输入向量本身，同时输入神经元不含活化函数。图6描述了第n次迭代时，除输入层外的神经元j的输入信号流图，其诱导局部域的函数信号来自于其上层神经元输出及输出与神经元j之间的权值向量：Step 10 Solve the induced local field and neuron output of each neuron, where the induced local field of the input layer is the input vector itself, and the input neuron does not contain an activation function. Figure 6 depicts the input signal flow graph of neuron j except the input layer at the nth iteration, and the function signal of the induced local domain comes from the output of neurons in the upper layer and the weight vector between the output and neuron j :

神经元j的输出信号为：The output signal of neuron j is:

步骤12 输出层自适应调整学习率。在步骤9—11基础上，网络中的权值更新过程如下式：Step 12 The output layer adaptively adjusts the learning rate. On the basis of steps 9-11, the weight update process in the network is as follows:

其中c_j(n)和均为常数，z_i为神经元j的输入信号，e(n)为误差信号，而LMS算法的权系数更新公式为：where c _j (n) and are constants, z _i is the input signal of neuron j, e(n) is the error signal, and the weight coefficient update formula of the LMS algorithm is:

w(n+1)＝w(n)+μx(n)e(n)w(n+1)=w(n)+μx(n)e(n)

结合LMS的应用原理可知，线性化后的神经网络权值调整与LMS权系数具有近似相同的结构，LMS算法只是BP网络的特殊形式。由于本发明中提出的T-S模糊推理在活化函数中的应用可以在精度较高的情况下将函数进行局部线性化，从而解决了LMS无法在非线性递归函数中的应用问题。在LMS滤波器中，当输入较大时，滤波器会遇到梯度噪声大的问题。为了克服这一困难，可使用归一化LMS滤波器——NLMS滤波器。根据上述理论可知，NLMS只是LMS的延伸，通过理论分析可知NLMS在线性递归函数的问题上存在着与LMS同样的局限性，因此本发明采用NLMS滤波器的步长变换思想，结合T-S模糊推理步骤，采用如下方案自适应调整学习率：Combined with the application principle of LMS, it can be seen that the weight adjustment of the linearized neural network has approximately the same structure as the LMS weight coefficient, and the LMS algorithm is only a special form of the BP network. Since the application of the T-S fuzzy reasoning proposed in the present invention to the activation function can locally linearize the function with high precision, it solves the problem that the LMS cannot be applied to the nonlinear recursive function. In LMS filters, when the input is large, the filter suffers from the problem of large gradient noise. To overcome this difficulty, a normalized LMS filter—NLMS filter—can be used. According to the above theory, NLMS is only an extension of LMS. Through theoretical analysis, it can be known that NLMS has the same limitations as LMS on the problem of linear recursive functions. Therefore, the present invention adopts the step size transformation idea of NLMS filter, combined with T-S fuzzy reasoning steps , adopt the following scheme to adaptively adjust the learning rate:

其中k_k(n)＝s′_k(v_k(n))＝常数c_k。由于小波网络的权值更新过程远比自适应滤波器复杂，尤其是隐层的更新过程，因此上式中的μ_k表示输出层第k个神经元的学习率，而隐层的学习率还需进一步讨论。为了避免当输入x(n)较小时，也会很小，这样有可能出现数值计算的困难，因此采用σ_v(0＜σ_v＜1)以克服这个问题。where k _k (n)=s′ _k (v _k (n))=constant c _k . Since the weight update process of the wavelet network is far more complicated than that of the adaptive filter, especially the update process of the hidden layer, μ _k in the above formula represents the learning rate of the kth neuron in the output layer, and the learning rate of the hidden layer is also Further discussion is required. To avoid when the input x(n) is small, It will also be very small, which may cause numerical calculation difficulties, so σ _v (0<σ _v <1) is used to overcome this problem.

步骤13 输出层自适应学习率的范围Step 13 The range of the adaptive learning rate of the output layer

步骤14 隐层的学习率调节。隐层权值的更新以输出层更新为基础，根据更新规则可知每个输出层的神经元在隐层更新中均有相同的贡献。因此本发明在步骤12输出层学习率自适应调节的基础上，对隐层的每个神经元j均采用相同的学习率：Step 14 Adjust the learning rate of the hidden layer. The update of the hidden layer weights is based on the update of the output layer. According to the update rules, it can be known that the neurons of each output layer have the same contribution in the update of the hidden layer. Therefore, on the basis of the adaptive adjustment of the learning rate of the output layer in step 12, the present invention adopts the same learning rate to each neuron j of the hidden layer:

步骤15 神经元突触权值的训练过程。突触权值的调整过程中引入步骤12和步骤14的自适应学习率，但为保持活化函数原有的优势，其局部梯度仍采用非线性的δ(n)，具体描述如下：Step 15 Training process of neuron synaptic weights. The adaptive learning rate of step 12 and step 14 is introduced in the process of synaptic weight adjustment, but in order to maintain the original advantages of the activation function, its local gradient still adopts nonlinear δ(n), which is described in detail as follows:

w(n+1)＝w(n)+Δw(n)w(n+1)=w(n)+Δw(n)

Δw(n)＝μ(n)δ(n)x(n)Δw(n)=μ(n)δ(n)x(n)

步骤17 将控制信号输入执行机构并与系统进行计算融合，在一定的外界干扰条件下，输出被控参数值，并与预期量进行比较，完成闭环反馈控制的一个过程。如附图7和8所示，网络分别对非线性函数norm＝a₁sin(b₁πn)+c₁log_dn和norm＝a₂cos(b₂πn)进行拟合训练，分别采用固定学习率与本发明的变学习率方法进行对比仿真验证。Step 17 Input the control signal into the actuator and perform calculation and fusion with the system. Under certain external disturbance conditions, output the value of the controlled parameter and compare it with the expected value to complete a process of closed-loop feedback control. As shown in Figures 7 and 8, the network performs fitting training on the nonlinear functions norm=a ₁ sin(b ₁ πn)+c ₁ log _d n and norm=a ₂ cos(b ₂ πn) respectively, using fixed The learning rate and the variable learning rate method of the present invention are compared and verified by simulation.

Claims

1. the adaptive learning rate Wavelet Neural Control method based on normalization minimum mean-square auto adapted filtering, is characterized in that, comprises the steps:

(1) set up control system model: adopt wavelet network to carry out parameter tuning to enhanced PID controller, making wavelet network is the Multi-Layer Feedback web frame of MIMO, and each neuron function is activation functions, and the state space of neural network is:

\{\begin{matrix} W_{k} = W_{k - 1} + φ_{k} \\ Z_{k} = ψ (W_{k}, U_{k}), \end{matrix}

Wherein W _kfor weights space, U _kfor network input, Z _kfor network output, φ _kfor right value update function, ψ (W _k, U _k) be parameterized nonlinear function, the weights space of wavelet network is W _k, the each weights in weights space are generated to equally distributed random number on [1,1] interval;

(2) getting equally distributed random number on [1,1] interval is weights initial value, and all weights of wavelet network are carried out to unit by layer;

(3) small echo neuron weights are optimized: centered by the neuron taking wavelet function as excitation function, weights in former and later two network layers are carried out associated with wavelet function type, neuron number respectively, if the excitation function in J layer is wavelet function, I, K are respectively the two-layer of J layer front and back, W _lM, W _mNfor two weight matrixs of three interlayers after unit, by expression formula associated with it with neuron number wavelet function type be:

W_{IJ} = K_{j} \cdot W_{IJ} \sqrt[J]{I}

W_{JK} = K_{J} \cdot W_{JK} \sqrt[K]{J},

Wherein K _jfor normal value;

(4) introduce training sample set { x (n), norm (n) }: input vector x (1) successively, x (2) ... x (n), record network output z (1), z (2) ... z (n), solves error signal e (n) and training cost ε (n):

e(n)＝norm(n)-z(n)

ϵ (n) = \frac{1}{2} e^{2} (n)

(5) adopt the derived function staging treating of step function to activation functions: function is divided into M section, activation functions is carried out to matching, the functional value of the corresponding derivative of the slope of every section of activation functions;

(6) fuzzy rule of formulation matching derived function: the input variable of T-S model:

x = [x_{1}, x_{2}] &DoubleRightArrow; \{\begin{matrix} x_{1} = x (n) \\ x_{2} = x (n) - b^{m} \end{matrix}

Output quantity is derived function value k (n), and its fuzzy rule form is:

R^{i} : if x_{1} is A_{1}^{i} and x_{2} is A_{2}^{i}, then k_{i} = p_{i} x_{1} + q_{i} x_{2} + r_{i} (i = 1,2, . . ., c),

Wherein with represent the fuzzy set in i rule, b ^mrepresent m section (m=1,2 ..., M) left margin, p _i, q _iand r _iit is the constant of fuzzy set;

(7) determine subordinate function: adopt Gauss type function as subordinate function, each input variable x _jdegree of membership be:

\begin{matrix} μ_{A_{j}^{i}} (x_{j}) = \exp (- {(\frac{x_{j} - c_{j}^{i}}{σ_{j}^{i}})}^{2}) & j = 1,2,3; i = 1,2, . . ., M \end{matrix}

In formula with be respectively center and the width of membership function;

(8) determine each fuzzy rule shared proportion in derived function value: every fuzzy rule is for input quantity x=[x ₁, x ₂] relevance grade μ _iand activity for:

μ_{i} = Π_{j = 1}^{3} μ_{A_{j}^{i}} (x_{j})

{\hat{μ}}_{i} = \frac{μ_{i}}{Σ_{i = 1}^{c} μ_{i}};

(9) output T-S fuzzy system, linearization show activation functions: the linear forms of activation functions:

s (v (n)) = \{\begin{matrix} a, & v (n) > θ_{1} \\ k (n) v (n) + d (n), & θ_{2} < v (n) < θ_{1} \\ b, & v (n) < θ_{2} \end{matrix},

The activation functions that s (x) is linear forms, a and b are respectively the border, left and right of function, θ ₁and θ ₂be respectively the threshold value on border, k (n) and d (n) are range of linearity coefficient, and λ is normal value coefficient;

(10) determine each neuronic induction local field and neuron output: the output signal of induction local field and neuron j is respectively:

v_{j} (n) = Σ_{i = 1}^{I} w_{ij} x_{i} (n) = X_{i}^{T} (n) W_{ij} (n)

Wherein v _j(n) be induction local field, w _ijfor weights, x _i(n) be the output of upper strata neuron, W _ijand X _i(n) be respectively w _ijand x _i(n) vector forming, I is upper strata neuron sum, for the activation functions of j layer;

(11) solve each partial gradient function δ _j(n), partial gradient δ _j(n) be:

After the activation functions linearization of step (9), function δ _jL(n) linearization is expressed as:

(12) output layer self-adaptation regularized learning algorithm rate: adaptive learning rate is:

\begin{matrix} μ_{k} (n + 1) = g (x_{1} (n), x_{2} (n) {. . . x}_{J} (n), k_{k} (n)) \\ = 1 / [k_{k}^{2} (n) (x_{1}^{2} (n) + x_{2}^{2} (n) + . . . + x_{j}^{2} (n)) + σ_{v}^{2}], \end{matrix}

Wherein k _k(n)=s ' _k(v _k(n))=constant c _k, σ _v(0 < σ _v< 1);

(13) determine the scope of output layer learning rate:

Threshold value:

μ_{k} (n + 1) = \{\begin{matrix} μ_{\max} & μ_{k} > μ_{\max} \\ g (x_{1} (n) {, x}_{2} (n) . . . x_{J} (n), k_{k} (n)) & μ_{\min} < μ_{k} < μ_{\max} \\ μ_{\min} & μ_{k} < μ_{\min} \end{matrix};

(14) learning rate of hidden layer regulates: the each neuron to hidden layer all adopts identical learning rate:

μ_{j} (n + 1) = \frac{1}{k} Σ_{k = 1}^{K} μ_{k} (n),

Wherein μ _jrepresent a hidden layer j neuronic learning rate, K is output layer neuron number;

(15) training synapse weights: introduce the adaptive learning rate of step (12) and step (14), partial gradient adopts nonlinear δ (n):

w(n+1)＝w(n)+Δw(n)

Δw(n)＝μ(n)δ(n)x(n)；

(16) cycle index adds 1, returns to step (10), until meet stopping criterion, Output Tracking Control signal;

(17) control signal inputted to topworks and calculated fusion with system, exporting controlled parameter value, comparing with desired amount, completing close-loop feedback control.