JP2851901B2 - Learning control device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Application Field] The present invention relates to a control system for giving a predetermined output result to an input command based on a preset response characteristic, and particularly to a control system for a robot. The present invention relates to a learning control device suitable for control.

[Conventional technology]

Conventionally, when trying to automatically adjust each gain of a control system using a PID (proportional / integral / derivative) control law using a learning device called a neural network, the 6th Annual Meeting of the Robotics Society of Japan Academic Lecture Papers (Showa 63-1
0), as described on pages 143 to 144, teacher signals used for learning a neural network are prepared in advance by humans experimentally or analytically.

However, in the hybrid control of the position and the force in the robot manipulator, since the dynamic characteristics of the control target are included in the feedback loop of the entire control system, stable control cannot be performed without knowing the dynamic characteristics of the control target. In addition, in the case of control using a fixed PID gain, the adaptability is limited. Therefore, it is necessary to appropriately change the PID gain depending on an object to be handled.

Therefore, in the above-described conventional technique, a method is used in which the PID gain, which is a parameter of the control system, is appropriately changed by using a neural network.

In this method, the neural network forms itself by learning how to determine the optimal PID gain value according to the input. In other words, the input of the neural network is input to the position of the tip of the manipulator, the force to be fed back,
And the target value of force, and the output of the neural network is used as each PID gain.

In the learning using the neural network, data representing a desired output given to an input and output data of an actual neural network are compared, and the network is adjusted so as to minimize the error. By using a backpropagation algorithm which changes the state of the backpropagation and repeating this backpropagation algorithm, the output of the neural network approaches the desired output.

By the way, the desired output data to be given to the input used here is called a teacher signal. In the above-mentioned conventional technique, as the teacher signal, an experiment or an experiment is carried out so that a control system can be stabilized by a human in advance. The PID gain obtained by analyzing the control system was used.

Using some of the teacher signals thus obtained, the neural network has learned some of the characteristics of the control system. After the learning, the neural network estimates the dynamic characteristics of the control system from the learned characteristics and outputs a PID gain according to the input.

[Problems to be solved by the invention]

The above-mentioned conventional technique requires, as a teacher signal used for learning by the learning device, a signal prepared by a human in advance experimentally or analytically so that the evaluation value of the output of the control system becomes a target value.

For this reason, when good experimental results cannot be obtained or when a model for analysis cannot be constructed, a teacher signal cannot be created in advance, and the learning device cannot learn. There was a problem of getting it.

Further, in the prior art, even if a teacher signal can be created, there is a problem that an enormous amount of time is required to create a large amount of uniform teacher data.

An object of the present invention is to enable learning of a learning device without human intervention as much as possible, and by using a learning device after learning,
It is an object of the present invention to provide a learning control device having a function of automatically generating a teacher signal capable of setting an evaluation value of an output of a control system to a target value.

[Means for solving the problem]

In order to achieve the above object, in addition to the operation mode as a normal learning control device, a learning mode for automatically performing learning of the learning device is provided. In this learning mode, an adjustment signal of the control system is automatically and randomly generated. It is provided with a teacher signal generating device which functions to generate the signal.

Further, an evaluation device is provided for evaluating the input of the control system and the output of the control system with respect to the random adjustment signal generated by the teacher signal generation device.

Also, a target evaluation value setting device for setting a target value for an evaluation value of an output of the control system in an operation mode as a normal learning control device is provided.

Further, in the above two modes, there is provided a switching device for disconnecting the teacher signal generation device and switching between the evaluation device and the target evaluation value setting device.

[Action]

In the learning mode described above, the teacher signal generation device randomly generates an adjustment signal for operating the control system, and passes this value to the control system as an adjustment signal.

When an input is given to the control system, the control system outputs a response according to the input / output characteristics determined by the adjustment signal.

Therefore, the evaluation device evaluates the output, and thereby obtains a set of learning data, that is, an adjustment signal, a response of the control system corresponding thereto, and an evaluation value for the response without human intervention. It will be.

Thereafter, the learning data is passed to the learning device, and the learning device
The relationship between the output of the control system, the evaluation value of the evaluation device, and the adjustment signal generated by the teacher signal generation device is learned by changing the state inside the learning device.

By repeating the above process and repeating the learning, the random teacher signal and the output of the learning device come close to each other, and the output of the control system, the evaluation value of the evaluation device, and the adjustment signal The association is completed. That is, the characteristics of the control system can be learned.

After learning, the switching device switches the learning control device to the operation mode. At this time, the teacher signal becomes unnecessary, so the teacher signal generation value is cut off. In addition, an evaluation device is not required, and a target evaluation value setting device is connected instead. When a human sets a target evaluation value in the target evaluation value setting device, the target evaluation value is input to the learning device.

Since the learning device can internally associate the output and evaluation value of the control system with the adjustment signal, input the output of the control system at that time and the target evaluation value that has been set. Output an adjustment signal. By inputting this adjustment signal to the control system, the input / output characteristics of the control system can be adjusted so that the output evaluation value of the control system matches the set target evaluation value.

〔Example〕

Hereinafter, a first embodiment of the present invention will be described with reference to FIGS. 1 to 5, and a second embodiment will be described with reference to FIGS. 6 to 9. FIG.

In the first embodiment, the control system is applied to a force control system using a PID control law, and each gain of the PID is controlled by using a learning object called a neural network so as to suppress the vibration of the force control system. This is an example of adaptive control that is appropriately adjusted. In this embodiment, a robot hand is applied.

First, FIG. 2 shows an example of a robot hand control system to which an embodiment of the present invention is applied. In FIG. 2, a current flows through the servomotor 2 according to a force command value given to the control device 1. Then, the ball screw 3 rotates. As the ball screw 3 rotates, the movable finger 5 to which the force sensor 4 is attached moves so as to pinch the object 6 to be grasped.

When the movable finger 5 grasps the object 6 to be grasped, the grasping force is fed back by the force sensor 4, and a current flows from the control device 1 to the servomotor 2 until the deviation from the force command value becomes zero. Finally, the gripping force is made to match the force command value.

FIG. 3 is a block diagram of the above control system, and uses a PID control law as a control law. The object to be grasped Go includes a transfer coefficient of a force command value to a motor current, a conversion coefficient of a motor current to a torque, and a reduction ratio of a ball screw, in addition to a transfer function of a secondary system regarded as a spring / dashpot / mass system. In.

By the way, in this control system, hunting and vibration occur unless PID gains are set in accordance with the grip target Go. Therefore, in this embodiment, the setting of each PID gain is automatically performed using a learning device called a neural network. Hereinafter, the setting processing of each PID gain will be described. .

First, FIG. 5 shows a mode transition flow, which is composed of a learning mode 51 and an operation mode 52.
In the mode in which the neural network learns how to adjust each gain, the operation mode 52 adjusts the PID gains as appropriate using the learned neural network,
In this mode, the vibration of the force control system is suppressed.

In order to appropriately adjust the PID gains using the neural network, first, the neural network is learned in the learning mode 51, and thereafter, the mode is switched to the operation mode 52, and the PID gains are adjusted. .

Next, FIG. 1 is an overall system configuration diagram showing one embodiment of the present invention. When roughly divided, a control system 11 and a proportional element gain adjustment signal and an integral element gain adjustment signal shown in FIG. And an adjustment device (learning device for generating a teacher signal) 12 for generating a teacher signal composed of three types of differential element gain adjustment signals.

The control system 11 includes a control system using the above-described PID control law, and the adjustment device 12 includes therein an evaluation device 13, a teacher signal generation device 14, a learning device 15, and a target evaluation value setting unit 1.
6, it has a switching device 17;

The adjustment device 12 adjusts the value of each PID gain, which is a parameter of the input / output characteristics of the control system 11, and the switching device 17 switches between the learning mode 91 and the operation mode 92 described above. The switching position indicated by the solid line indicates the learning mode, and the switching position indicated by the broken line indicates the operation mode.

 First, the learning mode will be described.

The evaluation device 13 serves to evaluate the degree of vibration appearing in the output of the control system 11 and generate a predetermined evaluation function.

The teacher signal generation device 14 is constituted by a random number generator, and functions to randomly generate a numerical value in a limited range and randomly determine an adjustment signal for setting the value of each PID gain.

The learning device 15 includes a neural network 18, predicts the output of the teacher signal generation device 14 from the command value and output of the control system 11, and the output of the evaluation device 13, and
It serves to change the internal state of the neural network 18 so that the difference from the actual output of the teacher signal generator 14 becomes zero.

 Next, the operation mode will be described.

The target evaluation value setting device 16 is a register that stores the target evaluation value, and has a function of storing the degree of vibration that the control system 11 targets.

The learning device 15 outputs an adjustment signal for each PID gain from the command value and output of the control system 11 and the target evaluation value stored in the target evaluation value setting device 16 this time. This output is an adjustment signal such that the output evaluation of the control system 11 matches the target evaluation value because the neural network 18 has learned how to adjust each PID gain.

FIG. 4 is a basic configuration diagram of the neural network 18. As shown in FIG.

This neural network 18 includes multiple neurons.
The multi-input element 41 is a multi-input, one-output element that performs a product-sum operation with weighting for each independent input.

The neural network 18 in this embodiment combines the neurons 41 in a layered manner, and includes an input layer 43, a middle layer 44, and an output layer.
The input information provided to the input layer 43 is processed by being sequentially transmitted to the intermediate layer 44 and the output layer 45 in this order. Then, the neural network 18 forms itself by learning how to process this information.

This learning is performed by adjusting the strength of the connection 42 between each neuron, that is, by changing the weight of the input of the neuron 41. The learning algorithm is as follows. A system called back projection is known, and in this embodiment,
This backpropagation system is used.

Therefore, in this embodiment, as described above, each gain of the PID can be appropriately adjusted using the neural network 18 so as to encourage the vibration of the force control system, and as a result, According to this embodiment, there is no need to analyze the PID force control system included in the control system 11, and labor for that can be omitted.

Further, since the target value of the degree of vibration can be input to the adjustment device 12 without converting the target value into a parameter, labor for that can be omitted.

In addition, there is an effect that the target control can be performed only by changing the evaluation element of the evaluation device 13 and learning, without making any other modification.

 Next, a second embodiment of the present invention will be described.

This embodiment is an example in which the object of the control system is a position servo system of an XY plotter, and the characteristics of this servo system are compensated for by using a neural network.

FIG. 6 is a diagram showing an X target of the second embodiment of the present invention.
A block diagram when the position servo system of the X axis of the Y plotter is identified as having a sufficiently high response of the speed servo system included in the position servo system as compared with the response of the position servo system itself. In this case, the position servo system 61
Is composed of an integral element 62 and an adjustment gain 63 thereof.

The position servo system 61 operates with the characteristics of the primary system as is apparent from the block diagram of FIG. Since this embodiment is an XY blotter, the same position servo system is also provided for the Y axis.

FIG. 7 shows an X-axis having the position servo system shown in FIG.
This shows the locus of the Y plotter, and the locus 71 of P0 → P1 is
Although the trajectory is drawn as instructed, the trajectory 72 of P0 → P2 → P1
Deviates from the commanded locus around the locus P2.

Therefore, when analyzing the position servo system 61, since the command here is a ramp input,
When input to 61, it is understood that the response of the position servo system 61 is delayed with respect to the command by a certain amount determined by the speed which is the slope of the ramp input and the adjustment gain 63.
Consider a method for compensating for this delay below.

FIG. 8 shows a control system 11 in the embodiment of FIG.
Is a simplified equivalent block diagram showing a state in the learning mode when the control system 11 'is the object of the second embodiment. In this case, the control system 11'
13 and a known command value, a parameter K (corresponding to an adjustment signal output from the switching device 17 in the embodiment of FIG. 1) is input and an evaluation value A (also the control evaluation device 13) is input. Can be represented by a transfer function G of A = G (K).

On the other hand, as described above, the adjusting device 12 has the neural network 18 therein. Therefore, assuming that the transfer function is G ', it can be expressed as K' = G '(A).

Therefore, this adjusting device 12 is set so that K = K ′.
Train the network 18. And K =
At the point of time K ′, the transfer function G ′ of the adjusting device 12 becomes
It is the inverse function of the transfer function G of the control system 11 '.

Then, the connection is switched as shown in FIG.

FIG. 9 shows a control system in the embodiment of FIG.
11 is the control system 1 to which the embodiment of FIG.
In the simplified equivalent block diagram in the operation mode in the case of 1 ', the control system 11' is the same as that of Fig. 8, and the adjusting device 12 is at the time when K = K '. .

When the target evaluation value A 'is input, the adjusting device 12 outputs a parameter K' determined by the transfer function G '. Then, the control system 11 'calculates the parameter K'
Is output, the evaluation value A is output. However, since the transfer function G 'of the adjusting device 12 is an inverse function of the transfer function G of the control system 11', the overall transfer function is 1, and A = A '.

Therefore, if A is a position and A 'is a position command,
The position can be controlled without loss or phase shift.

According to this embodiment, there is an effect that the characteristics of the servo system can be compensated even if a strict block diagram of the servo system cannot be defined.

〔The invention's effect〕

According to the present invention, as long as the human gives the target evaluation value, the output evaluation value of the control system becomes the target evaluation value,
Can be controlled.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of a learning control device according to the present invention, FIG. 2 is a block diagram showing an example of a robot hand control system to which an embodiment of the present invention is applied, and FIG. FIG. 4 is a diagram showing the basic configuration of a neural network, FIG. 5 is an explanatory diagram of a mode transition flow, and FIG. 6 is an X diagram for the second embodiment of the present invention.
-Block diagram of X-axis position servo system of Y plotter, 7th
FIG. 8 is an explanatory diagram showing the trajectory of an XY blotter having a position servo system, FIG. 8 is a simplified equivalent block diagram showing a state in a learning mode, and FIG. 9 is a simplified equivalent block diagram in an operation mode. 1 ... Control device, 2 ... Servo motor, 3 ... Ball screw, 4 ... Force sensor, 5 ... Movable finger, 6 ... Grip target,
11 ... control system, 12 ... adjustment device, 13 ... evaluation device, 14 ... teacher signal generation device, 15 ... learning device, 16 ...
Target evaluation value setting device, 17 ... Switching device, 18 ... Neural network.

──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Hitoshi Saito 7-1-1 Higashi Narashino, Narashino City, Chiba Prefecture Inside Hitachi Keiyo Engineering Co., Ltd. (56) References JP-A-2-136904 (JP, A) JP-A-Hei 3-105662 (JP, A) JP-A-3-105663 (JP, A) Kazushige Saga, 3 others, "Study of self-learning method", 1989 IEICE Autumn National Convention Lecture Volume 6, (Information and systems), The Institute of Electronics, Information and Communication Engineers, Japan Patent Office, accepted on August 24, 1989, p. 6.278-6.279, Hideki Hashimoto, two others, (using neural networks Servo Control, ”Proceedings of the 7th Annual Conference of the Robotics Society of Japan, Robotics Society of Japan, November 2, 1989, P. 745-746 Demeter Ps. lits, and two others, "A Multilayered Neural Network Controller", IEEE Control System Magazine, The Institute of Electronics in the United States, 8th of August. 2, Victor C. Chen, P. 17-21, and one other person, "Learning Control with Neural Networks", Proceeding of IEEE International, International Conference on Robotics, New York, USA p.1448-1453 (58) investigated the field (Int.Cl. ^6, DB name) G05B 13/00 - 13/04 G06F 15/18 - 15/18 560 JIC T file (JOIS)

Claims (3)

(57) [Claims] A control system for providing a predetermined output result in response to a predetermined input command; and a predetermined signal generation characteristic based on the input command, the output result, and a predetermined target evaluation value set in advance. And a learning device for generating a predetermined adjustment signal under the control of the control system. The control system sets the response characteristic of the control system by the adjustment signal, and takes an arbitrary value within a predetermined value range. Signal generation means for generating a teacher signal; evaluation means for determining an output result of the control system; and switching means for switching between a learning mode and an operation mode. When the learning mode is switched by the switching means, Generated from the learning device in a state where the response characteristics of the control system are set by the teacher signal while taking in the output of the evaluation means instead of the target evaluation value. The learning device operates so as to change the signal generation characteristic of the learning device in a direction in which the adjustment signal converges on the teacher signal. When the operating mode is switched by the switching means, the control system is controlled by the adjustment signal. A learning control device configured to operate the control system in a state where the response characteristics of the learning control device are set. 2. The learning control method according to claim 1, wherein said signal generation means is a random number generator, and said signal generation characteristic of said learning device is given by a neural network. apparatus. 3. The invention according to claim 1, wherein said control system is a PID force control system, and said adjustment signal is a proportional element gain adjustment signal, an integral element gain adjustment signal, and a differential element gain in said PID force control system. A learning control device, which is an adjustment signal.