JP6774637B2 - Control device and control method - Google Patents

Description

The disclosed embodiments relate to control devices and control methods.

Patent Document 1 describes a method of automatically setting control parameters based on the evaluation result of each state quantity detected when a predetermined adjustment sequence operation is executed.

Japanese Unexamined Patent Publication No. 2003-061377

However, in order to satisfy the demand for high control accuracy, it is necessary to repeatedly execute various sequence operations and adjust the control parameters by comprehensive judgment depending on the skill such as the intuition and experience of a skilled engineer. ..

The present invention has been made in view of such problems, and an object of the present invention is to provide a control device and a control method capable of improving the convenience of the control device.

In order to solve the above problem, according to one aspect of the present invention, the deviation between the reference command and the control amount output by the external control target is input to the controller controlled by a predetermined control parameter, and the control is performed. A control device having a feedback control unit that controls the control target with an operation amount output by the device and an adjustment unit that adjusts the control parameters based on the learning content in the machine learning process is applied.

Further, according to another aspect of the present invention, an operation in which a deviation between a reference command and a control amount output by an external control target is input to a controller controlled by a predetermined control parameter and output by the controller. A control method for controlling the controlled object by a quantity and adjusting the control parameter based on the learning content in the machine learning process is applied.

According to the present invention, the convenience of the control device can be improved.

It is a figure which shows an example of the schematic system block structure of the machine control system of embodiment. It is a control block diagram which shows the outline of the adjustment method of the control parameter in a motor control system. It is a figure which shows the transmission / reception relationship of each information between a feedback control system including a feedback control part, and an adjustment part. It is a figure which represented each state quantity data at the time of executing the same sequence operation for adjustment by time series pattern data. It is a figure which shows an example of the schematic model structure of the neural network of the adjustment part when deep learning is applied. It is a figure which illustrates the data set for adjustment part learning corresponding to various sequence operations. It is a figure which shows the flowchart of the processing procedure when the machine learning process is executed by data learning. It is a figure which shows the flowchart of the processing procedure when the machine learning process is executed by reinforcement learning. It is a figure which illustrates the data set for learning of the adjustment part corresponding to the addition of various disturbances. It is a figure which shows an example of the schematic model composition of the neural network of the adjustment part when the machine constant is divided so that it can be output. It is a figure which shows an example of the system block composition of the motor control device when the estimation part is provided. It is a system block diagram which shows the hardware composition of a motor control device.

Hereinafter, embodiments will be described with reference to the drawings.

<Outline configuration of machine control system>
FIG. 1 shows an example of a schematic system block configuration of a machine control system including the control device of the present embodiment. This mechanical control system is a system that controls the linear movement of the slider by controlling the drive of the rotary motor. In FIG. 1, the machine control system 1 includes an upper control device 2, a motor control device 3, a rotary motor 4, and a drive machine 5.

The host control device 2 is composed of, for example, a general-purpose personal computer including a CPU, ROM, RAM, an operation unit, a display unit, and the like (not shown). The host control device 2 generates a position command for positioning the slider of the drive machine 5, which will be described later, at a desired position based on various settings and commands input from the operator via the operation unit, and the motor control device 2. Output to 3.

The motor control device 3 generates a torque command based on the position command output from the host control device 2 and outputs the torque command to the rotary motor 4. At this time, the motor control device 3 performs position feedback control based on the detection position output from the encoder 41a described later included in the rotary motor 4. The motor control device 3 has a feedback control unit 31 and an adjustment unit 32. The motor control device 3 corresponds to the control device according to each claim.

The feedback control unit 31 is a calculation unit that generates a torque command based on the above-mentioned position command and detection position. The control block of the feedback control unit 31 will be described in detail later in FIG.

The adjusting unit 32 is a processing unit that adjusts control parameters (described later) used in the calculation processing of the torque command in the feedback control unit 31 based on the position command, the detection position, and the torque command in an appropriate situation. The processing content of the adjusting unit 32 will also be described in detail later.

The processing and the like in the feedback control unit 31, the adjusting unit 32, and the like described above are not limited to the example of sharing these processing, and for example, processing is performed by a smaller number of processing units (for example, one processing unit). It may be processed by a further subdivided processing unit. Further, the motor control device 3 may be implemented in software by a program executed by the CPU 901 (see FIG. 12) described later, or a part or all of the motor control device 3 may be implemented by ASIC, FPGA, other electric circuits, or the like (neuromorphic). It may be implemented in hardware by an actual device (such as a chip).

The rotary motor 4 is, for example, a synchronous three-phase AC motor, and integrally includes an encoder 41a that outputs the rotational position of the output shaft of the rotary motor 4 as a detection position.

In the illustrated example, the drive machine 5 is an actuator that converts the rotational position, which is the shaft output of the rotary motor 4, into the translational position of the slider 52 via the feed screw 51. The drive machine 5 has a coupling 53, a feed screw 51, and a slider 52.

A feed screw 51 is connected to the output shaft of the rotary motor 4 via a coupling 53. Further, the slider 52 is a pedestal on which an object can be placed, and the feed screw 51 is screwed into the lower portion thereof. As a result, when the rotary motor 4 rotates in the forward or reverse direction, the slider 52 is fed by the feed screw 51 and follows the direction corresponding to the rotational direction of the rotary motor 4 (left-right direction in the drawing). Driven to move straight.

In the above configuration, the motor control device 3 controls a position feedback that controls a torque command output to the rotary motor 4 so that the position of the slider 52 follows the above position command based on the detection position output from the encoder 41a. Take control.

<Characteristics of this embodiment>
For example, in the motor control device 3 as in the above embodiment, the output position (detection position in the case of the above embodiment) and the output speed, which are the control amounts of the servomotor (rotary motor 4 in the case of the above embodiment), are fed back. Then, the deviation from the reference command (position command or speed command in the case of the above embodiment), which is the target value, is input to the controller (described later), and the operation amount (torque in the case of the above embodiment) output by this controller is input. In many cases, feedback control is performed to control the servo motor with a command). Even in a drive machine driven by this motor control, semi-closed feedback control that feeds back the output position and output speed of the servomotor itself connected to the drive machine as a control amount, or the final output position and output of the drive machine The position control and speed control of the driving machine (for example, the slider 52) can be performed by the fully closed feedback control in which the speed is fed back as a control amount.

The controller included in the motor control device 3 as described above outputs an operation amount by arithmetic processing using a large number of control parameters, and the mechanical constants of the servomotor and the drive machine to be controlled, and the disturbance that can be added. It is desirable to set the control parameters appropriately according to. However, when the number of parts constituting the control target (servo motor or drive machine) is large and the configuration is complicated, high control accuracy (positioning accuracy in the case of the above embodiment) and short settling time are required. It is necessary to strictly adjust the control parameters according to individual differences (part manufacturing error, assembly error, etc.) of the controlled object, operating conditions, aging deterioration status, and usage environment status.

So far, the machine constant to be controlled is estimated based on the reference command, the operation amount, and the control amount detected when the predetermined adjustment sequence operation is executed with the servomotor actually connected to the drive machine. However, there was a method of mechanically setting control parameters based on this. However, in order to satisfy the demand for high control accuracy, it is necessary to repeatedly execute various sequence operations and adjust the control parameters by comprehensive judgment based on the detected data. Adjustment of such complicated and delicate control parameters had to be performed inefficiently by repeating manual work depending on the skill such as the intuition and experience of a skilled engineer. For this reason, not only the initial adjustment at the time of introduction of the drive machine system, but also the readjustment when the usage environment changes or the deterioration over time progresses even after the start of operation, a specialized engineer directly works at the factory etc. It was necessary to go to the work site and make adjustments over a long period of time, which was a factor that greatly impaired the convenience of the control device.

On the other hand, the motor control device 3 of the present embodiment has an adjustment unit 32 that adjusts the control parameters of the controller based on the learning content in the machine learning process. As a result, by executing the machine learning process based on the data detected at the time of an appropriate sequence operation or the addition of disturbance, the adjusting unit 32 can set an appropriate control parameter according to the learning content. That is, it is possible to mechanically and automatically adjust the control parameters autonomously by the motor control device 3 itself without using human hands. Therefore, it is possible to save labor and time costs for adjusting the control parameters in the motor control device 3, and it is possible to ensure uniform and stable control accuracy regardless of the skill difference of individual engineers.

<Outline of control parameter adjustment method>
FIG. 2 shows a control block diagram showing an outline of a control parameter adjustment method in the motor control device 3. In FIG. 2, the deviation between the reference command r input from the outside and the control amount y output by the control target P is input to the controller K (ρ). The controller K (ρ) outputs an operation amount u by arithmetic processing based on the control parameter ρ, and the control target P is controlled by the operation amount u. The reference command r, the operation amount u, the control target P, and the control amount y referred to here correspond to the position command, the torque command, the rotary motor 4, the drive machine, and the detection position in the above-described embodiment, respectively (controller K). The correspondence of (ρ) will be described later).

In the feedback control system as described above, since the controller K (ρ) is a predetermined calculation model using the variable control parameter ρ, it can be said that its characteristics and behavior are completely known. On the other hand, regarding the control target P, even if the schematic model based on the design value is known, as described above, the individual differences (part manufacturing error, assembly error, etc.) and operating conditions of the control target P itself, It can be said that its substantive and detailed characteristics and behavior are unknown due to changes in aging deterioration and usage environment. That is, the control parameters need to be adjusted so as to adapt to specific mechanical constants and disturbances in the actually applied control target P.

Here, it is considered that unknown mechanical constants and disturbance factors in the controlled object P have a great influence on the correlation between the reference command r and the controlled variable y. Therefore, in the example of the present embodiment, the reference command r, the operation amount u, the control amount y, and other operation conditions c at the time of various sequence operations or disturbance addition correspond to the control parameters set at that time. The attached data set is recorded and saved in the database DB. Then, the coordinating unit Cal executes a machine learning process based on those data sets, and is appropriate for the machine constants and disturbances of the controlled object P estimated from the correspondence between the reference command r and the controlled variable y. The control parameter ρ _new can be set.

<Specific application example of feedback control unit and adjustment unit>
FIG. 3 shows an example of the transmission / reception relationship of each information between the feedback control system including the feedback control unit 31 and the adjustment unit 32. The feedback control system shown in the figure is represented by a control block in the form of a transfer function. In FIG. 3, the feedback control unit 31, the motor / drive machine 60, the adjustment unit 32, and the database DB are shown.

The feedback control unit 31 includes a subtractor 34, a position loop gain Kp, a subtractor 35, an integrator (1 / T · s), an adder 36, a speed loop gain Kv, and a speed calculator 37.

The subtractor 34 subtracts the detection position detected from the motor / drive machine 60 from the position command input from the outside, and outputs the position deviation between them. The speed command is output by multiplying this position deviation by the position loop gain Kp. In the example of the present embodiment, the position loop gain Kp functions as a position control controller (control K (ρ) in FIG. 2 above) and is configured to perform so-called position proportional control.

The subtractor 35 subtracts the detection speed output from the speed calculator 37 described later from the above speed command and outputs the speed deviation between them. The integrator (1 / T · s) performs an integral calculation based on the speed loop integration time constant T for the speed deviation, and the adder 36 adds the output of this integrator (1 / T · s) and the speed deviation. And output. A torque command is output by multiplying this added output by the speed loop gain Kv. In the example of the present embodiment, these integrators (1 / T · s) and the speed loop gain Kv also function as speed control controllers (control K (ρ) in FIG. 2 above), so-called speed integration proportionality. It is configured to provide control.

The speed calculator 37 is a calculator that outputs the detection speed (output speed of the rotary motor 4) based on the detection position detected from the motor / drive machine 60, and specifically, if it is configured by the differentiator s, Good.

The motor / drive machine 60 (corresponding to the control target P in FIG. 2) corresponds to the rotary motor 4 and the drive machine 5 in FIG. 1, and the rotor of the rotary motor 4 and the movable portion of the drive machine 5 are formed. This is a mathematical model based on the moment of inertia J of the entire connected movable mechanism. Although not particularly shown, when further multiplied by the moment of inertia J ₀ of the rotor of the rotary motor 4 to the speed loop gain Kv is a mathematical model in the motor-driving machine 60 defined by the moment of inertia ratio May be good.

As described above, the feedback control system of the example of the present embodiment composed of the feedback control unit 31 and the motor / drive machine 60 has a double loop configuration of a feedback loop of the position proportional control system and a feedback loop of the speed integration proportional control system. (So-called P-IP control). In the present embodiment, the description of the current control unit that outputs the drive current by PWM control to the motor / drive machine 60 based on the torque command and the feedback loop of the current control system provided inside the current control unit will be simplified. It is omitted for the sake of.

For the above feedback control system, when acquiring the data set for learning the adjustment unit, the values of the position loop gain Kp, the speed loop integration time constant T, and the speed loop gain Kv corresponding to the control parameter ρ are appropriately provisional. In the set state, the position command (corresponding to the reference command r), the torque command (corresponding to the operation amount u), and the detection position (corresponding to the control amount y) when the predetermined adjustment sequence operation is executed are recorded. In the example of the present embodiment, each of the position command, the torque command, and the detection position (hereinafter, these are collectively referred to as state quantity data) is when the adjustment sequence operation is executed as shown in FIG. It is recorded as time-series pattern data in which instantaneous values are sequentially recorded in the same temporary series. In the example shown in FIG. 4, the time series pattern data (thick solid line waveform data) of each state quantity when the sequence operation is executed by the positioning operation in which the position command is increased from 0 to a predetermined position by the inching control is obtained. Shown. Further, d shown in FIG. 4D represents a position deviation, which will be described later.

In the illustrated example, the case where the position command control is performed is shown, but in addition to this, the sequence operation may be executed by the speed command control or the torque command control. For example, as shown in FIG. 4B, the speed command is input and the sequence operation is executed with the trapezoidal time-series pattern as it is, which has a section in which the acceleration during acceleration and deceleration is constant and the steady speed is kept constant. You may. Alternatively, the sequence operation is executed by inputting a polynomial that guarantees the continuity of acceleration at the switching portion between acceleration / deceleration and constant speed in such a time series pattern, a speed command expressed by a trigonometric function or an exponential function, and the like. You may let me.

Then, in the database DB, for each of the various adjustment sequence operations, one adjustment is made by associating the time-series pattern data of each state quantity data with the control parameters Kp, T, Kv temporarily set during the adjustment sequence operation. Create and save the data set for part learning (see the dashed arrow in FIG. 3). The database DB may be configured by a storage device provided inside the motor control device 3, or may be configured by an external storage device connected to the motor control device 3 so as to be able to send and receive information (the above). Not shown in FIG. 1).

After a sufficient number of adjustment unit learning data sets are acquired in the database DB, the adjustment unit 32 is machine-learned (data learning) by batch learning (offline learning) using these adjustment unit learning data sets. Then, in the phase in which the adjusting unit 32 sets each control parameter Kp, T, Kv based on the learning content, each when the actual operation sequence operation of the machine control system 1 is executed with each control parameter temporarily set. The time-series pattern data of the state quantity data is input to the adjusting unit 32, and the adjusting unit 32 outputs the optimum control parameters Kp, T, Kv to the feedback control unit 31. By actually setting these output control parameters Kp, T, and Kv in the feedback control unit 31, high control accuracy in the feedback control system during operation can be ensured (see the dotted arrow in FIG. 3).

<Specific configuration of the adjustment unit>
Various machine learning methods can be applied to the adjustment unit 32. In the following, for example, an example in which deep learning is applied to a machine learning algorithm will be described. FIG. 5 shows an example of a schematic model configuration of the neural network of the adjustment unit 32 when deep learning is applied.

In FIG. 5, the neural network of the adjusting unit 32 corresponds to each time-series pattern data of the position command, the torque command, and the detection position, which are the state quantity data input from each unit, between the state quantity data. It is designed to output control parameters Kp, T, Kv that appropriately correspond to the mechanical constants of the motor / drive machine 60 estimated from the relationships, especially the correspondence between the position command and the detection position. In the schematic model configuration example of the neural network shown in FIG. 5, the values and signals corresponding to the above mechanical constants (for example, the values and signals representing the features) are not shown.

Here, with respect to the input of each state quantity data which is time series pattern data, the instantaneous values sampled in the same predetermined sampling cycle are input to each input node arranged in chronological order. Further, the control parameters Kp, T, and Kv output by each output node of the adjustment unit 32 are output by multi-value output (continuous value) by regression problem processing, respectively. The setting process of these control parameters Kp, T, Kv is based on the learning content in the machine learning process in the learning phase of the adjusting unit 32. That is, the neural network of the adjustment unit 32 learns the feature quantity representing the correlation between the state quantity data and the appropriate control parameters Kp, T, Kv.

Regarding the machine learning process of the adjustment unit 32, after the multi-layer neural network designed as described above is implemented in software (or hardware) on the motor control device 3, a large number of adjustments are stored in the database DB. The coordinating unit 32 is trained by so-called supervised learning using the data set for part learning. As shown in FIG. 6, for example, the adjustment unit learning data set used here includes each state quantity data (time series pattern data) and evaluation value when various adjustment sequence data are executed, and provisionally set at that time. This is a data set in which the control parameters Kp, T, and Kv are associated with each other. The evaluation value of the illustrated example is an index indicating the evaluation of the responsiveness in the sequence operation of the feedback control system to which the control parameters Kp, T, Kv of the corresponding data set are applied. This evaluation value is, for example, the position deviation d shown in FIG. 4D, the vibration amplitude at the time of overshoot or undershoot, the settling time until the end of the transition period, and until the control amount follows the command. The rise time of the data set, the magnitude of the ripple of the torque command value, the power consumption value that can be estimated from the torque command value and the speed, and the like may be comprehensively obtained based on the state quantity data of the data set. Further, a torque command combining a plurality of sine waves may be input, and the values of the phase margin, the gain margin, and the sensitivity function calculated from the response may be used as an index. In FIG. 6, this evaluation value is represented by an index of two stages of "high" and "low", but it may be expressed by an index of three or more stages or a numerical value.

In the learning phase of the adjusting unit 32 in the example of the present embodiment, the input layer and the output layer of the neural network of the adjusting unit 32 are used by using the teacher data of the combination of the state quantity data as the input data and the control parameters as the output data. Learning is performed by so-called backpropagation processing that adjusts the weighting coefficient of each edge that connects each node so that the relationship between them is established. In this backpropagation processing, only the data set having a particularly high evaluation value may be extracted from a large number of data sets, and only this data set may be used as the teacher data to adjust the weighting coefficient of each edge. Alternatively, all the data sets may be used as teacher data, and the weighting coefficient of each edge may be adjusted to be strengthened or weakened according to each evaluation value. In addition to such backpropagation, various known learning methods such as so-called autoencoder, restricted Boltzmann machine, dropout, noise addition, and sparse regularization are used in combination to improve processing accuracy. May be good. The learning phase of the coordinating unit 32 corresponds to the machine learning process described in each claim.

As described above, the machine learning algorithm of the adjustment unit 32 is not limited to the one based on the illustrated deep learning, but other machine learning algorithms (not shown in particular) using, for example, a support vector machine or a Bayesian network are applied. You may. Even in that case, the basic configuration of outputting the control parameters appropriately corresponding to the input state quantity data is the same.

<About data learning of machine learning process>
The specific procedure of the machine learning process for the adjustment unit 32 configured by the neural network will be described below. FIG. 7 shows a flowchart of a processing procedure when the CPU 901 of the motor control device 3 (see FIG. 12 described later) executes a machine learning process by data learning in the example of the present embodiment. The data learning process shown in this flow starts when, for example, a command is input from the host control device 2 to execute the machine learning process.

First, in step S5, the CPU 901 temporarily sets each control parameter (Kp, T, Kv in this example). The temporary setting of each control parameter is set by a combination of random values within an appropriate settable range. Alternatively, an artificial design value considered to be appropriate for the feedback control system to be applied may be individually increased or decreased and set.

Next, the process proceeds to step S10, and the CPU 901 selects one of various adjustment sequence operations prepared in advance.

Next, the process proceeds to step S15, and the CPU 901 inputs the position command of the adjustment sequence selected in step S10 to the feedback control unit 31 to cause the motor / drive machine 60 to execute the corresponding sequence operation.

Next, the process proceeds to step S20, and the CPU 901 records each state quantity data of the position command, the torque command, and the detection position during the execution of the adjustment sequence operation in step S15.

Next, the process proceeds to step S25, and the CPU 901 creates one adjustment unit learning data set with each state quantity data recorded in step S20 and each control parameter tentatively set in step S5. The dataset also includes evaluation values for responsiveness obtained based on each state quantity data and other methods.

Next, the process proceeds to step S30, and the CPU 901 determines whether or not a data set has been created for various adjustment sequence operations prepared in advance. If the adjustment sequence operation for which the data set has not been created remains, the determination is not satisfied, and the process returns to step S10 and the same procedure is repeated.

On the other hand, when a data set is created for all the adjustment sequence operations prepared in advance, the determination is satisfied and the process proceeds to step S35.

In step S35, the CPU 901 changes the temporarily set control parameters and newly repeats various adjustment sequence operations to determine whether or not to create the adjustment unit learning data set. In other words, it determines whether to end the creation of the dataset. When temporarily setting a new control parameter and continuing the creation of the data set, the determination is not satisfied, the process returns to step S5, the new control parameter is temporarily set, and the same procedure is repeated.

On the other hand, when the creation of the data set is completed, the determination is satisfied, and the process proceeds to step S40.

In step S40, the CPU 901 executes data learning of the adjustment unit 32 using the created adjustment unit learning data set. Then, this flow ends.

The machine learning process by data learning processing in offline learning (batch learning) as described above may be performed before the start of operation of the machine control system 1, or the rotary motor 4 and the drive machine 5 may be aged. It may be performed when necessary after the start of operation for the purpose of improving the control accuracy according to the deterioration status and the change of the usage environment.

<Effect of this embodiment>
As described above, the machine control system 1 of the present embodiment has an adjustment unit 32 that adjusts the control parameters of each controller included in the feedback control unit 31 based on the learning content in the machine learning process. .. As a result, by executing the machine learning process based on the state quantity data detected during the appropriate sequence operation, the adjusting unit 32 can set appropriate control parameters according to the learning content. That is, it is possible to mechanically and automatically adjust the control parameters autonomously by the motor control device 3 itself without human intervention. Therefore, it is possible to reduce human cost and time cost for adjusting the control parameters in the motor control device 3, and it is possible to ensure uniform and stable control accuracy regardless of the skill difference of each engineer. As a result, the convenience of the control device can be improved.

In the present embodiment, the case where the control parameters are adjusted only for the feedback control system has been described, but the present invention is not limited to this. For example, the method of the adjusting unit 32 of the present embodiment may be applied to the control system including the feedforward control system for adjusting the control parameters such as the feedforward gain. Further, when the control system includes various observers and compensators, the same method may be applied to the adjustment of various control parameters used inside them. Further, the present invention is not limited to the so-called motion control as in the present embodiment, and the same method may be applied to the adjustment of the control parameters of the so-called process control.

Further, in the present embodiment, in particular, the adjusting unit 32 sets control parameters corresponding to the mechanical constants (or disturbances described later) of the motor / drive machine 60 estimated from the correspondence between the position command and the detection position. As a result, the adjusting unit 32 appropriately corresponds to the mechanical constant (or disturbance described later) of the motor / driving machine 60 (controlled object P) specifically estimated for each controller of the feedback control unit 31. Control parameters can be set.

Further, in the present embodiment, particularly in the machine learning process, data learning (offline learning) using a data set for adjusting unit learning in which at least one of a position command, a torque command, and a detection position is associated with a temporarily set control parameter is used. , Batch learning) to make the adjustment unit 32 machine learn. As a result, it is possible to execute a specific machine learning process by supervised learning using a data set for the adjustment unit 32. Further, by performing the data learning by online learning, it is possible to always set appropriate control parameters in response to changes in the aging deterioration state and the usage environment state during the operation of the machine control system 1. It should be noted that the adjustment unit detects environmental data (ambient temperature, attitude, etc. of the drive machine 5) that may affect the control accuracy in the sequence operation of the motor / drive machine 60 with a separate sensor and includes it in the adjustment unit learning data set. You may let 32 learn the data.

Further, in the present embodiment, in particular, the position command, torque command, and detection position of the adjustment unit learning data set are time-series pattern data at the time of the same sequence operation. As a result, the adjusting unit 32 can perform machine learning corresponding to the change mode of each state quantity data, and can set highly versatile control parameters appropriately corresponding to various sequence operations. The same feedback control unit 31 is changed to various motors / drive machines 60 to create a data set for learning the adjustment unit, and the same adjustment unit 32 learns the data set, so that the adjustment unit 32 has various motors. -Control parameters that flexibly correspond to the drive machine 60 can be adjusted, and the versatility of the feedback control unit 31 is improved.

Further, in the present embodiment, in particular, the adjustment unit learning data set includes evaluation values, and in the machine learning process, machine learning is performed based on the evaluation values. As a result, in the machine learning process of the adjustment unit 32, the data learning of each adjustment unit learning data set can be machine-learned according to the evaluation value, and the control accuracy of the feedback control unit 31 can be further improved.

Further, in the present embodiment, in particular, the control parameter includes at least one of the position loop gain Kp, the speed loop gain Kv, and the speed loop integration time constant T (and the moment of inertia ratio of the motor / drive machine 60). This makes it possible to set suitable control parameters for each controller included in the feedback control unit 31. Further, if necessary, other control parameters such as model tracking control gain, torque command filter time constant, notch filter frequency, notch filter Q value, or notch filter depth may be included.

<Modification example>
It should be noted that the embodiments described above can be variously modified within a range that does not deviate from the purpose and technical idea.

<Transformation example 1: When the machine learning process is performed by reinforcement learning>
In the above embodiment, the machine learning process of the appropriate control parameters corresponding to the state quantity data is performed by data learning, but the present invention is not limited to this. In addition, the neural network of the adjustment unit 32 may be learned by the machine learning process of the reinforcement learning process as illustrated in FIG. The reinforcement learning process of this modified example shown in the figure is performed based on the so-called Q-learning method.

First, in step S105, the CPU 901 temporarily sets each control parameter. The initial temporary setting of each control parameter is set by an artificial design value considered to be appropriate for the feedback control system to be applied.

Next, the process proceeds to step S110, and the CPU 901 searches for new control parameters. Regarding the search for this new control parameter, from the control parameter set as being appropriate (the so-called Q value described later is the maximum) up to that point, the control parameter corrected in a random direction with a small probability is set, or The control parameters that are considered appropriate for most other probabilities remain unchanged.

Next, the process proceeds to step S115, and the CPU 901 executes a predetermined adjustment sequence operation with the control parameters searched and set in step S110.

Next, the process proceeds to step S120, and the CPU 901 records each state quantity data of the position command, the torque command, and the detection position during the execution of the adjustment sequence operation in step S115.

Next, the process proceeds to step S125, and the CPU 901 evaluates the current adjustment sequence operation based on each state quantity data recorded in step S120, and calculates a reward according to this evaluation.

Next, the process proceeds to step S130, and the CPU 901 performs reinforcement learning on the neural network of the adjustment unit 32 by so-called Q-learning based on the control parameters searched and set in step S110 and the reward calculated in step S125. The control parameters are learned so that the Q value indicating the effectiveness of the set control parameters is maximized with respect to the trial of the sequence operation in step S115. For the reinforcement learning by Q-learning, a known method may be used, and detailed description thereof will be omitted here.

Next, the process proceeds to step S135, and the CPU 901 determines whether or not machine learning has been performed on the neural network of the adjusting unit 32 with a sufficient number of trials. If the machine learning is insufficient and the reinforcement learning process is not completed, the determination is not satisfied, and the process returns to step S110 and the same procedure is repeated.

On the other hand, when the machine learning is sufficient and the reinforcement learning process is terminated, the determination is satisfied and this flow is terminated.

As described above, in the machine control system 1 of this modification, the machine learning process machine-learns control parameters by reinforcement learning (Q-learning, etc.) based on at least one of a position command, a torque command, and a detection position. .. As a result, it is possible to set control parameters that realize high control accuracy without preparing a large number of learning data sets in advance.

Further, in the present modification, in particular, the reinforcement learning is performed based on the reward for setting the control parameter at the time of the predetermined adjustment sequence operation. As a result, the adjusting unit 32 can set highly responsive control parameters that appropriately correspond to various sequence operations.

<Modification example 2: When setting appropriate control parameters for disturbance addition>
In the above embodiment, the adjusting unit 32 sets the control parameters from the viewpoint of improving the responsiveness in the sequence operation of the feedback control system, but the present invention is not limited to this. In addition, the adjusting unit 32 may set control parameters from the viewpoint of improving robustness against disturbances (particularly dynamically added disturbances) added to the controlled object.

In this case, for example, as shown in FIG. 9, the adjustment unit learning data including the time-series pattern data of each state quantity data recorded at the time of the same disturbance addition that can be assumed when the machine control system 1 is in operation. Create a set and learn data. In the adjustment unit learning data set of the illustrated example, each state quantity data when an object of a predetermined weight is placed on the moving slider 52 and an evaluation value for robustness are exemplified. The difference between the disturbance addition and the sequence operation in the above embodiment may be used as the operation condition c shown in FIG. 2 to distinguish the adjustment unit learning data set.

As described above, in the machine control system 1 of this modification, the position command, torque command, and detection position state quantity data of the adjustment unit learning data set are time-series pattern data when the same disturbance is added. .. As a result, the adjusting unit 32 can set control parameters that can improve robustness against various disturbances.

Further, the adjusting unit 32 may learn the setting of the control parameter for such disturbance addition by the reinforcement learning described above. In this case, the reinforcement learning is performed based on the reward for setting the control parameter at the time of adding a predetermined disturbance. Even with such reinforcement learning, the adjustment unit 32 can set control parameters that can improve robustness against various disturbances.

<Modification example 3: When the adjustment part is divided and the machine constant is output>
In the above embodiment, an example in which the adjusting unit 32 is configured by a continuous integrated neural network has been described, but the present invention is not limited to this. Alternatively, as shown in FIG. 10, the neural network constituting the adjusting unit 32 may be divided into two in the layer direction, and a mechanical constant (or disturbance) may be output between them. For example, the neural network on the input layer side (left side in the figure) for inputting state quantity data is, for example, the first neural network 71, and the neural network on the output layer side (right side in the figure) for outputting control parameters is, for example, the second neural network. Let the neural network 72 be. In the figure, in order to avoid complication of illustration, each neural network is shown by a rectangular block by omitting the illustration of nodes and edges.

In this case, in the first neural network 71, for each time series pattern data of the position command, the torque command, and the detection position, which are the state quantity data input from each part, from the correspondence relationship between the state quantity data. It is designed to output an estimated mechanical constant (or disturbance). In the illustrated example, the moment of inertia, elastic modulus, first resonance frequency, and second resonance frequency are illustrated as mechanical constants (in the case of disturbance, for example, the added disturbance mass or disturbance elasticity; not shown). In the machine learning process of the first neural network 71, for example, each state quantity data is recorded at the time of a predetermined adjustment sequence operation, and the machine constant of the controlled object P is accurately detected by a separate detection machine, detection test, or the like. , A training data set for the first neural network 71 that puts them together is created. Then, by executing data learning using a large number of training data sets, the first neural network 71 learns a feature quantity representing the correlation between each state quantity data and the machine constant.

The control parameters temporarily set during the above adjustment sequence operation are set as temporary setting control parameters (Kp', Kv', T'shown in the figure), and as shown in the figure, the first neural network 71 together with the state quantity data. The estimation accuracy of the mechanical constant (or disturbance) to be output by inputting may be improved. In this case, the temporary setting control parameters are also recorded together with the learning data set and used for data learning of the first neural network 71. In the provisional control parameters input to the first neural network 71, the control parameters to be output to the second neural network 72 are referred to as adaptive control parameters as shown in the figure below.

Further, in the second neural network 72, the adaptive control parameters Kp, Kv that are appropriate for the mechanical constants (or disturbances) input from the first neural network 71 based on the correspondence between these mechanical constants. , T is designed to be output. In the machine learning process of the second neural network 72, adaptive control parameters are artificially set separately in order to perform a predetermined adjustment sequence operation on a large number of control targets having known machine constants (or disturbances). A training data set for the second neural network 72 is created by setting appropriately and summarizing these machine constants and adaptive control parameters. Then, by executing data learning using a large number of training data sets, the second neural network 72 learns the features representing the correlation between each machine constant (or disturbance) and the adaptive control parameter. ..

As described above, the machine control system 1 of this modification can output the machine constant (or disturbance) as a specific value, and by referring to this, the adaptive control parameter set by the adjusting unit 32 can be used. The validity can be easily examined.

<Modification example 4: When the adjustment unit is pre-learned>
In the above embodiment, the method of causing the adjustment unit 32 to learn data by using only the adjustment unit learning data set has been described from the beginning, but the present invention is not limited to this. In addition, a control parameter that is generally appropriate may be calculated by a known calculation method, and the adjusting unit 32 may be trained in advance using the control parameter.

For example, as shown in FIG. 11, an estimation unit 33 for calculating a temporary setting control parameter based on each state quantity data is provided in the motor control device 3A, and the calculated temporary setting control parameter is input to the adjusting unit 32. In this case, the estimation unit 33 obtains the machine constant of the control target P from each state quantity data detected by the predetermined adjustment sequence operation by a known calculation method, and obtains the machine constant and the state equation corresponding to the feedback control system. Estimate the temporary setting control parameters based on this. The temporary setting control parameters estimated by the estimation unit 33 do not strictly correspond to changes in individual differences (part manufacturing error, assembly error, etc.), operating conditions, aging deterioration status, and usage environment status of the control target P. However, it is generally appropriate in the mathematical model by design, and there is only a slight difference from the adaptive control parameters that are finally obtained. Therefore, by temporarily setting the temporary setting control parameters estimated by the estimation unit 33, the machine learning process of the adjustment unit 32 only performs fine adjustment (fine tuning), that is, pre-learning can be performed.

As described above, the machine control system 1 of this modification has an estimation unit 33 that estimates control parameters by a predetermined sequence operation, and the machine learning process pre-learns with the control parameters estimated by the estimation unit 33. .. As a result, the learning time required for the machine learning process of the adjusting unit 32 can be significantly reduced.

Further, although not particularly shown, the estimation unit 33 may estimate only the mechanical constant of the control target P from each state quantity data based on a known calculation method and input it to the adjustment unit 32. In this case, the pre-learning of the portion corresponding to the first neural network shown in FIG. 10 can be performed, and the learning time can be significantly shortened.

<Hardware configuration example of motor control device>
Next, with reference to FIG. 12, a hardware configuration example of the motor control device 3 that realizes processing by the feedback control unit 31, the adjustment unit 32, and the like implemented in software by the program executed by the CPU 901 described above will be described. explain.

As shown in FIG. 12, the motor control device 3 includes, for example, a CPU 901, a ROM 903, a RAM 905, a dedicated integrated circuit 907 constructed for a specific application such as an ASIC or an FPGA, an input device 913, and an output device. It has a 915, a recording device 917, a drive 919, a connection port 921, and a communication device 923. These configurations are connected so that signals can be transmitted to each other via the bus 909 and the input / output interface 911.

The program can be recorded in, for example, a ROM 903, a RAM 905, a recording device 917, or the like.

Further, the program can be temporarily or permanently recorded on, for example, a magnetic disk such as a flexible disk, an optical disk such as various CDs, MO disks, or DVDs, or a removable recording medium 925 such as a semiconductor memory. .. Such a recording medium 925 can also be provided as so-called package software. In this case, the program recorded on these recording media 925 may be read by the drive 919 and recorded on the recording device 917 via the input / output interface 911, the bus 909, or the like.

The program can also be recorded on, for example, a download site, another computer, another recording device, or the like (not shown). In this case, the program is transferred via a network NW such as LAN or the Internet, and the communication device 923 receives this program. Then, the program received by the communication device 923 may be recorded in the recording device 917 via the input / output interface 911, the bus 909, or the like.

Further, the program can be recorded in an appropriate externally connected device 927, for example. In this case, the program may be transferred via the appropriate connection port 921 and recorded in the recording device 917 via the input / output interface 911, the bus 909, or the like.

Then, the CPU 901 executes various processes according to the program recorded in the recording device 917, so that the processes by the feedback control unit 31, the adjusting unit 32, and the like are realized. At this time, for example, the CPU 901 may directly read the program from the recording device 917 and execute it, or may execute it after loading it into the RAM 905 once. Further, for example, when the CPU 901 receives the program via the communication device 923, the drive 919, or the connection port 921, the CPU 901 may directly execute the received program without recording it in the recording device 917.

Further, the CPU 901 may perform various processes based on signals and information input from an input device 913 such as a mouse, keyboard, microphone (not shown), if necessary.

Then, the CPU 901 may output the result of executing the above processing from an output device 915 such as a display device or an audio output device, and the CPU 901 may connect the processing result to the communication device 923 or the communication device 923 as needed. It may be transmitted via the port 921, or may be recorded on the recording device 917 or the recording medium 925.

In the above description, when there is a description such as "vertical", "parallel", "plane", etc., the description does not have a strict meaning. That is, these "vertical", "parallel", and "planar" mean "substantially vertical", "substantially parallel", and "substantially flat", with design and manufacturing tolerances and errors allowed. ..

Further, in the above description, when there is a description such as "same", "same", "equal", "different", etc. in the external dimensions, size, shape, position, etc., the description is not a strict meaning. That is, those "same", "equal", and "different" are allowed for design and manufacturing tolerances and errors, and are "substantially the same", "substantially the same", "substantially equal", and "substantially equal". It means "different".

In addition to the above, the methods according to the above-described embodiment and each modification may be appropriately combined and used. In addition, although not illustrated one by one, the above-described embodiment and each modification are implemented with various modifications within a range that does not deviate from the purpose.

1 Machine control system 2 Upper control device 3,3A Motor control device (control device)
4 rotary motor (controlled object)
5 Drive machine (controlled object)
31 Feedback control unit 32 Adjustment unit 33 Estimate unit 60 Motor / drive machine (controlled object)
71 1st neural network (adjustment part)
72 Second neural network (adjustment section)
DB database

Claims (11)

A feedback control unit that inputs the deviation between the reference command and the control amount output by the external control target to the controller that controls with a predetermined control parameter, and controls the control target with the operation amount output by the controller. ,
An adjustment unit that adjusts the control parameters based on what is learned in the machine learning process,
Have a,
The adjusting part
A first neural network that inputs the reference command, the manipulated variable, and the controlled variable and outputs the mechanical constant or disturbance of the controlled object.
A second neural network that inputs the mechanical constant or disturbance output from the first neural network and outputs the control parameters,
A control device characterized by having . In the machine learning process, data learning is performed using a first data set in which the reference command, the operation amount, and the control amount are associated with the machine constant or disturbance of the control target detected by the detection machine or the detection test. to machine learning the first neural network, Ru is machine learning the second neural network by data learning using second data set that associates the control parameter set known mechanical constant or disturbance and artificially The control device according to claim 1 , wherein the control device is characterized by the above. The control device according to claim 2 , wherein the reference command, the operation amount, and the control amount of the first data set are time-series pattern data at the time of the same sequence operation. The control device according to claim 2 or 3 , wherein the reference command, the operation amount, and the control amount of the first data set are time-series pattern data at the time of adding the same disturbance. The first data set or the second data set contains evaluation values.
The control device according to any one of claims 2 to 4 , wherein in the machine learning process of the first neural network or the second neural network, machine learning is performed based on the evaluation value. And in the the machine learning process of the first neural network, the reference command, the operation amount, and a machine constant or disturbance of the controlled object machine learning by reinforcement learning based on the control amount, the machine of the second neural network in the learning process, the control device according to claim 1, characterized in that the machine learning the control parameter by reinforcement learning based on the mechanical constant or disturbance. The reinforcement learning of the first neural network is reinforcement learning based on the reward for setting the machine constant or the disturbance of the controlled object at the time of a predetermined sequence operation , and the reinforcement learning of the second neural network is the predetermined The control device according to claim 6, wherein reinforcement learning is performed based on a reward for setting the control parameter at the time of sequence operation . The reinforcement learning of the first neural network is reinforcement learning based on the reward for setting the mechanical constant or the disturbance of the controlled object at the time of adding a predetermined disturbance, and the reinforcement learning of the second neural network is the predetermined The control device according to claim 6 or 7, wherein reinforcement learning is performed based on a reward for setting the control parameter when a disturbance is added . The control parameter according to any one of claims 1 to 8 , wherein the control parameter includes at least one of a position loop gain, a speed loop gain, a speed loop integration time constant, and a moment of inertia ratio of the controlled object. Control device. Further having an estimation unit for estimating the control parameters in a predetermined sequence operation,
The machine learning process of the first neural network is pre-learned by the machine constant or disturbance of the controlled object estimated by the estimation unit, and the machine learning process of the second neural network is estimated by the estimation unit. The control device according to any one of claims 1 to 9 , wherein the control device is pre-learned with control parameters. The deviation between the reference command and the control amount output by the external control target is input to the controller controlled by a predetermined control parameter, and the control target is controlled by the operation amount output by the controller.
Adjusting the control parameters based on what is learned in the machine learning process,
Have,
Adjusting the control parameters
Inputting the reference command, the manipulated variable, and the controlled variable in the first neural network to output the mechanical constant or disturbance of the controlled object.
In the second neural network, the mechanical constant or disturbance output from the first neural network is input and the control parameter is output.
A control method characterized by having .

Images