JP7118249B2 - motor controller - Google Patents

Description

The present invention relates to a motor control device that generates a model imitating a controlled object.

High-speed and high-precision control methods and failure detection methods are required for motor control devices that control machines to be controlled, such as robots and machine tools. For these performance improvements, a model that accurately simulates the controlled object is essential. However, in the actual operation of a motor control device, errors in physical model parameters such as mass, center of gravity position, and inertia tensor due to wiring in the controlled object, attachment of end effectors, etc. cannot be ignored. On the other hand, many nonlinear components are also included. Therefore, it has been difficult to provide a model capable of reproducing these with high accuracy in advance.

In recent years, control methods have been proposed for suppressing force and torque estimation errors in such models. For example, in Patent Document 1, an electric motor control device acquires an M-sequence or sinusoidal command, and corrects the inertia tensor, friction coefficient, and spring constant for each sampling period so that the torque estimation error is minimized. A technique is disclosed for correcting the error contained in the initial values of the physical model parameters during operation by doing so.

JP 2013-128387 A

However, in the technique described in Patent Document 1, it is assumed that the motion of the motor control device can be modeled by inertia tensor, friction, and torque from the spring, and disturbances that are not considered as a model cannot be expressed. There was a problem. In addition, since the friction model is also simplified, there is a problem that it does not have the ability to express highly nonlinear and complex phenomena such as the Stribeck effect and hysteresis.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a motor control device capable of learning a model with improved torque estimation accuracy and generalization performance and improving the speed and accuracy of abnormality detection. do.

In order to solve the above-described problems and achieve the object, the present invention is a motor control device for controlling a motor that drives a target machine. The motor control device includes a motor state observation unit that observes the state of the motor and outputs the observation result as a motor state signal, and an estimated value of the torque of the motor using the equation of motion based on the motor state signal and the physical model parameters of the target machine. A first torque estimator that outputs a first estimated torque and a physical state variable that is a component of the first estimated torque, and a difference between the actual torque and the first estimated torque included in the motor state signal a physical model storage unit that learns and stores physical model parameters to be reduced; a feature amount generation unit that generates and outputs a feature amount from at least one of the first estimated torque, the physical state variable, and the motor state signal; Prepare. Further, the motor control device calculates a machine learning correction amount from the feature amount and the machine learning model parameters using machine learning including an autoregressive term in the input/output relationship, and applies the calculated machine learning correction amount to the first estimated torque. A second torque estimator that outputs a second estimated torque by addition, and a machine learning model parameter learned by supervised learning so that the machine learning correction amount approaches the difference between the actual torque and the first estimated torque. and a machine learning model storage unit that stores learned machine learning model parameters.

Advantageous Effects of Invention According to the present invention, a motor control device can learn a model with improved torque estimation accuracy and generalization performance, and can improve the speed and accuracy of abnormality detection.

Block diagram showing a configuration example of a motor control device Schematic diagram showing a target machine to be controlled by the motor control device A diagram showing an example of a nonlinear friction model, which is a physical model parameter estimated in the physical model storage unit. Block diagram showing a configuration example of the feature generator and the machine learning model calculator Diagram showing autoregression terms in the configuration of the feature generator and machine learning model calculator Flowchart showing the flow of learning physical model parameters and machine learning model parameters in a motor control device FIG. 4 is a diagram showing a motor speed representing an operation of a target machine controlled by the motor control device, a first estimated torque estimated by the motor control device, and a second estimated torque estimated by the motor control device; FIG. 4 is a diagram showing an example of a case in which a processing circuit included in a motor control device is configured with a processor and a memory; FIG. 4 is a diagram showing an example of a case where a processing circuit included in a motor control device is configured with dedicated hardware;

A motor control device according to an embodiment of the present invention will be described in detail below with reference to the drawings. In addition, this invention is not limited by this embodiment.

Embodiment.
FIG. 1 is a block diagram showing a configuration example of a motor control device 100 according to an embodiment of the invention. The motor control device 100 controls a motor that drives the target machine 10 . A motor control device 100 includes a motor state observation unit 1 , a physical model calculation unit 2 , a feature quantity generation unit 3 , and a machine learning model calculation unit 4 . The physical model calculator 2 includes a first torque estimator 21 and a physical model storage 22 . The machine learning model calculator 4 includes a second torque estimator 41 and a machine learning model storage 42 .

A target machine 10 to be controlled by the motor control device 100 will be described. FIG. 2 is a schematic diagram showing the target machine 10 to be controlled by the motor control device 100 according to the present embodiment. Here, as the target machine 10, a horizontal articulated robot will be described as an example. A horizontal articulated robot, which is the target machine 10, has a first axis 11j, a second axis 12j, a third axis 13j, a fourth axis 14j, a first link 11l, a second link 12l, and a third link. 13l, the fourth link 14l, and the end portion 15e. The motor control device 100 drives the second link 12l by the actuator mounted on the first shaft 11j, drives the third link 13l by the actuator mounted on the second shaft 12j, and drives the third shaft 13j and the fourth shaft. By driving the fourth link 14l with an actuator mounted on the 14j, the position and orientation of the hand portion 15e of the horizontal articulated robot can be controlled. Here, the third axis 13j corresponds to the rotational movement of the fourth link 14l about its axis in the direction of the hand portion 15e, and the fourth axis 14j corresponds to the translational movement of the fourth link 14l in the direction of the hand portion 15e.

In the present embodiment, a horizontal articulated robot is exemplified as the target machine 10, but it is not limited to this, and the motor control device 100 can be widely applied to target machines 10 driven by motors. Examples of the target machine 10 further include vertical articulated robots, NC (Numerical Control) machine tools, mounting machines, and the like.

The motor state observation unit 1 observes the state of the motor that drives the target machine 10 and outputs the observation result as a motor state signal. Specifically, the motor state observation unit 1 detects at least one of the position, velocity, and acceleration of each actuator corresponding to the first axis 11j, the second axis 12j, the third axis 13j, and the fourth axis 14j. , and the actual torque, which is the torque of the motor, as a motor state signal. Here, the actual torque may be a value directly measured from a sensor, or may be a converted value of an input current value obtained from an actuator. Also, the motor status signals are not limited to the position, velocity, acceleration, and actual torque signals of each actuator. The motor status signals are sensing data from internal or external sensors provided on each axis of the robot or at specified locations, such as temperature data from actuators on each axis, data obtained from hand force sensors, infrared sensors, Values such as distance data obtained from a sensor such as an ultrasonic sensor, moving image data obtained from a vision sensor, and feature amount data obtained by image processing thereof may be included.

The first torque estimating unit 21 receives the motor state signal and the physical model parameters described later, which are the outputs of the physical model storage unit 22, and calculates the first torque, which is the estimated value of the actual torque, which is the torque of the motor. Estimated torque is calculated by the equation of motion. That is, the first torque estimator 21 estimates the first estimated torque, which is the estimated torque of the motor, using the equation of motion based on the motor state signal and the physical model parameters of the target machine 10 . The first torque estimator 21 outputs a first estimated torque and physical state variables that are components of the first estimated torque. The physical state variables include not only inertia terms, centrifugal force terms, Coriolis force terms, and gravitational terms, but also Coulomb friction terms, viscous friction terms, arm vibration terms, and elastic deflection terms. A term of a function introduced by simulating some physical phenomenon in calculating the first estimated torque, such as a non-linear friction model, may be included. By including the estimated friction value in the physical state variables, the motor control device 100 corrects the uncertainty of the friction model and the error of the estimated friction value with respect to the friction component of the actual torque in the machine learning model calculation unit 4, which will be described later. to enable.

The motor state signals and physical model parameters used in the equations of motion correspond to the position, velocity and acceleration of each actuator and the mass, center of gravity position and inertia tensor of each link, respectively. That is, the physical model parameters include the mass of the link mechanism of the target machine 10 , the center of gravity position of the link mechanism of the target machine 10 , and the inertia tensor of the link mechanism of the target machine 10 . As a result, the motor control device 100 learns the mass of the coefficients, the position of the center of gravity, and the inertia tensor as physical model parameters in the inverse dynamics using the information about the known mechanism of the target machine 10, and the first estimated torque with respect to the actual torque. can improve the estimation accuracy of Further, in the motor control device 100, the correction value in the machine learning model calculation unit 4, which will be described later, becomes relatively small, and the generalization performance of the second estimated torque can be improved. Here, as the equation of motion, for example, the Newton-Euler method, the Lagrangian equation of motion, or the like can be used.

The first torque estimator 21 uses an equation of motion including a continuous and smooth friction model that outputs an estimated value of the frictional component of the torque with the speed of the motor obtained from the motor state signal as input. As will be described later, the machine learning model calculation unit 4 uses a neural network, but since the neural network can calculate a continuous function with respect to the input, the feature amount and the input to the feature amount generation unit 3 are continuous. is desirable. Therefore, the motor control device 100 defines a continuous friction model including near-zero speed, so that even the friction component that cannot be learned in the physical model storage unit 22 can be learned and corrected using a neural network. becomes possible.

The physical model storage unit 22 uses the motor state signal obtained when the target machine 10 is operated for a certain period of time as teaching data, and stores the physical model parameters so as to reduce the difference between the actual torque included in the motor state signal and the first estimated torque. to learn. The physical model storage unit 22 stores learned physical model parameters. By learning the physical model parameters, the motor control device 100 can correct changes in physical properties due to attachment of sensors and wiring, processing errors, assembly errors, etc., and can improve the accuracy of estimating the actual torque. In general, physical model parameters can be roughly calculated from specification tables, CAD (Computer Aided Design) data, etc. By giving these values as initial values at the time of learning, the speed of learning can be improved. , convergence, etc. can be improved. However, if these values are unknown or difficult to calculate, they may be initialized with arbitrary constants.

For the learning of the physical model parameters in the physical model storage unit 22, linear equations of an overdetermined system obtained by transforming the above equation of motion and separating only the physical model parameters are used. Newton's equations of motion, Euler's equations of motion, and Lagrangian's equations of motion, when none of the nonlinear terms for position, velocity, and acceleration are considered, have constant coefficients consisting of mass, center of gravity position, and inertia tensor, position, It can be expressed as a linear combination of products with variables consisting of velocity and acceleration. That is, if the error in the dynamics calculation with respect to the actual torque is Δf, the equation of motion for each axis consists of the physical model parameter w consisting of the mass, the position of the center of gravity, and the inertia tensor, and the coefficient matrix A (v , a) can be used to describe in a variable-separated form as shown in Equation (1).

Δf=A(v,a)w (1)

Therefore, the physical model storage unit 22 can calculate the physical model parameter of each axis by solving the equation (1) for w for each axis by the least squares method. In addition, the physical model storage unit 22 converts the equation (1) regarding w of each axis into one linear By summarizing the equations and solving them simultaneously, a more plausible solution can be obtained for the physical model calculation unit 2 as a whole. The physical model storage unit 22 may use, in addition to the least squares method, the regularized least squares method, the Kalman filter, Bayesian estimation, a genetic algorithm, a neural network, etc., for the identification method of the physical model parameter w.

In addition, when considering the effect of friction in the equation of motion, the physical model storage unit 22 stores a linear combination of a function of velocity and a friction coefficient, such as a linear friction model that is a friction model consisting of Coulomb friction and viscous friction shown below. By expressing the friction with a separable equation, the friction coefficient can be simultaneously estimated as a physical model parameter in the same manner as the equation (1). The coefficient of friction estimated by the physical model storage unit 22 is expressed by Equation (2). In equation (2), which is captured as an image, and equation (3), which will be described later, a dot is added above q _i , but this cannot be expressed in the description. Therefore, in the explanation part, q _i with a dot above it is simply written as q _i .

where r _FC,i is the Coulomb friction coefficient and r _FV,i is the viscous friction coefficient for a friction model F _i of axis i operating at velocity q _i . However, in the operation of multi-axis mechanisms such as industrial robots, the viscous friction does not have a completely linear relationship with the speed. It is often confirmed that it is smaller than the In such a case, the friction is estimated using a friction model with a higher degree of freedom that considers the nonlinearity with respect to speed, and after removing the friction component in the difference between the actual torque and the first estimated torque, other physical It is desirable to estimate the model parameters. An example of such a non-linear friction model is the following function represented by Equation (3).

where ω _ls is the boundary speed for low speed motion, ω _hs is the boundary speed for high speed motion, and r _ST,i is the saturation coefficient of friction. A graph of the nonlinear friction model is shown in FIG. FIG. 3 is a diagram showing an example of a nonlinear friction model, which is a physical model parameter estimated by the physical model storage unit 22 according to this embodiment. In FIG. 3, the horizontal axis indicates motor speed and the vertical axis indicates friction. The nonlinear friction model has a total of four parameters in addition to the Coulomb friction coefficient rFC _,i and the viscous friction coefficient _{rFV,i .} is obtained using the Levenberg-Marquardt method, which is a nonlinear least-squares method, from the difference from the estimated torque. Since the Levenberg-Marquardt method has initial value dependence, once linearity is assumed for friction, the Coulomb friction coefficient r _FC,i and the viscous friction coefficient r _FV,i are added to the physical model parameter w, and the above equation ( After calculating each friction coefficient by 1), by using them as initial values, estimation failure due to initial value dependence can be prevented. That is, the physical model storage unit 22 can use the least-squares method, the regularized least-squares method, the nonlinear least-squares method, the Kalman filter, or the like in learning the physical model parameters. The motor control device 100 can estimate unknown coefficients in the motion equation and the state equation of the target machine 10 from the operation data by using the least squares method, the nonlinear least squares method, the nonlinear least squares method, or the Kalman filter. Become. Operational data is, for example, data contained in a motor status signal.

The physical model storage unit 22 stores the coefficients of the nonlinear friction model obtained here as physical model parameters, and adds the output of the nonlinear friction model at each time to the physical state variables. By storing coefficients of the friction model as physical model parameters in the physical model storage unit 22, the motor control device 100 can detect changes in viscous friction and Coulomb friction due to deterioration over time and abnormalities. The physical model storage unit 22, for example, alternately estimates coefficients of the nonlinear friction model and other physical model parameters in learning the physical model parameters. By alternately estimating coefficients of the friction model and other physical model parameters, the motor control device 100 can design the friction model in a more complex and detailed manner. In addition, since the motor control device 100 can also estimate other physical model parameters in a state in which the influence of friction is almost eliminated, the physical model parameters can be learned with higher accuracy.

However, the nonlinear friction model is not limited to a function of velocity, and may be a multivariable function that depends on position, acceleration, motor temperature, etc. Each coefficient of the nonlinear friction model is Gauss-Newton method, steepest descent method , an extended Kalman filter, an unscented Kalman filter, or the like. As the nonlinear friction model, a support vector regression model, regression tree, random forest, multivariate adaptive regression spline model, neural network, or other model generation method based on machine learning algorithms may be used. In the calculation unit 4, the first estimated torque is corrected by the machine learning unit 411 including the neural network. It is sufficient to consider it as a function of only the velocity given to .

The feature amount generation unit 3 receives the first estimated torque, the physical state variable, and the motor state signal as inputs, and outputs the feature amount that is the input of the machine learning unit 411 . The actual torque, which is the input and the estimation target of these feature quantity generators 3, is a value that changes in time series. Since the machine learning model storage unit 42 also needs to consider the correlation in the direction of the time axis, it is given as a vector containing both the feature amount and the value for a certain period in the past. In addition, in a multi-axis mechanism such as a horizontal articulated robot targeted in this embodiment, since the influence of mutual interference between axes is not small, It is desirable to However, the torque component in the feature quantity is given as the difference between the actual torque and the first estimated torque, and only past values are used for the actual torque to be estimated.

FIG. 4 is a block diagram showing a configuration example of the feature quantity generation unit 3 and the machine learning model calculation unit 4 according to this embodiment. In FIG. 4, z is a unit delay operator, and _Nd is an integer greater than or equal to 0 indicating the number of unit times of delay. The unit delay operator shown in FIG. 4 can also be said to be a storage device provided in the feature generator 3 . That is, the feature quantity generation unit 3 includes a storage device capable of storing the first estimated torque, the physical state variables, and the motor state signal, and the first estimated torque, the physical state variables, and What is extracted from the motor state signal may be output as the feature quantity. The motor control device 100 includes information on time-series changes related to the physical state variables and the motor state signal as input in the feature amount, so that the machine learning model calculation unit 4 can predict the operation of the target machine 10 in more detail. It is possible to improve the estimation accuracy of the second estimated torque with respect to the torque.

The feature amount generation unit 3 may generate and output the feature amount using only some of the first estimated torque, the physical state variable, and the motor state signal without using all of them. That is, the feature quantity generator 3 may generate and output a feature quantity from at least one of the first estimated torque, the physical state variable, and the motor state signal.

The second torque estimating section 41 includes a machine learning section 411 . The second torque estimating unit 41 is the feature quantity generated by the feature quantity generating unit 3, the first estimated torque output from the first torque estimating unit 21, and the output of the machine learning model storage unit 42. Machine learning model parameters are input, and a machine learning correction amount is calculated based on machine learning of the machine learning unit 411 using the feature amount and the machine learning model parameters. The second torque estimator 41 outputs the result obtained by adding the machine learning correction amount to the first estimated torque as the second estimated torque.

The machine learning unit 411 receives the feature quantity generated by the feature quantity generation unit 3 and the machine learning model parameters output from the machine learning model storage unit 42, and determines that the error of the second estimated torque with respect to the actual torque is the minimum. It is composed of a neural network that outputs the machine learning correction amount so that That is, the ideal value of the machine learning correction amount for the output matches the difference between the actual torque and the first estimated torque, and the difference between the past actual torque delayed by the input feature value and the first estimated torque , the machine learning unit 411 has an autoregressive term in the input/output relationship as shown in FIG.

FIG. 5 is a diagram showing autoregression terms in the configurations of the feature amount generation unit 3 and the machine learning model calculation unit 4 according to this embodiment. FIG. 5 shows a portion related to the autoregressive term extracted from FIG. Here, machine learning model parameters refer to the weight matrix and bias vector in the neural network. Also, as a type of neural network, use a neural network called a recurrent type such as a recurrent neural network, LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), which has a recursive structure in the input/output relationship of the intermediate layer. is also an effective means for improving the prediction accuracy for such time-series data. Also, in order to reduce the learning load and the amount of calculation, polynomial models, multiple regression models, support vector regression models, regression trees, random forests, multivariate adaptive regression spline models, etc. may be used instead of neural networks, In cases such as when mutual interference between axes can be ignored, the machine learning unit 411 may be applied independently for each axis.

However, general-purpose function approximation models such as neural networks used in the machine learning unit 411 are generally not suitable for approximating discontinuous functions. Therefore, when the frictional torque is approximated and corrected by the linear friction model of formula (2) in the first estimated torque, the difference between the actual torque and the first estimated torque causes a discontinuous jump, and these There is a possibility that the learning result of the machine learning model parameter and the calculation result of the machine learning correction amount of the machine learning unit 411 that includes the values in the input and output and form the autoregressive term will deteriorate significantly. Furthermore, in the actual drive machine, the stick-slip phenomenon, which occurs when adhesion and slip alternate, and the friction related to lubrication, which is represented by the Stribeck curve, are affected. Due to the strong nonlinearity, it is very difficult to give these models explicitly. Therefore, by using a friction model in which the vicinity of the speed 0 is continuously and smoothly connected like the nonlinear friction model shown in Equation (3), each value of the first estimated torque and the feature amount becomes continuous, and the above-mentioned Even for the friction component that is difficult to reproduce with the first estimated torque and the nonlinear friction model, it is possible to estimate with high accuracy by adding the machine learning correction amount. Furthermore, the feature amount is a vector that simultaneously includes the velocity for a certain period of time in the past, the output of the nonlinear friction model, and the like. Therefore, the machine learning unit 411 can also learn factors of estimation errors that change depending on the immediately preceding state, such as the hysteresis characteristics of friction, and correct them as machine learning correction amounts.

In this way, the second torque estimator 41 uses machine learning including an autoregressive term in the input/output relationship to calculate the machine learning correction amount from the feature amount and the machine learning model parameters. The second torque estimator 41 adds the calculated machine learning correction amount to the first estimated torque and outputs the second estimated torque. Further, the second torque estimator 41 may use machine learning using a neural network as machine learning including an autoregressive term in the input/output relationship. By using a neural network as a machine learning method, the motor control device 100 can learn and store a torque component, which is difficult to express on the equation of motion, as a nonlinear multi-degree-of-freedom model. In addition, the second torque estimator 41 calculates the difference between the actual torque and the first estimated torque as an autoregressive term in machine learning that includes an autoregressive term in the input/output relationship used to calculate the machine learning correction amount. may be used. A difference between the actual torque and the first estimated torque is the target value of the correction amount for the first estimated torque. Since the target value of the correction amount changes in time series with the operation of the motor of the target machine 10, the motor control device 100 uses the past target value of the correction amount when estimating the optimum correction amount for the next time. By providing an autoregressive term included in the input, it is possible to estimate with higher accuracy.

The machine learning model storage unit 42 inputs the feature amount and the difference between the actual torque and the first estimated torque when the target machine 10 is operated for a certain period after the physical model parameters are learned in the physical model storage unit 22 . As training data for the output, machine learning model parameters are learned so as to reduce the error of the machine learning correction amount with respect to the difference between the actual torque and the first estimated torque. That is, the machine learning model storage unit 42 learns machine learning model parameters by supervised learning such that the machine learning correction amount approaches the difference between the actual torque and the first estimated torque. The machine learning model storage unit 42 stores learned machine learning model parameters. The machine learning model parameters at the start of learning may be initialized with arbitrary constants. have. Therefore, in the machine learning unit 411, in addition to the uniform distribution and the normal normal distribution, the weight matrix is initialized with a probability distribution derived from the normal distribution such as the truncated normal distribution, the He normal distribution, and the Grorot normal distribution. is common.

The machine learning model storage unit 42 also uses the mini-batch gradient descent method and the error backpropagation method as learning methods, and updates the machine learning model parameters represented by the weight matrix and the bias vector for each batch. That is, the machine learning model storage unit 42 may use the error backpropagation method as a supervised learning method used when learning the machine learning model parameters. However, if a sufficient amount of teacher data cannot be prepared, the machine learning model storage unit 42 may use a normal gradient descent method, a stochastic gradient descent method, or the like instead of the mini-batch gradient descent method. In addition, in order to improve the generalization performance of the neural network in the machine learning unit 411, the machine learning model storage unit 42 has a dropout that randomly excludes neurons during learning, and monitors errors for each epoch to quickly terminate learning. It is also effective to use a technique such as early stopping. When the machine learning model storage unit 42 applies dropout during learning, it is necessary to apply the dropout probability to the weight matrix during inference of the machine learning correction amount. Therefore, such hyperparameters are also added as machine learning model parameters. and save.

The flow of learning the physical model parameters and the machine learning model parameters up to this point will be described using flowcharts. FIG. 6 is a flow chart showing the flow of learning of physical model parameters and machine learning model parameters in motor control device 100 according to the present embodiment. The physical model storage unit 22 sets the initial values of the physical model parameters of mass, center of gravity position, and inertia tensor of each link from the values in the specification table (step S1). The motor state observation unit 1 outputs time-series data of actual torque, position, speed, and acceleration, which are motor state signals of the target machine 10, as teacher data (step S2).

The first torque estimator 21 calculates a first estimated torque by dynamic calculation with respect to the position, velocity, and acceleration (step S3). The first torque estimator 21 adds a linear friction model term to the dynamics equation (step S4). The physical model storage unit 22 adds the Coulomb friction coefficient and the viscous friction coefficient to the physical model parameters (step S5). The physical model storage unit 22 identifies physical model parameters from the difference between the actual torque and the first estimated torque, using the dynamic equation transformed into a linear equation and the least-squares method (step S6). The physical model storage unit 22 uses the Coulomb friction coefficient and the viscous friction coefficient in the identified physical model parameters as initial values, identifies the nonlinear friction model by the Levenberg-Marquardt method, and adds each obtained coefficient to the physical model parameters ( step S7). The physical model storage unit 22 subtracts the first estimated torque and the value of the nonlinear friction model from the actual torque, and uses the dynamics equation transformed into a linear equation by removing the terms of the friction model and the least squares method. , the physical model parameters other than the friction coefficient are identified again (step S8). The first torque estimator 21 adds a nonlinear friction model term to the dynamics equation, and uses the learned physical model parameters to calculate the first estimated torque again (step S9).

The feature amount generator 3 receives the first estimated torque, the physical state variable, and the motor state signal as inputs, puts them together into a vector that also includes values for a certain period of time in the past, and outputs the vector as a feature amount (step S10). The machine learning model storage unit 42 initializes the weight matrix and the bias, which are machine learning model parameters, with a normal distribution and the bias with a constant (step S11). The machine learning model storage unit 42 sets hyperparameters such as early stopping and dropout as learning conditions, and adds these values to the machine learning model parameters (step S12). The second torque estimating unit 41 uses the input as a feature value and the output as an ideal value of the machine learning correction amount that is the difference between the actual torque and the first estimated torque, and the error is calculated according to the conditions at the time of learning in the machine learning model parameters. The weight matrix and bias of the machine learning model parameters are learned by the backpropagation method (step S13).

Here, for improving the generalization performance of the second estimated torque, as described above, the physical model parameters are learned in advance in the physical model storage unit 22, and the machine learning correction amount for the first estimated torque is autoregressed. As a term, learning of the machine learning model parameters in the machine learning model storage unit 42 greatly contributes. This is because the motor control device 100 distinguishes between the physical model parameters and the machine learning model parameters. This is because the absolute value of the machine learning correction amount, which is the output of the machine learning unit 411, becomes smaller. In other words, since the explicitly given torque estimation error component is not included in the machine learning correction amount, motor control device 100 can also reduce the number of parameters included in the machine learning model parameters. It is possible to prevent over-learning and increase the speed of learning in the unit 42, and to reduce teacher data used for learning.

Similarly, for the autoregression term related to the machine learning correction amount, the difference between the actual torque and the calculation result by the equation of motion using the initial values for the physical model parameters is learned by the machine learning model calculation unit 4 and used as the machine learning correction amount. Alternatively, compared to a configuration in which the machine learning model parameters are learned by directly using the second estimated torque as the output of the machine learning unit 411, the machine learning correction amount , the absolute value of becomes smaller, and the torque estimation error range by a neural network can be suppressed. Furthermore, as an advantage of using an autoregressive term, there are effects of cogging torque, torque ripple, and backlash in torque transmission between gears that depend on the motor characteristics. One of the advantages is that it is easy to learn estimation error factors that change over time. FIG. 7 shows the first estimated torque after learning the physical model parameters and the second estimated torque actually learning the machine learning model parameters from the difference between the actual torque and the first estimated torque. FIG. 7 shows the motor speed representing the operation of the target machine 10 controlled by the motor control device 100 according to the present embodiment, the first estimated torque estimated by the motor control device 100, and the FIG. 10 is a diagram showing a second estimated torque; In each item, the horizontal axis indicates time. As shown in FIG. 7, the second estimated torque is closer to the actual torque than the first estimated torque.

In addition, when a neural network is used for the machine learning unit 411, it is not clear which parameter included in the feature amount has an effect on the machine learning correction amount. Predicting change is very difficult. However, in the learning of the physical model parameters, each value of these physical properties can be directly confirmed in the present embodiment. It can be said that it is a composition.

As an example of abnormality detection using the physical model calculation unit 2 and the machine learning model calculation unit 4 that have performed learning, a method of mainly monitoring the difference between the actual torque and the second estimated torque can be considered. This is because the motor control device 100 monitors the difference between the actual torque and the second estimated torque, so that the observed values are concentrated around 0, and the absolute value can be regarded as the degree of abnormality of the actual torque. A specific abnormality detection method may be a simple method of detecting whether the absolute value of the difference between the actual torque and the second estimated torque exceeds a preset threshold. As a two-class classification problem of normal and abnormal with the difference between and the second estimated torque as an input, prepare labeled teacher data and use logistic regression, support vector machine, decision tree, neural network, etc. can be determined.

Motor control device 100 over-samples abnormal operation data acquired using SMOTE (Synthetic Minority Over-sampling TECHnique), ADASYN (ADAactive SYNthetic), etc., when it is difficult to collect operational data in abnormal conditions, which is teacher data. may be performed, or by unsupervised learning such as the K-means method or self-organizing map, the clusters may be classified into two clusters, normal and abnormal. In addition, in the motor control device 100, the number of classes and the number of clusters are not limited to two, normal and abnormal, and may be set in finer steps according to the degree of abnormality. Any degree of abnormality other than the absolute value of the difference from the estimated torque may be set.

Regarding the learning timing of the physical model parameters in the physical model storage unit 22 and the machine learning model parameters in the machine learning model storage unit 42, which are indicated by the dashed lines in FIG. It is desirable that the motor control device 100 be executed in an ideal state for comparison of the degree of deterioration, such as during test operation. However, industrial robots such as horizontal articulated robots often have force sensors and end effectors attached to their hands. , we need to relearn both the physical model parameters and the machine learning model parameters at each state.

However, in general, when a sensor, an end effector, etc. are attached, it takes time and effort to set the influence on these physical model parameters by a robot controller, etc., but in this embodiment, the physical model storage unit In 22, by learning them from the motor state signals when the target machine 10 is operated for a certain period of time, it is possible to automate this setting process as well. In addition, by setting the values of the physical model parameters correctly, there is also the advantage that the cycle time, trajectory deviation, pressing force at the time of collision, etc. are reduced. This technology can also be applied to increase the speed and accuracy of control, and the application of the above-described method is not limited to abnormality detection.

Next, the hardware configuration of the motor control device 100 will be described. In motor control device 100, motor state observation unit 1, physical model calculation unit 2, feature value generation unit 3, and machine learning model calculation unit 4 are implemented by processing circuits. The processing circuitry may be a processor and memory executing programs stored in the memory, or may be dedicated hardware.

FIG. 8 is a diagram showing an example in which a processing circuit included in motor control device 100 according to the present embodiment is configured by a processor and a memory. When the processing circuit is composed of processor 91 and memory 92, each function of the processing circuit of motor control device 100 is implemented by software, firmware, or a combination of software and firmware. Software or firmware is written as a program and stored in memory 92 . In the processing circuit, each function is realized by the processor 91 reading and executing the program stored in the memory 92 . That is, the processing circuitry includes a memory 92 for storing programs that result in the processing of the motor controller 100 being executed. It can also be said that these programs cause a computer to execute the procedures and methods of the motor control device 100 .

Here, the processor 91 may be a CPU (Central Processing Unit), a processing device, an arithmetic device, a microprocessor, a microcomputer, or a DSP (Digital Signal Processor). The memory 92 includes non-volatile or volatile memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable ROM), EEPROM (Electrically EPROM). semiconductor memory, magnetic disk, flexible disk, optical disk, compact disk, mini disk, or DVD (Digital Versatile Disc).

FIG. 9 is a diagram showing an example in which a processing circuit included in motor control device 100 according to the present embodiment is configured with dedicated hardware. When the processing circuit is composed of dedicated hardware, the processing circuit 93 shown in FIG. 9 is, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit) An FPGA (Field Programmable Gate Array) or a combination thereof is applicable. Each function of the motor control device 100 may be realized by the processing circuit 93 for each function, or the functions may be collectively realized by the processing circuit 93 .

It should be noted that each function of the motor control device 100 may be partly realized by dedicated hardware and partly realized by software or firmware. Thus, the processing circuitry may implement each of the functions described above through dedicated hardware, software, firmware, or a combination thereof.

As described above, in the present embodiment, the model of the motor control device 100 is a physical model for estimating the torque using the information of the physical phenomena explicitly given on the mechanical properties of the target machine 10 and the equation of motion. A combination of a model calculation unit 2 and a machine learning model calculation unit 4 that includes a correction amount for the estimated torque of the physical model calculation unit 2 as an autoregressive term and estimates a highly nonlinear torque component that is difficult to express on the equation of motion. expressed in By sequentially learning these, the motor control device 100 can quickly learn a model of the target machine 10 with high torque estimation accuracy and generalization performance with respect to the actual torque from a small amount of operation data. In addition, the motor control device 100 can detect even minor abnormalities by comparing the actual torque and the estimated torque value, that is, by comparing the actual machine and the learned model.

That is, the motor control device 100 sequentially learns the physical model parameters and the machine learning model parameters to obtain the mechanical properties of the target machine 10, the torque component due to the physical phenomenon explicitly given in the equation of motion, and the equation of motion. It can be learned and estimated by distinguishing it from the highly nonlinear torque component, which is difficult to express above. As a result, the motor control device 100 shortens the time required for the entire learning, and since the absolute value of the machine learning correction amount becomes smaller due to the learning of the physical model parameters, the generalization performance of the second estimated torque with respect to the actual torque is improved. can be made In addition, the motor control device 100 allows the second torque estimator 41 to efficiently learn the correlation between the feature quantities in the time axis direction using the autoregressive term of the input/output relationship, thereby estimating the second estimated torque. Accuracy can be improved.

In the present embodiment, a horizontal multi-joint robot is controlled. However, the motor control device 100 can be applied regardless of the number of machine axes, the linear motion of the joints, and the rotation mechanism. It can also be applied to articulated robots, NC machine tools, mounters, and the like.

The configuration shown in the above embodiment shows an example of the contents of the present invention, and it is possible to combine it with another known technology, and one configuration can be used without departing from the scope of the present invention. It is also possible to omit or change the part.

1 motor state observation unit 2 physical model calculation unit 3 feature amount generation unit 4 machine learning model calculation unit 10 target machine 11l first link 11j first axis 12l second link 12j second axis 13l Third link 13j Third axis 14l Fourth link 14j Fourth axis 15e Hand part 21 First torque estimation part 22 Physical model storage part 41 Second torque estimation part 42 Machine learning model storage part, 100 motor control device, 411 machine learning part.

Claims (10)

A motor control device for controlling a motor that drives a target machine,
a motor state observation unit that observes the state of the motor and outputs an observation result as a motor state signal;
Based on the motor state signal and the physical model parameters of the target machine, a first estimated torque, which is an estimated value of the torque of the motor, and a physical state variable, which is a component of the first estimated torque, are calculated using an equation of motion. a first torque estimator that outputs;
a physical model storage unit that learns and stores the physical model parameters so as to reduce the difference between the actual torque included in the motor state signal and the first estimated torque;
a feature quantity generator that generates and outputs a feature quantity from at least one of the first estimated torque, the physical state variable, and the motor state signal;
A machine learning correction amount is calculated from the feature amount and the machine learning model parameters using machine learning including an autoregressive term in the input/output relationship, and the calculated machine learning correction amount is added to the first estimated torque to obtain a second a second torque estimator that outputs two estimated torques;
Machine learning model storage for learning the machine learning model parameters by supervised learning so that the machine learning correction amount approaches the difference between the actual torque and the first estimated torque, and storing the learned machine learning model parameters. Department and
A motor control device comprising: The second torque estimating unit uses machine learning using a neural network as machine learning including an autoregressive term in the input/output relationship,
The machine learning model storage unit uses an error backpropagation method as a supervised learning method used when learning the machine learning model parameters,
2. The motor control device according to claim 1, wherein: In machine learning including an autoregressive term in the input/output relationship used to calculate the machine learning correction amount, the second torque estimator uses the difference between the actual torque and the first estimated torque as an autoregressive term. using
3. The motor control device according to claim 2, wherein: The feature amount generating unit includes a storage device capable of storing the first estimated torque, the physical state variable, and the motor state signal, and the first estimated torque stored in the storage device and the physical outputting state variables and features extracted from the motor state signal as features;
4. The motor control device according to any one of claims 1 to 3, characterized in that: The physical model parameters include a mass in a linkage of the target machine, a center of gravity position in the linkage of the target machine, and an inertia tensor in the linkage of the target machine.
5. The motor control device according to any one of claims 1 to 4, characterized in that: The first torque estimating unit uses a motion equation including a continuous and smooth friction model that outputs an estimated value of the frictional component of the torque with the speed of the motor determined by the motor state signal as an input,
6. The motor control device according to any one of claims 1 to 5, characterized in that: the physical model storage unit learns and stores coefficients of the friction model as the physical model parameters;
7. The motor control device according to claim 6, wherein: wherein the physical state variables include the friction model;
8. The motor control device according to claim 7, characterized in that: The physical model storage unit alternately estimates coefficients of the friction model and other physical model parameters in learning the physical model parameters.
9. The motor control device according to claim 8, wherein: The physical model storage unit uses a least-squares method, a regularized least-squares method, a nonlinear least-squares method, or a Kalman filter in learning the physical model parameters.
10. The motor control device according to claim 9, characterized in that:

Images